Let’s cut to the chase, everyone knows about user testing, but no one thinks enough about the impact it has on a Custom Android Development timeline.
When I suggest this to clients they initially unintentionally ignore the suggestion until the benefit becomes apparent during prototype testing. Rarely do companies realize how important this is until problems come to light that should have been caught earlier on.
By default doing a proper trial production and the beta test takes time or inevitably happens during mass production because not enough time is budgeted during development, which could lead to problems or even more time/money spent rectifying.
Beta testing, for many time-pressed and inexperienced companies, is No Man’s Land where dreams of product success stories or nightmares of ‘should-have-done’ are quickly and eternally created. You all spend time conceptualizing how your custom Android device will work, how it will look, how many you’ll sell, and how much fun it will be to work with Hatch, but…stop now before you get slapped…you must also think about USER TESTING!
Here’s the deal. During your custom Android development process we (‘we’ being Hatch and you) will talk for hours, days…no, months about little changes to minor and major details of your product, so much that you start dreaming about ridiculous nonsense like data logs and pogo pins.
Finally, after beating your head repeatedly against a ceramic toilet seat for months, you get a working prototype and it actually works. How amazing! Your little baby actually has eyes (camera), ears (mic), and a mouth (speaker).
After receiving that working prototype you’ll feel like ‘wow, I must have just gone through digital pregnancy, feels so good to be done with this shit,’ yet you are so wrong! In reality, this is where one layer of work ends and another layer of complexity begins. Most products require revisions following usage in real life scenarios by actual users.
After thoroughly testing the prototype, your CNC baby, that just popped out of Shenzhen and arrived at your doorstep it’s NOT time to head to mass production and if you think otherwise it’s time for your cold shower to wake TFU. After your prototype seems perfect rest assured that you’re wrong, there are still problems with it.
If nothing else it will have a problem passing certification testing, as discussed in last month’s article, because most engineers, especially those in Shenzhen, don’t make products that pass certification testing on the first try.
Ok, let’s call that minor detail as failed certification testing won’t necessarily mean your product will get returned, but beyond that, it’s almost guaranteed that your device has usability issues that you weren’t able to catch while testing the prototype.
These problems can, and usually do, include issues with the firmware that you never saw before. Sometimes, although much less frequently, there are aspects of the hardware (besides issues related to certification) that negatively affect usability which was not detected in prototype testing and must change. Developing a custom product is an exercise in problem-solving.
On the same note, you must expect the pains associated with resolving those problems. The question is never about facing problems because that’s a given, the question is finding and fixing them in a controlled environment before their presence can negatively affect your brand or financial health.
The best way to find these problems is by using beta testers. This means giving the product to a diverse base of 50-200 end users for real-life testing. Of course, these testers must know that they’re guinea pigs so they don’t squeal too loudly when frustrated by the inevitable problems they’re going to face.
Because finding problems during beta user testing happens so often it’s a part of the development process that must be included in all development schedules. Going from prototype straight to mass production without user testing is like sky diving without first checking the parachute. There are variations to how a development schedule can look depending on the time pressure and risk threshold of the client. Highlighting these different scenarios is a key part of this article.
Before going into each of the different scenarios here are important details related to user testing from a manufacturing standpoint. Beta test devices conveniently come from what we in the manufacturing world call ‘Trial Production’, a low volume production run of 50-250 pcs.
Trial production happens after prototypes have been approved and after mass production tooling has finished and, by definition, before mass production. The purpose of trial production is to replicate the mass production process with a low volume of devices. What user testing is to a brand, trial production is to a manufacturer.
Trial production allows manufacturers to find ways to improve the assembly process and, ultimately, the product. This could mean:
- Making improvements to the mold (the ‘mold’ is a heavy piece of steel used for making the Android phone or tablet case in a process called plastic injection molding),
- Making jigs or other devices for use on the assembly line to improve manufacturing,
- Testing and optimizing the assembly standard operating procedures (SOP)
- Testing the end product. Often important oversights are caught during trial production. Finding problems during trial production, very much like during beta user testing, reduces the risk of finding problems during mass production.
With this in mind please note these key scenarios and different approaches:
Scenario 1 – Customer in a rush to receive product and trying to save money.
This is a high-risk scenario. In this scenario, at the point of trial production, the manufacturer would order all the materials for the mass production order. This means ordering all the PCBAs (PCBA is PCB board with all the components on it), which, for Android devices, takes 30-35 days to produce, batteries, screens, touch panels, etc.
The reason for ordering all the parts in advance is to avoid the duplicate lead time of having to order once for trial production and then again for mass production. Also buying all at once saves cost since there are extra fees associated with purchasing in small quantity. The obvious risk with ordering all parts in advance is that if there’s a problem with the electronics design a lot of those parts may go to waste.
Scenario 2 – The customer has budgeted time and cost for trial production.
This is the more conservative and ideal scenario where the customer allocates 6 weeks for preparing materials, which happens in parallel with making the mold anyway and doing the trial production. In this scenario, only the amount of parts needed for trial production are ordered.
This way if any hardware problems are found during trial production (or during beta user testing) the customer hasn’t already paid for a mass production quantity of parts that may can’t be used.
Scenario 3 – A middle ground between scenarios 1 and 2, but still more risky than scenario 2.
In this scenario the customer orders all the long lead time parts before trial production, such as PCB (and doesn’t solder the components on the board), battery, wall charger, and whatever else takes a while, but doesn’t order the quicker lead time parts (minimizing the amount of cash outlay).
Not putting the components on the PCB is an important detail because if a problem is found with the PCB design then the loss is limited to the cost of the empty PCBs, and possibly not even if only a value change is required for a (some) component(s).
If the components are already put on the PCB then the cost implication is much higher as Android CPUs and memory are major cost drivers. Also applying components to the PCBs only takes a few days. In this scenario the financial loss is limited and the lead time before mass production can be reduced.
So when you’re ready to do your next custom Android development project remember to budget enough time and cost as in scenario 2, if possible, for proper trial production. If you ever get in a jam trying to follow the concepts outlined in scenario 3.
