Limit Samples, AQL Tables, and Manufacturing Variance, oh my!
These are some of my very favorite topics, but not everyone is keen to wrap their head around what they are and their importance in
mass production. My goal, today, is to share with you this epiphany.
Mass production is obvious, right? You decide what you’re making and then the factory perfectly replicates it on an assembly production line. It’s a relatively inexpensive way to get 30,000 or more of your groovy, unique item created fairly quickly for all your fans. Except for one thing – it’s a bit more complicated than it sounds.
What Is Mass Production?
Let’s start with a clearer understanding of mass production. You are correct that at its core, mass production is the uniform manufacture of large quantities of a product using assembly lines or perhaps, automated technology. More formally, it’s an efficient production methodology designed to achieve high volume with detailed organization of material flow, careful control of quality standards, use and control of safe materials and division of labor. The key here is the reference to quality standards. Despite all the standardized processes put in place during mass production, the end products Can. Have. Variation. (Yep, I added that fancy punctuation on purpose.)
Variation, my friends, is the crux of today’s blog. How to control deviation in manufacturing to consistently achieve (nearly) identical products that match your original vision. Product defects (or shall we say, “imperfections”), large or small, are an inevitable consequence of mass manufacturing. No factory is perfect, no manufacturing condition is perfect, and in the world of hand painting and hand assembly, no human is perfect. The challenge is being able to foresee potential issues to prevent them and to monitor production to catch issues in progress so you can course correct on the spot.
Quality is a very over-used word, so it’s easily glossed over as a basic requirement for all businesses. Yeah, yeah, we all know quality is important. But, it’s MISSION CRITICAL (there goes my crazy punctuation again) in manufacturing. Quality Assurance plans ensure that the quality standards pre-set for your product are met throughout the entire production process. As you might imagine, it’s far more effective to plan for quality in advance than simply try to inspect for quality later.
The Golden Sample is the Benchmark
Therefore, it makes sense to start with an example of your product that is considered the “golden sample” – one that has been approved for factory replication on all specifications – size, materials, deco, functionality, weight, etc. This physical piece will be referenced throughout all stages of production as the “ideal unit” to duplicate on a large scale. Having a perfect point of reference helps confirm the factory is aligned with your definition of quality (there’s that word, again), up front.
For sake of discussion, let’s say we’re about to make a statue of Guan Yu for the Total War: Three Kingdoms Collector’s Edition Franchise (actually, we just finished this awesome project for Sega!). We begin with a client-approved version of the statue that is pre-made from a “sculpted” 3D file and hand-painted. Like this (feel free to zoom in for more detail – it’s pretty amazing, if I do say so myself!):
What could go wrong?
You’ve given the factory your Golden Sample, so now all they need to do is make it!
Well, yes, but…
I’m guessing that many of your peers bestowed a lot of love and attention to creating and then approving all the fine details of your golden piece of perfection. However, when factory production ramps up on a large scale and 400 painters are ready to make a bit of magic, some of these details can get interpreted a little differently, especially when there are multiple steps, like deco, that are often done repetitively by hand. This is simply the nature of mass production. To reinforce the level of attention to detail needed by factory workers, we step back and ask ourselves, “What could go wrong?”
To really get a bit nerdy, we call this process an Idea Planet FMEA – “Failure Mode Effect Analysis.” It’s used to anticipate and evaluate everything that could possibly happen and then put process controls in place to help mitigate these factors.
Areas of missed paint or stain
Globs of too much paint – or runs
Bubbles or pock marks that appear from humid conditions in the factory
Scratches from handling
You get the idea.
Then, when we move into pre-production and pilot run off the tools, everyone is on high alert for any additional anomalies not present until you are actually in a production environment.
Setting Limit Samples
Here’s where limit samples come into play. A limit sample identifies a potential quality deviation that could arise during the production process and the level of tolerance for acceptance or rejection of the item based on that deviation. Obviously, you don’t want any quality deviations, but limit samples should be set for anticipated issues that cannot be fully pre-empted by other countermeasure attempts in the production process.
Better buckle up – we’re about to get a bit more granular. Let’s say we’re talking about the finish on our Guan Yu statue. It’s supposed
to have a nice sheen all over. If certain areas have less sheen, the overall look will not be consistent. So, we visually define and limit (through pictures and samples) the size and locations of acceptable “missed varnish” (areas with a matte finish), knowing that this deviation is likely to happen. Then this limit sample is used during inspections to cull unacceptable product – or to call attention to corrective actions if the issue is occurring too often.
These deviations in quality are typically segmented into four categories:
Let’s talk briefly about each.
Acceptable deviations are those that are expected as part of a normal production process – but the extent of what is allowed is clearly defined. For instance, it could be the exact trim line on a printed instruction sheet, or the length of thread left on a sewn plushie. Acceptable measurements away from the precise target shown in the golden sample are specified and agreed to by all parties in advance.
Minor deviations are considered less than desirable to the end user but do not adversely affect the product’s marketability, sale-ability, form, fit or function and as such, are unlikely to result in a return to the seller. They are also clearly defined and measurable, but only a limited quantity is allowed to pass through inspections (more on this shortly). An example might be a small press mark left from a die-cutting machine.
Major deviations are a step up from minor. Majors cover any condition that adversely affects the product’s marketability and sale-ability and/or its form, fit or function, thereby increasing the likelihood the end user will return the item to the seller. This could be anything from a bent piece to missing deco paint to very large press marks from said die-cutting machine. For inspections, there are stringent measurements and more highly restricted pass-through quantities.
Critical deviations are always considered an inspection failure and are ejected from the production line based upon the age grading that has been established at the product development stage. That’s because critical deviations are connected directly to the safety of a product and could cause potential injury or harm to the end user or others in the vicinity of the product’s use. For instance, an unexpectedly sharp metal piece. Or, a small part on a product marketed to children that could be inadvertently swallowed.
Just to be clear, there ae NO acceptable limit samples for a safety issue. This is simply an area that has no deviation and no limit or tolerance at any level.
To summarize (such a teacher thing to say…), all mass production quality deviations fall into these categories and have clearly defined, preferably measurable, pre-set “limits” as to what is acceptable and what is not. Implementation occurs during in-process inspections where trained quality control auditors pull samples from the production line and make a pass/fail assessment. Here’s how it works (warning, there’s a bit of math involved, so don’t let that freak you out).
Acceptable Quality Limit (AQL)
It makes sense to acknowledge that 100% inspection does not guarantee 100% compliance and would be way too time consuming and costly. So, the question that may pop in your head is, “How many samples do I need to inspect to get a true read on quality?” My friend, the answer is AQL (Acceptable Quality Limit) Sampling. AQL is an international industry methodology that outlines statistical sample sizes to confidently indicate whether the products in a production lot are meeting your quality specifications & tolerance level for deviations.
AQL tables determine the sampling size based on your order quantity and quality tolerance level, which have pre-set thresholds of I, II or III to allow for product subjectivity (for example, less deviation tolerance for toys or medical products). The standard AQL sampling plan used for most product is Level II.
There are 2 basic tables to direct you to the proper sample size. The first table indicates which code letter to reference for the second table.
AQL Table 1
So, if we were producing 450,000 Guan Yu statues and had a Tolerance Level of II, we would refer to Line P in the second table for our sample size and acceptable number of deviations. Or, if our production quantity was 175 with a Tolerance of I, we’d use Line E instead. Clear as mud?
The second table can seem intimidating at first – across the top is a range of percentages that indicate the specific acceptable level of deviation for the full production quantity. Idea Planet, for instance, typically follows an AQL of 1.5, which means that no more than 1.5%, on average, of the full production quantity may have an acceptable quality deviation. If we refer to Line P, as noted above, under the 1.5 AQL level, then we’d need to pull 800 random samples for our inspection – and of that 800, no more than 21 can have major deviations (per limit samples) for the inspection to pass. If 22 or more have major deviations, then the batch fails inspection.
AQL Table 2
Did that make your head spin? Let’s do one more. For Line E from our earlier example, we’d need to pull 13 random samples – and there are no acceptable major deviations (follow the arrow up to Line D). If even 1 is found, the batch is rejected. By the way, since critical deviations are a safety issue, it’s common to not allow any in a random sample – even one would mean rejection of the lot. That lot would need to be quarantined, reinserted, and reworked before reentering the routine inspection process.
Without going deeper down the rabbit hole, suffice it to say that AQL Sampling, using pre-set and agreed upon Limit Samples, is an important inspection process during mass production to ensure you actually get the quality (now used 22 times in this blog!) you expect.
I think that’s about my ‘limit’ for today’s blog – get it? Ha!
Until next time…