Tail Wagging Its Dog (Sick Sigma – part 2)

by Dr A. D. Burns

Most people in a company using Six Sigma know how to determine a "sigma level". Simply count the number of defects per million opportunities then read from a modified normal distribution table to determine the "sigma level". If the "sigma level" is 6, it is claimed that we have a "world class" process or organization¹. Whilst most people may understand the procedure, few understand what it means or it’s implications. Fewer people understand the meaning of "sigma".

Sigma or s is a Greek symbol representing the standard deviation of a population. For the technically minded, the standard deviation is defined as a function of the sum of the squares of deviations of values from the average value. Sigma is hence a measure of the dispersion of a population. In simple terms, it is a measure of the overall shape of the distribution of all the values from which data is taken. If sigma is large, the distribution is fat. That is, there is a broad spread of data about an average value. If sigma is small, data is tightly grouped around the average.

There are a variety of ways to estimate sigma. We could estimate sigma by carrying out calculations on all the data available. This would allow us to determine the spread of data for the samples and hence estimate the spread for the entire population. The Six Sigma methodology has chosen a different approach. Rather than examine all the data, Six Sigma only counts the number of data points or defects, at the extreme tail, then estimates sigma from a table. However, this procedure is analogous to trying to estimate the shape of a dog by looking at the extreme tip of its tail !

Whilst we may accurately count the number of hairs in the tip of the tail of the dog, we cannot tell whether the dog is shaped like a Dachshund or a Doberman. It is the same with processes. Because of the tiny amount of data in the extreme tip of the tail, the slightest variation from normality as assumed in Six Sigma Tables, can make our estimate of sigma wildly inaccurate. In fact there is just as much variety in process distributions as there is in dogs. We can no more assume that a process is normally distributed than we can assume that all dogs are shaped like Poodles. Normal distributions assume a perfect theoretical randomness. Normal distributions are rare in the real world. Real distributions may have many forms. They may be quite asymmetric or skewed as is common with time related processes such as in call centres. They may be truncated; bimodal; plateau; square and many other variations. We can get much more information about the shape of a distribution, simply by using the available data to plot a histogram. A histogram gives us information about the process. For example, a bimodal (two peaked) distribution often indicates data from two different sources.

The estimate of sigma from Six Sigma tables is placed further in error by the anomalous 1.5 sigma drifting process average, as discussed in detail in "Sick Sigma. Question authority and you'll win." ³. Six Sigma theory suggests that it is impossible to keep any process on target and average values will drift by the huge amount of 1.5 sigma "long term" ("long term" originally being one day according to Six Sigma theory).

Do we need an estimate of sigma ? An estimate of sigma is useful because it enables us to create process control charts. A wonderful feature of control charts is that they do not depend on normal distributions. They work with any type of distribution. Whilst based on probability theory, control chart control limits are not probability limits⁴. They provide signals as to when process variation is likely to be caused by other than chance and when it's appropriate to take action. Another lovely feature of control charts is that the estimate of sigma does not require complex calculations. Sigma is based on averages and ranges taken from the data. Estimates of sigma from Six Sigma Tables should never be used for control charts. Users might be surprised to find just how different estimates of sigma are from Six Sigma Tables, compared to those from control charts. Similarly, control charts should always use three sigma control limits, never six sigma, two sigma or other numbers. Control limits are derived from the process behaviour. They cannot be set by users. Control charts are "the voice of the process".

A simple example of a process with which most people should be familiar, illustrates the use of control charts. My daily run along the beach run takes 30 minutes. I plot my run times on an XmR control chart and find a sigma of 60 seconds. I'm fit and healthy. My control chart shows that my times are "in control". I can guarantee that my run will take between 27 and 33 minutes. That is, my times are between 3 sigma control limits. One day I twist my knee. A special cause is now present. My time that day is 45 minutes. My control chart becomes "out of control". My run times are now unpredictable. It's impossible to say that tomorrow my knee will cause an even slower time, or that it will improve. Despite this, Six Sigma suggests that I have only a 0.00034% of not achieving a time between 24 and 36 minutes !

Counting defects tells us very little about the process. It only indicates that the process has moved out of specification. The specification limits may be set anywhere. The simplest way for a consultant to guarantee to cut defects is simply to broaden the specification limits. Unlike the fixed three sigma control limits, specification limits may be set at six sigma, or any other value. The broader the specification limits, the fewer defects can be expected, provided the process is "in-control". If the process is not running within three sigma control limits, it will be unpredictable and an unpredictable level of defects may be produced, no matter where specification limits have been set, even at six sigma. Only a predictable process can be "capable" of producing product or service within specification.

Rather than broadening specification limits in order to reduce defects, we should aim to bring processes into control and to reduce variation. As variation is reduced, control limits are tightened and specification limits may be narrowed whilst still being assured of low defect rates.

Consider the impact on the customer. On one hand we could say "Yes, we have Six Sigma processes. Our process average floats up and down by 1.5 sigma which means our processes will be wildly out of control about 13-14% of the time², or about one day in seven. However because of our Six Sigma program we claim only .00034% defective. Of course we have had to widen our specification limits in order to claim that." Conversely "Our processes are always in-control. This allows us to predict defect free product with tighter specification limits than any of our competition." Tighter specifications, with in-control processes and hence predictable outcomes, give happy customers. World class quality can only mean on target with minimum variation.

Wherever possible, process data should be collected by taking measurements. Variable data contains more information about the process than the count of values that have exceeded specification. However in some instances this is not possible, for example in counting blemishes in the paintwork of an object. Six Sigma Tables use DPMO or defects per million opportunities. Whilst blemishes may be counted, "opportunities" in a surface area is meaningless. Furthermore, blemishes follow a Poisson distribution rather than a normal one as assumed by Six Sigma. Where only attribute data is available, data should be plotted using the appropriate p, pn, c, u or XmR control charts.

We have looked at the problems in counting defects and using defect counts to estimate sigma. In the previous paper "Sick Sigma"³ we looked in detail at Six Sigma’s erroneously unavoidable 1.5 sigma floating average, why six sigmas are not better than three, the excessive number of tools and black belt elitism. On a final note, many suggest that Six Sigma is unique because of its DMAIC (Define Measure Analyse Improve Control) and DMADV (Define Measure Analyse Design Verify) methodologies. Previously the most common methodology was PDCA (Plan Do Check Act) although many major companies had their own 5 step, 6 step, 7 step and 8 step improvement methodologies. These methodologies have a great deal in common with "Induction; Deduction; Testing; Confirmation" or what is known as "The Scientific Method". This may be re-stated as: "Define the problem"; "Formulate hypotheses"; "Collect data"; "Analyse data"; "Statement of conclusions regarding confirmation or invalidation of the hypotheses". The origins of The Scientific Method have been attributed to Sir Francis Bacon (inductive reasoning) and René Descartes (deductive reasoning) in 17 century. There is little to indicate that one methodology is significantly better than another or that The Scientific Method should be forgotten.

In summary, counting defects at the extreme tail of a process distribution tells us almost nothing about the process. It is a very poor way to estimate the dispersion parameter, sigma. Rather than using defects to estimate sigma, histograms should be used to indicate the shape of the distribution and control charts should be used to ensure the process is in-control and therefore predictable.

References