Learn how to measure and improve process capability for non-normal data, and how to choose the best method for your data and analysis.

Normal and Non-Normal Data I’ve often encountered non normal data more often than normal data in service organizations. Before certain analytical tools are used on your data, you need to determine if your data is normal or non-normal. If you use certain tools under the assumption you have normal data and it’s not, then your results will be skewed. Matt Rebeiro - Challenge the Status Quo #BusinessProcessImorovement #LeanSixSigma #data #statistics

It is important to remember that not all processes produce data that follows a normal distribution, even though the normality assumption is the basis for many traditional process improvement metrics. Processes with inherent skewness, outliers, or multiple peaks can challenge the use of metrics such as Cp or Cpk, leading to misleading or inaccurate assessments of process capability. Relying solely on these metrics in these instances could lead to incorrect conclusions about process quality, subsequently guiding efforts for improvement in the wrong direction. To avoid this, it is crucial to first conduct tests for data normality, such as the Shapiro-Wilk test, Anderson-Darling test, before committing to specific process improvement metrics.

Data transformation is appropriate where n>100 otherwise non transformative method should use. Probability graph method or Pearson curve method could be choice of non transformative method.

How do you set realistic and meaningful process improvement goals for non-normal data?

Process capability is a measure of how well a process can meet customer specifications or expectations. It is often used to set improvement goals and monitor progress in business process improvement (BPI) projects. But what if your process data is not normally distributed, meaning it does not follow the familiar bell-shaped curve? How do you calculate and interpret process capability for non-normal data? In this article, we will explain the basics of process capability, the challenges of non-normal data, and some methods and tools to deal with them.

1 Why process capability matters

Process capability is a key indicator of process performance and quality. It compares the natural variation of a process with the tolerance limits or specifications defined by the customer or the market. A high process capability means that the process can consistently produce outputs that meet or exceed the requirements, while a low process capability means that the process is prone to defects, errors, or rework. By measuring and improving process capability, you can reduce waste, increase customer satisfaction, and achieve your BPI goals.

Add your perspective

Mohammed Abdin

BIE, Data Engineering @Amazon | AWS Cloud | Ex. Meta, Tesla
(edited)
Report contribution
Process capability is a key indicator of process performance and quality. It compares the natural variation of a process with the tolerance limits or specifications defined by the customer or the market. A high process capability means that the process can consistently produce outputs that meet or exceed the requirements, while a low process capability means that the process is prone to defects, errors, or rework. By measuring and improving process capability, you can reduce waste, increase customer satisfaction, and achieve your BPI goals.

Like
Matt Rebeiro

Strategic Advisor | Startup | SaaS | GTM | AI Automation Strategy | Founder of the Tech Retriever | New York | San Francisco
Report contribution
Normal and Non-Normal Data I’ve often encountered non normal data more often than normal data in service organizations. Before certain analytical tools are used on your data, you need to determine if your data is normal or non-normal. If you use certain tools under the assumption you have normal data and it’s not, then your results will be skewed. Matt Rebeiro - Challenge the Status Quo #BusinessProcessImorovement #LeanSixSigma #data #statistics

Like
Amanpreet Singh Banga

Sr. Manager Operations Excellence | Process Improvement & Continuous Improvement Expert | Certified Lean Six Sigma Black Belt | Management Consultant | Increase Profit & Revenue, Cut Costs & Boosts Customer Experience
Report contribution
Setting realistic process improvement goals for non-normal data involves understanding data characteristics, defining clear objectives, and focusing on key metrics. Consider the context of the process and set incremental improvement targets. Implement robust monitoring mechanisms to track progress and adjust strategies as needed. Celebrate achievements to maintain motivation and momentum. By following these steps, organizations can drive continuous improvement and enhance overall performance.

Like

2 The normality assumption

Many process capability metrics, such as Cp, Cpk, Pp, and Ppk, are based on the assumption that the process data follows a normal distribution. A normal distribution is symmetrical, with most of the data clustered around the mean and fewer data points at the extremes. It is often described by two parameters: the mean, which indicates the average or central value, and the standard deviation, which indicates the spread or variation. The normal distribution has some useful properties, such as the 68-95-99.7 rule, which states that about 68%, 95%, and 99.7% of the data fall within one, two, and three standard deviations from the mean, respectively.

Add your perspective

Rahul Narain Saxena

Founder – TYG Consulting | SAP Solution Architect | Digital Transformation Expert | Simplifying SAP for Career & Business Growth | Mentor & Guide
Report contribution
It is important to remember that not all processes produce data that follows a normal distribution, even though the normality assumption is the basis for many traditional process improvement metrics. Processes with inherent skewness, outliers, or multiple peaks can challenge the use of metrics such as Cp or Cpk, leading to misleading or inaccurate assessments of process capability. Relying solely on these metrics in these instances could lead to incorrect conclusions about process quality, subsequently guiding efforts for improvement in the wrong direction. To avoid this, it is crucial to first conduct tests for data normality, such as the Shapiro-Wilk test, Anderson-Darling test, before committing to specific process improvement metrics.

Like

3 The challenge of non-normal data

However, not all processes produce normally distributed data. Some processes may have skewed, bimodal, or multimodal distributions, meaning that the data is not symmetrical or has more than one peak. Some examples of non-normal data are cycle times, defect counts, customer ratings, or service times. Non-normal data can pose a challenge for process capability analysis, because the normality assumption may not hold, and the standard metrics may not be valid or meaningful. For instance, if the data is skewed, the mean and the standard deviation may not adequately represent the central tendency and the variation of the data, and the 68-95-99.7 rule may not apply.

Add your perspective

4 How to deal with non-normal data

When dealing with non-normal data for process capability analysis, there are several methods and tools available. Transforming the data by applying a mathematical function, such as a logarithm or a square root, can reduce skewness or multimodality and make the standard metrics more applicable. However, this also means that the transformed data may not be easily interpretable or comparable with the original data or the specifications. Alternatively, non-parametric methods, such as the median, the range, the interquartile range, or the percentiles, can be used to describe the central location and the variation of the data without using the mean and the standard deviation. These methods may not be as sensitive or powerful as the parametric methods, and they may not have a clear link to the specifications or the customer expectations. Lastly, distribution-specific methods, such as the Weibull, the lognormal, the gamma, or the Poisson distribution, can be used to calculate distribution-specific metrics, such as the Z-score, the Cpm, or the DPMO. These methods can help to account for the shape and the characteristics of the data, but they may require more data and more expertise to identify and fit the appropriate distribution, and they may not be widely available or supported by the software or the standards.

Add your perspective

Tejas Shah

Validation Consultant at Quascenta Pte. Ltd. (Views are My Own)
Report contribution
Data transformation is appropriate where n>100 otherwise non transformative method should use. Probability graph method or Pearson curve method could be choice of non transformative method.

Like

5 How to choose the best method

When it comes to dealing with non-normal data for process capability analysis, there is no definitive answer as to which method is best. The choice depends on several factors, such as the type and source of the data, the purpose and scope of the analysis, the availability and reliability of the specifications, the software and tools used, and the level of knowledge and experience of the analyst. However, some general guidelines can be followed. It is important to check the normality of the data before applying any method, which can be done through graphical methods such as histograms, box plots, or normal probability plots, or statistical tests such as the Shapiro-Wilk test, the Kolmogorov-Smirnov test, or the Anderson-Darling test. If the data is reasonably normal, the standard metrics can be used without any transformation or adjustment. Additionally, it is important to compare and validate the results of different methods, which can be done through graphical methods such as capability plots, or numerical methods such as confidence intervals. Furthermore, it is essential to communicate and document the method and the assumptions used for the process capability analysis, especially if the data is non-normal and the method is non-standard. This can help to ensure the transparency and credibility of the analysis, and to avoid any confusion or misunderstanding among the stakeholders.

Add your perspective

How do you set realistic and meaningful process improvement goals for non-normal data?

1

2

3

4

5

1 Why process capability matters

2 The normality assumption

3 The challenge of non-normal data

4 How to deal with non-normal data

5 How to choose the best method

Business Process Improvement

Rate this article

Thanks for your feedback

More articles on Business Process Improvement