The Case of the "Unsuccessful" Experiment

Download a PDF copy.

Because of the sensitive and confidential nature of some commercial aerospace programs, the client’s identity and product details are not provided in this case.

This case involves the application of Design of Experiments (DOE) methodology to improve throughput in a Computer Numerical Control (CNC) machine. The customer manufactures aircraft components that utilize combinations of premium materials, fabrics, metals, and composites. They routinely develop, produce, and deliver new products that meet rigorous aerospace requirements and specifications.

This project also demonstrates Quality Support Group’s ability to teach and help clients to apply statistical methods to improve processes. It also demonstrates QSG’s expertise in delivering training in a format that can be quickly applied to clients’ real problems and processes.

The client was interested in having a number of their engineers and professionals trained in process improvement, and Quality Support Group asked me to handle the project. The training took the form of my Process Improvement Certificate Program that included five two-day workshops on a variety of process improvement and statistical methods, ranging from the basic tools (process mapping, brainstorming, fishbone diagrams, etc.) through the more advanced Failure Mode and Effects Analysis (FMEA) and Design of Experiments (DOE) techniques.

The seminars were presented over a three-month period, and between each session three cross-functional teams applied the tools to their assigned projects. The projects were selected and assigned by the client’s management team, and each project team had one of the managers as their project sponsor.

The selection of good projects and teams is critical for the certificate program. It’s not enough to have attendees complete the training; they must also successfully complete their assigned projects. I met with the leaders to review criteria for what makes a “good” process improvement project and provided guidance so good projects were indeed selected and assigned.

One of the projects had the objective of reducing the cycle time to improve the throughput of a CNC drill operation.In the plant’s overall value-stream process map (VSPM), the CNC drill operation was identified as a major bottleneck and source of delays in manufacturing lead time.

The second-to-last course in the certificate program was the two-day Introduction to Design of Experiments (DOE) workshop. At the end of that workshop, all three teams designed experiments to be applied in their project efforts.

The CNC drill team designed a 2 x 3 (two levels, three factors) factorial experiment. One response recorded at the end of each run was the hole diameter. A second response was an attribute observation having to do with fraying of the fabric in and around the hole.One of the factors selected was throughput speed. The low level was established at one-half the standard speed, and the high level was established at fifty percent higher than the standard throughput speed.

After the team completed their runs and analyzed the response data, they discovered that none of the factors or interactions had a significant effect on hole diameter and fabric fraying. The team leader sent an email to me and reported that their experiment was not successful because none of the factors had a significant effect on the responses.

I started to write an email back to the team leader to explain that when no factors are found to be significant, that is in fact a successful experiment. You don’t have to put a lot of time and effort into controlling them and you can select new factors for study.

But then I remembered that one of the factors was throughput speed. I called the team leader and asked, “You mean it didn’t matter whether you pushed the parts through at half the standard speed or fifty percent higher than the standard speed?” The team leader replied, “No. No significant difference.”

I asked, “Then what are you going to do?” The team leader replied, “Increase the speed, I guess.”

As a result of the project, the plant experienced a significant improvement in throughput at the CNC drill in the short term. Within a few months after the completion of the project, all back orders from the client’s major customer had been eliminated.

The solution to the throughput problem was more the effective application of DOE than the complete certificate program; although a lot of the foundation for the experiment was established by the team throughout the course of the project. That firm foundation consisted of the process map, definition of good key measures, assuring that the CNC process was in a state of statistical control prior to experimentation, and other tools the team learned and applied during their project.

In closing, on behalf of Quality Support Group I was able to help the client achieve dramatic reductions in the CNC process cycle time and overall manufacturing lead time. Increased throughput and the elimination of back orders also resulted from what the team initially thought was an “unsuccessful” experiment!

© 2019 James F. Leonard. All Rights Reserved.

Copyright ©2019. All Rights Reserved.