Dr. W. Edwards Deming’s "System of Profound Knowledge"

Download a copy (PDF)

W. Edwards Deming was a physicist by training, but his passion for much of his life lay in transforming American management. Deming drew deeply from physics in formulating his ideas. Unfortunately, his early efforts garnered little acceptance in the United States. Deming's concepts won credence here only after his management philosophy was credited with turning Japan into a world economic power.

A physicist I know (a research scientist in a Fortune 100 company) once told me, “The successful industrial physicist will be more familiar with Deming than Einstein.” Regrettably, most industrial and academic physicists know little of Deming, who envisioned management as a system whose components worked in elegant interdependence, or how his work emphasized the critical role played by the science of physics in any rational business strategy to compete in this new economic age.

The antecedents to Deming's ideas lay in the work of physicist Walter A. Shewhart of Bell Laboratories. Deming graduated from the University of Wyoming in 1921 with a degree in Electrical Engineering and earned his PhD in mathematics and physics from Yale University in 1928. During the summers of 1925 and 1926, he worked at Western Electric Company's Hawthorne Works near Chicago, and there he first heard of Shewhart and his theory of variation.

Shewhart focused on understanding and improving systems, and he invented analytic statistics and the statistical process control chart to help him in his work [see Figure 1]. According to Donald Wheeler and David Chambers in Understanding Statistical Process Control, Shewhart concluded that “while every process displays variation, some processes display controlled variation while others display uncontrolled variation."1

Shewhart's theory greatly impressed Deming, who incorporated parts of it into his own principles and teachings. In the mid-1930s, Deming arranged for Shewhart to deliver a series of lectures at the U.S. Department of Agriculture. He later wrote, "Even if only 10% of the listeners absorb part of Dr. Shewhart's teachings, the number may in time bring about change in the style of Western management."² Those words foreshadowed his own influence and impact. Although only about 20% of Japanese businesses adopted Deming's management concepts, this proved to be a critical mass that significantly changed the course of the Japanese economy in the decades of the sixties and seventies.

Deming joined the faculty of the New York University Graduate Business School in 1948 and taught there until his death in 1993. His influence expanded through his seminars, worldwide consulting services, numerous papers, and his texts Out of the Crisis and The New Economics for Business, Government, Education.

Profound Knowledge

Deming’s principles for transforming management rest on what he called “a system of profound knowledge.” This system consists of four components, each of which interacts with the others.

1. Appreciation for a system. Deming defined a system as a network of interdependent components that work together to accomplish some aim. In his short paper about his system of profound knowledge he wrote, “The aim of the system must be stated by management. Without an aim, there is no system.” He cited a good orchestra as an example of an optimal system. “The players are not there to play solos as prima donnas, to catch the ear of the listener. They are there to support each other. They need not be the best players in the country.”³
   
   Peter Senge noted that schools, businesses and other human endeavors are systems, “bound by invisible fabrics of interrelated actions, which often take years to play out their effects on each other. Since we are a part of the lacework ourselves, it’s doubly hard to see the whole pattern of change. Instead, we tend to focus on snapshots of isolated parts of the system, and wonder why our deepest problems never seem to get solved.”⁴
    
2. Some knowledge of the theory of variation. Deming stressed that one need not be eminent in any part of profound knowledge in order to understand it as a system, and to apply it. One need not have a PhD in statistics to understand variation. Rather, Deming placed emphasis on understanding and differentiating between controlled, random, or common cause variation and uncontrolled, non- random, special cause variation.
   
   Common causes of variation come from within the process. They generate outcomes that are different, but not significantly different. Special causes intervene from outside the process and produce outcomes that are not different, they’re significantly different.
   
   Making this distinction is critical for managers and scientists in determining an appropriate improvement strategy. For common cause variation, the appropriate strategy is to change and improve the process. For nonrandom, special cause variation, the appropriate action is to find, remove and prevent the reoccurrence of the special cause. No amount of work on a process will address a special cause, because uncontrolled variation comes from outside the process.
   
   Imagine that a manufacturer experiences a significant increase in product defects. The actual source of the defects was an abnormally high level of contaminants in a supplier’s material; a special cause of variation from outside the manufacturer’s process. The manufacturer, however, reacts to the defects as though they were generated by its own process, and invests capital in new processing equipment. Failing to understand the variation and its true source, the manufacturer will have made a very costly mistake. The special cause of the defects from the supplier’s out-of-control inputs will come screaming in without warning – unpredictable by its very nature – and produce defects on the new capital equipment just as it did on the old machines. The manufacturer will have flushed its capital investment right down the toilet!
   
   Heero Hacquebord’s six-year old daughter experienced the effects of mistakes made due to an absence of knowledge of theory of variation. Deming reported the incident as follows:

   Need a teacher understand something about variation? Mr. Heero Hacquebord sent his six-year-old daughter to school. She came home in a few weeks with a note from her teacher with the horrible news that she had so far been given two tests, and this little girl was below average in both tests... The little girl learned that she was below average in both tests. She was humiliated, inferior. Her parents put her into a school that nourishes confidence. She recovered.⁵

   Fortunately, the little Hacquebord girl’s story had a happy ending. How many students’ stories have a sad or tragic ending because they have teachers who do not possess some knowledge of the theory of variation?

3. Theory of knowledge. Deming wrote that this third component of profound knowledge helps people to understand that management in any form requires prediction – and that prediction must be based on some theory. He added, “The theory of knowledge teaches us that a statement, if it conveys knowledge, predicts future outcome, with risk of being wrong, and it fits without failure observations of the past.”⁶ Thus, Deming insisted that examples and case studies without theory teach nothing – a daunting thought to people in organizations managed by graduates of business schools with curricula based on case studies!
   
   Based on the insight that examples without theory teach nothing, I learned that the best response to requests for alternatives to merit pay in businesses and grading systems in schools is in the form of a question.⁷ I now respond to such requests by asking, “If I give you an alternative to your merit pay (or grading) system, do you promise me you’ll implement it?” In every case, the answer to my question has been a resounding no.
   
   I then ask, “Well, if I give you an alternative to your merit pay (or grading) system, what will you do with it?” The most common response: “We’ll examine it.” I ask, “You’ll examine it relative to what?” The answer: “Relative to our current system.”
   
   I’m slower than most, but applying theory of knowledge helped me realize that I was doing people a disservice handing out alternatives to merit pay and grading systems like candy. I don’t what them to compare it to what they’re doing now. I want them to compare it – the alternative – and what they’re doing now to a rational theory.
   
   One best way to teach the theory is to conduct Deming’s classic bead experiment, which provides insights and appreciation of a system and knowledge of theory of variation. Beyond that, however, the bead experiment demonstrates the destructive effects of grading and ranking on the people who work in the system. That’s why I include the bead experiment – even though it deals with pretty advanced theory of systems and variation – in my introductory courses on Six Sigma and process improvement.
   
   Just as daunting is Deming’s contention that experience without theory teaches nothing. Videocassette recording (VCR) technology was invented in the United States; the United States has more experience in manufacturing than any other nation on earth. Before the dawn of DVDs, where could you buy an American- made VCR? There weren’t any left! All of our manufacturing experience, absent sound management theory, failed to teach us enough to be able to make and sell VCRs at a profit. We lost the market due to bad management. As Deming taught, experience without knowledge of rational theory teaches nothing.

4. Psychology. Deming wrote, “Psychology helps us to understand people, interaction between people and circumstances, interaction between a manager and his people and any system of management.”⁸ His philosophy for leadership rests on the belief that people are intrinsically motivated. They strive naturally for dignity, pride and joy in their work. Unfortunately, the current American management system destroys intrinsic motivation by substituting extrinsic motivators such as merit pay, sales commissions and grades in school. Thus, too many students strive for high grades, not knowledge. Too many workers strive for merit increases and high rankings, not quality or the intrinsic joy one experiences from a job well done.
   
   As it relates to motivation, is the appropriate strategy to try to motivate people? Or is it to remove barriers to their own (intrinsic) motivation? Senge wrote about how this dilemma exists in efforts to achieve the goal of continuous improvement, “which remains an elusive target for most American organizations.”

   Motivate them. From an extrinsic perspective, the only way to get continuous improvement is to find ways to continually motivate people to improve... Otherwise, they will just sit there – or worse yet, slide backwards. This leads to what workers perceive as management continually raising the bar to manipulate them.
   
   Loose their own motivation with information and appropriate tools.  However, from an intrinsic perspective, there is nothing mysterious at all about continuous improvement. If left to their own devices, people will naturally look for ways to do things better. What they need is adequate information and appropriate tools. From the intrinsic perspective, people’s innate curiosity and desire to experiment, if unleashed, creates an engine for improvement that can never be matched by extrinsic rewards.⁹

Interaction of the Components

Dr. Deming insisted that the different components of the system of profound knowledge cannot be separated; they interact with each other. He described, for example, how knowledge of psychology is incomplete without knowledge of variation, just as understanding of variation will be incomplete without appreciation for a system.

Senge cited one way in which the first component (appreciation for a system) interacts with the fourth component (psychology). He observed that, over time, people will take on the characteristics of the system of which they are a part. “The systems perspective tells us that we must look beyond individual mistakes or bad luck to understand important problems. We must look beyond personalities and events. We must look to the underlying [systemic] structures which shape individual actions."¹⁰

I'm intrigued by the interaction of the third and fourth components of profound knowledge (knowledge and psychology). For example, Sigmund Freud and the other determinists worked for the most part with the deranged; but their findings and techniques are applied to people with healthy minds. Absent sound theory of knowledge as a guide, what damage might psychiatrists and psychologists be causing?

Similarly, the work of Skinner, Watson, Pavlov and the other behavioral psychologists was largely with animals; but some are applying their findings to people. How long will we continue to destroy our most important asset – our children – with grading practices based on behavioral carrots and sticks that worked with animals?

Performance Evaluation

Based on his system of profound knowledge, Deming insisted that most of the differences observed in workers’ performance levels have nothing to do with the workers. Rather, most of the performance differences are generated by the system, of which those people are but a part. Appreciation for a system would inform us that people can perform no better than the system allows.

A salesperson may control whether he or she visits Customer A or Customer B this morning; but the salesperson does not control product design, production quality, delivery performance, billing practices, after-sale technical service, and many other factors that influence whether or not a sale is made; whether or not there is a repeat sale. The sales commission system, however, ignores this reality. It assigns to the salesperson alone outcomes that were heavily influenced by many variables beyond his or her direct control. Then, salespeople are rewarded or punished based on the sales variance numbers for their territories – as if they had complete control over all the factors that generated those numbers. Sales commissions and merit pay confound the person with all the other variables that affect performance.

On the other hand, systems thinking assigns most performance differences to the system, not the people alone. Therefore, Deming urged the elimination of merit ranking and reward systems, as well as the abolishment of the grading system in America's schools. One way to consider the rational theory behind these radical proposals is to try to solve the following math problem.

If A + B + C + D + E + F = 73, what is the numerical value of "F"?

Thinking logically, one would conclude that this problem cannot be solved without knowing the values (or sum) of variables A through E. The American education system, however, is willing to give a student a low grade on an exam, ignoring the host of other factors that influence test scores (see Figure 2).

Thus, we hear that Molly got a 73 on her math exam, so she received a grade of C-minus. When we look at work and learning from an appropriate systems perspective, however, it becomes clear that sources of variation in test scores include more than simply Molly and her fellow students – just as sources of variation in project schedule performance and costs include more than just scientists and project engineers.

Teamwork

Recall that appreciation for a system requires a clear understanding of an overall system, as well as managing its individual components to work well together. Doing so will optimize the system’s performance. When consumers buy cars, they don’t want engines in those cars that are made up of perfect individual components. Rather, they want engines with components that work well together to move them to their destinations.

By the same token, workers and shareholders don’t need organizations made up of perfect individual divisions, and a plant manager doesn’t need a plant that’s made up of perfect individual departments or shifts. Rather, we need plants with departments that work well together and companies with divisions that work well together. Such optimization will assure that we’re still in business, capturing markets, providing jobs and paying dividends 10, 20 and 30 years from now.

Notes

¹ D. Wheeler and D. Chambers, Understanding Statistical Process Control, Second Edition, SPC Press, Inc., Knoxville, TN (1992), p. 4

² C. Kilian, The World of W. Edwards Deming, Mercury Press, Rockville, MD (1988), p. 58.

³ W.E. Deming, “A System of Profound Knowledge,” from J.F. Leonard, The New Philosophy for K-12 Education: A Deming Framework for Transforming America’s Schools, ASQ Quality Press (1996), p. 311.

⁴ P. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization, Doubleday/Currency, New York, NY (1990), p. 7.

5 W.E. Deming, The New Economics for Industry, Government, Education, Second Edition, MIT Center for Advanced Educational Services (1994), p. 101.

⁶ Ibid., p. 102.

⁷ Leonard, The New Philosophy for K-12 Education: A Deming Framework for Transforming America’s Schools, pp.183-184.

⁸ Deming, The New Economics for Industry, Government, Education, pp. 107-108.

⁹ P. Senge, “Building Learning Organizations,” Reprint from the Journal for Quality and Participation, March 1992, p. 3.

10 Senge, The Fifth Discipline, pp. 42-43.

© 2012 James F. Leonard.  All rights reserved.