About Metrics

Following is the nuts-and-bolts part about implementing business and manufacturing metrics. But before that, consider the strategic (big picture) side. If you are asked about how you would go about implementing quality metrics, consider this approach, an approach consistent with strategic leadership (e.g. Director or VP level thinking):

• Get an assessment of the current situation.

• Chose items to report on a scorecard for drivers that the company has decided are important for its long term health. (All groups within Engineering would ultimately prepare a scorecard that would be in concert with the overall business plan of the enterprise.)

• What causes the highest level of customer satisfaction with the company's products?

• Ask the senior leadership what is their vision for the company.

• Ask how you (e.g. Engineering Director) and your department (e.g. Engineering) can help their vision.

• Identify a quality plan that supports the overall goals of the organization.

• Remember that Engineering can effect two things: Cost; Throughput. (Profit is a function of pricing.)

Once the above is understood at the senior management level, to implement this with the Engineering staff the Director/VP would follow the following thought processes:

Metrics are measurements. If you can measure it, you can improve it. If you cannot measure it, how do you know if you have improved it? Consider the following quote:

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge of it is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced it to the stage of science.

Sir William Thompson, Lord Kelvin (1824-1907)

So how do you go about implementing metrics in the workplace? Where do you start? What do you do?

Assuming a good Lean Manufacturing environment exists (i.e., most of the effort is value added), the place to start for developing metrics is to assess the current "Cost of Quality," (also known as the "cost of poor quality"). The COQ are those financial expenses due to:

• test and inspection

• warranty costs

• rework and scrap

• lost customers

• customer dissatisfaction

Let's also extend our COQ to include "opportunity costs." Opportunity cost is defined as the cost of pursuing one alternative versus another. If we're spending resources (manpower, dollars) on improving products that already exists, we do not spend those same resources on new product development.

What metrics, then, apply to existing products and what would apply to new product development?

Current Products

For existing products we would use warranty reports to quantify costs and separate it out by quantity and cost for any given item. This works well using a Pareto Diagram. We can do the same thing for scrap, rework, test, and inspection - as these are all internal costs. Often these can be measured as a percent of sales or some other ratio and goals set accordingly. When a manufacturing process is involved, we can put metrics in place to understand and control its variability. Classical design of experiments and statistical process control methods can be used here. When the process mean and standard deviation is understood, it can be maintained by the use of run charts and other SPC charts.

Thus, for existing products, the heavyweight  tool sets that apply include

• Pareto Diagrams

• DOE (design of experiments)

• SPC (statistical process control)

New Products

Where the money really is is in new product development. Here the DFSS (design for six sigma) tools apply - these are tools that considers the customer as well as the manufacturing processes used to create the product. Gains in quality can be made with the DFSS approach that will never be realized with classical six sigma as applied to existing products. Thus DFSS becomes the bridge between COQ and new product development. Heavyweight tools here include:

• QFD/Kano

• Business Justification

• Specification Requirements

• Risk Mitigation

• FMEA

• DOE

• y=f(x)

• Optimization

• SPC

• Phase Gate Reviews

Any fool can plaster the walls with SPC charts and other charts (and many fools do). But before any COQ metrics are employed, ask the question: "what is the business benefit for improved quality?" Know what you are trying to accomplish and for what reason. Be able to quantify the cost benefit of whatever you do. And note again - the best way to achieve high quality in products is to design them first  according to the DFSS method and then release to production a product that meets the customer's needs, makes business sense, and can be manufactured to 6-sigma quality levels from the start.

More about Metrics... I liked this article so much I included it herein.

How to Measure Success: Uncovering The Secrets Of Effective Metrics

By: Dave Trimble
Senior Partner, ProSci

"Businesses that succeed and make money constantly assess themselves and improve in all dimensions of their business; metrics are the cornerstone of their assessment, and the foundation for any business improvement."

". . . measuring the success of a reengineering effort is a problem. More than half (54 percent) of the executives said they have no consistent, reliable way of measuring the reengineering benefits." CFO Magazine, May 1995. Successful metrics will help you avoid this problem.

"How's it going?"

How many times have you been asked that question about your business? How do you answer that question fully and completely?

Most of us can answer the question in terms of our current financial results, BUT ... is that a complete answer? Or just a partial or superficial response? And does our answer support the future plans for our business?

Just being able to answer the question is important. It means that you, in fact, do know how well you're doing, and where your company is headed. Businesses that succeed and make money constantly assess themselves and improve in all dimensions of their business; metrics are the cornerstone of their assessment, and the foundation for any business improvement.

On the converse, if you can't answer the question, that may be a precursor to dark times ahead for you and your business. This article uncovers the secrets behind SMART metrics that will produce results for your business.

What is a Metric?

A metric is nothing more than a standard measure to assess your performance in a particular area. Metrics are at the heart of a good, customer-focused process management system and any program directed at continuous improvement. The focus on customers and performance standards show up in the form of metrics that assess your ability to meet your customers' needs and business objectives.

Secret 1 - Measure the right things

Clearly, you need to measure your financial performance. However, today, most aggressively successful businesses find that they also need to assess other aspects of their business. Your measurement system should cover the following areas at a minimum:

CUSTOMERS

1. Performance against customer requirements

2. Customer Satisfaction

PERFORMANCE OF INTERNAL WORK PROCESSES

1. Cycle times

2. Product and service quality

3. Cost performance (could be productivity measures, inventory, etc.)

SUPPLIERS

1. Performance of suppliers against your requirements

FINANCIAL

1. Profitability (could be at the company, product line, or individual level)

2. Market share growth and other standard financial measures

EMPLOYEE

1. Associate satisfaction

Given a complete, or at least an adequate set, of current measures (remember, only you can judge the completeness of the measures), you need to ask yourself if these measures are driving you and your people to do the right thing. And right may be viewed as what will achieve the best results, both for today and for tomorrow. That leads to the second secret of effective measures.

Secret 2 - Create metrics that are SMART

Developing effective metrics may appear easy at first glance, but many have fallen into common traps that you can avoid. Examples of common pitfalls are:

1. Developing metrics for which you cannot collect accurate or complete data.

2. Developing metrics that measure the right thing, but cause people to act in a way contrary to the best interest of the business to simply "make their numbers."

3. Developing so many metrics that you create excessive overhead and red tape.

4. Developing metrics that are complex and difficult to explain to others.

What you need are metrics that are Specific, Measurable, Actionable, Relevant, and Timely or SMART objectives.

1. "Specific" in that your metrics are specific and targeted to the area you are measuring. For example, if you are measuring customer satisfaction, a good metric would be direct feedback from customers on how they feel about your service or product. A poorer metric would be the number of returned products or number customer complaints. While direct "internal" measure, they are indirect measures of customer satisfaction and, as such, can be misleading and produce unwanted surprises later on.

2. "Measurable" in that you can collect data that is accurate and complete.

3. "Actionable" in that the metrics are easy-to-understand, and it is clear when you chart your performance over time which direction is "good" and which direction is "bad", so that you know when to take action.

4. "Relevant" simply means don't measure things that are not important. A common downfall of process professionals or standards groups is to measure everything, which produces many meaningless measures.

5. "Timely" metrics are those for which you can get the data when you need it.

Metrics should be simple. If they require a lot of explanation and definition, then collecting data, and translating that data into actions becomes more difficult. Easy-to-understand metrics are easier to sell, and have a stronger impact on the process and the people who us it.

A few more tips on SMART metrics. Metrics generally fall into two categories:

1. performance metrics, and

2. diagnostic metrics

• Performance Metrics are high-level measures of what you are doing; that is, they assess your overall performance in the areas you are measuring. They are external in nature and are most closely tied to outputs, customer requirements, and business needs for the process.

• Diagnostic Metrics are measures that ascertain why a process is not performing up to expectations. They tend to be internally focused and are usually associated with internal process steps and inputs received from suppliers.

A common mistake is to start first with your diagnostic measures - measuring yourself internally, rather than beginning with an external focus, namely your customer. That leads us to the third secret, follow a process for developing your measures.

Secret 3 - Follow a proven process for developing metrics

One tried and true approach follows five simple steps.

1. First, identify your customers and outputs of your process. Customers may include end-users of products and/or services, process managers of downstream processes, and process users. Process Block Diagrams or Flowcharts may help at this point.

2. Second, determine your customer needs/requirements. Useful techniques include reviewing outputs with customers to gain their buy-in, establishing their needs and requirements, and asking them how to measure how well you are meeting their needs. You may want to use Interviews or Surveys. Use the same process with your suppliers as a way to measure the quality of their input to you and as a way to establish clear partnerships.

3. Third, ensure you understand the key goals of the business.

4. Fourth, determine effective measures, including both performance and diagnostic metrics. Here, we've found brainstorming and affinity diagrams to be particularly effective tools.

5. Finally, compare/filter/align your metrics for this process with those for the higher level processes of which they are a part. During this process you should be creating a table, where the rows of the table are labeled with the key measurement areas identified at the beginning of this article. The column headings would include items such as: metric description, current performance level, short-term objective, long-term objective, and competitive benchmark.

How Good Are Your Metrics?

When you have completed determining what you want to measure, you should step back for a sanity check. Ask yourself:

1. Do the metrics make sense?

2. How do they compare with your existing metrics?

3. Do they form a complete set (e.g., have you adequately covered the areas of time, quality, cost, and customer satisfaction)? and

4. Do they reinforce the desired behavior? For the long haul -- as well as for today?

Do not underestimate the last point. Your people will take action to achieve what you, by the metric, have told them is important. In some cases, the action they take may surprise you it may be very shortsighted and not at all what you had intended.

The question to ask yourself is "Will this metric drive the desired behavior?" "What type of behavior might this metric drive?" And "Will it help you move your business to where it needs to be in the months and years ahead?"

An Example

The metric areas of time, quality, cost, and customer satisfaction seem to be generic across industries. A product development organization maps to the first three of these areas include time-to-market, product reliability, and full-stream costs. In the area of customer satisfaction, they concern themselves with both end-users satisfaction, and downstream customers like Manufacturing and Services. Compare that to the key objectives of Lands End, a well-known direct-order clothing company. Their key objectives are:

1. Make your merchandise as good as you can.

2. Always, always price it fairly.

3. Make it a snap to shop, 24 hours a day.

4. Guarantee it. Period.

You can clearly see the elements of time (item 3), quality (item 1), cost (item 2) and customer satisfaction (item 4) are evident in their objectives as well.

Summary

We've covered a lot of ground at a relatively high level. Specifically, we've defined what we mean by a metric, looked at what we need to measure, given a short overview on some things that may help you establish your own metrics to help you determine what you should measure, and then given you some ways to help evaluate how good your resulting metrics are. And we've concluded with a short example.

Next Steps

Once you're satisfied with the evaluation of your metrics, have them in place, and are gathering data, you can begin tracking your progress; typically, you'll use run charts (performance over time) to show how you're doing. A run chart is a graph with time along the "x" axis and your performance measure on the "y" axis. As you gather data over time, you update the charts and you will begin to see a trend over time. When you see how you are doing, you're likely to ask, "Am I making progress fast enough?" and "Am I concentrating on the right objectives?" You will probably want to know how other are doing in the same area.  These questions can be answered by benchmarking against your competitors.

References

J. W. Wesner et al, Winning with Quality, Addison-Wesley, Reading, MA, 1995