Getting to World-Class Supply Chain Measurement

By Debra Hofman -- 10/1/2006

At AMR Research, we field inquiry calls from companies on a wide range of topics related to supply chain and technology. One topic that is becoming increasingly prevalent is supply chain measurement. The questions we receive run the gamut, from “What should we be measuring?” to “How do we get these numbers?” to “What do we do with the data once we get it?” What companies are in essence asking is, “How do we get to world class supply chain measurement?” Challenges of Supply Chain Measurement The road to world-class supply chain measurement is scattered with obstacles. Some of the challenges that we hear about from companies as they attempt to institute metrics programs include: Too many metrics. One of the most prevalent issues companies grapple with is having too many metrics. We see companies that are looking at hundreds, sometimes thousands of metrics. The result, of course, is that 1) it's very difficult to actually gather the data for so many metrics, and 2) if you do manage to gather the data, it's difficult to figure out what to do with it without drowning in the detail. What happens then is all too common: Eventually, pressing business needs take precedence, and the metrics and data sit on a shelf collecting dust. Endless debate over metric definition. Another issue we hear about is people endlessly debating the pros and cons of each metric. Clearly, some debate is healthy and, more importantly, necessary. It's important to vet each metric, challenge it, make sure it's going to provide useful information, and so on. It's also important to allow debate so that the organization buys into the metrics. There is a point, however, where debate becomes a form of resistance, providing a way to put off change. Recognizing when the organization has crossed the line and addressing it appropriately is critical to moving ahead. Constantly changing metrics. This is the flip side of the endless debate issue: You've identified the metrics, you've put them in place, and then you find them changing constantly. One company we talked to described this as the “metric-of-the-year syndrome.” Old data. Another challenge we hear about is an inability to collect the data in a time frame that allows it to be meaningfully used. If the data is old by the time it's collected, you've wasted valuable time. For example, one company we talked to said they had decided to look at on-shelf availability using point-of-sale data. However, it routinely took four weeks to collect and cleanse the data, and the result was that the data was useless by the time they got it. Gaming the system. This is a common challenge to any change effort but particularly to one that involves metrics. Inevitably, people will find creative ways to get around the system, particularly if they feel there will be personal negative repercussions otherwise. For example, in our benchmarking studies at AMR Research, we measure the perfect order, which is defined as an order that's complete, accurate, on-time, and in perfect condition. An order that is split because product is not in stock for some of the lines on the order is considered incomplete, and therefore the whole order is imperfect. What some companies have found when they instituted this metric is that in these situations, people were canceling the original order and replacing it with two new orders that could be filled, thereby keeping the perfect-order rating up. So what can companies do to address these challenges and get to world-class measurement? First, we'll describe the dimensions of good supply chain measurement, and then we'll identify some best practices to follow and pitfalls to avoid. Performance Measurement Maturity In our supply chain benchmarking studies at AMR Research, we've found substantial differences in companies' measurement maturity. What differentiates the leaders are two dimensions: first, their ability to measure, and second—and just as important—their ability as an organization to act on the results. We refer to these in Exhibit 1 as “measurement aptitude” and “results actionability.” Differentiating between these two is critical because many companies do not have both capabilities, and both are essential. Companies that score high on measurement aptitude: Know what to measure. Have a program in place to measure it. Are able to easily access the right data in a timely manner. Measurement aptitude is about quality, not quantity. Where a company falls on this axis is not about how much measuring they do, it's about how well they do it. There are companies that have an entire department dedicated to performance measurement and have made huge investments in it, but they still can't easily get at the metrics that matter at the level of the business that matters. Results actionability is made up of two components: The ability to accept the results. This can be particularly difficult when the numbers don't turn out the way some people expected or wanted them to. It's important to note, though, that accepting the results is not necessarily the same thing as agreeing with the results. We often see a lot of debate regarding the numbers and the interpretation of what they mean. In fact, some debate about the results is desirable. But at some point, it's time to stop the debate, move up from the details, come to the key overarching conclusions about what the data is telling you, and move forward. Often companies get mired in the minutiae of the way this or that metric was calculated and never get out of it, losing track of the ultimate goal. • The ability to act on the results. Once the results are accepted, there needs to be organizational mechanisms in place to act on them. If, for example, the measurement exercise identified an issue with supplier performance, initiatives and deadlines need to be put in place with resources assigned to address the issue. To get to a world-class supply chain measurement capability, managers need to know where their organization is along the measurement-maturity curve and where its strengths and weaknesses lie. We've seen companies generally fall into one of four categories: 1) Excellence Addicts. These are the companies that rate highly on both dimensions. They are always looking for ways to get better, to improve, and to constantly tweak their performance and excel. They go beyond simply accepting the results of a measurement project—these companies actively embrace the results and enthusiastically look for ways to implement change. In our benchmarking studies at AMR Research, these are the companies that tell us that if the measurement results only give them good news, the effort will have been a failure; if they're good in their peer group on a certain metric, they want to know what the best is outside their peer group and how they can achieve it. 2) The Right Stuff. Companies with the “right stuff” have the ingredients with which to act: the right level of executive involvement, an open attitude and culture with regard to measurement, and the necessary organizational structure and processes. What they lack is the hard data and metrics foundation on which to make sound decisions—they don’t know what to measure or how to collect the data. Typically we see this in situations where a) there has been a change in leadership of the supply chain organization, or b) there’s been a change in the marketplace that is driving the CEO to mandate measurement from the top down. 3) Ingrained Inertia. Companies in this quadrant rate low on measurement aptitude and results actionability. Sometimes it’s a case of not measuring at all. In other cases, a company does a lot of measuring, but it takes them three times longer than average to get at the numbers, and there is no organizational momentum or executive support to accept and act on the results. Sometimes the people who are responsible for measuring do not have the influence they need and are outside the group being measured. As a result, there is no buy-in to the data collection effort, and they often meet subtle (or not-so-subtle) resistance from the people that have the best access to the data. 4) Analysis Paralysis. Sitting in the fourth quadrant are the companies that measure well but do not have the organizational buy-in or acceptance to act on the results. The culprits here often are a lack of a strong and clear management directive to resolve conflict and an organizational culture that is defensive rather than open. There may be too many stakeholders with conflicting agendas, or one naysayer who has been not dealt with appropriately and derails the effort. Or it may just be a case of organizational disconnect—there’s a group who is doing the measuring, but there’s no action in the part of the organization that needs to actually do something with the results. What happens typically is that the company gets bogged down in the results and loses sight of the ultimate goal. The result: measuring simply for the sake of measuring. Moving up the maturity curve requires putting in place a comprehensive and ongoing measurement program. In implementing a measurement program, it’s important to distinguish between what it takes to define the metrics your organization will use versus what it takes to implement the measurement process itself. Below are some best practices related to each of these stages. Doing these well will allow companies to improve on both dimensions of the performance measurement maturity model and move up the curve. Metric Definition Best Practices Design different metric portfolios for different goals. The first and most important thing you can do before you start deciding what metrics to collect is to clearly define your goal. Who will be using these metrics and for what purpose? Different goals or purposes will require different metrics, and each should have its own distinct portfolio. For example, the metrics that a CFO needs in order to know how the supply chain is performing are very different from the metrics that the vice president of supply chain needs to manage the supply chain and fi x any problems that arise. There may be some overlap, certainly. They both want to see inventory levels, but the CFO wants to see them in terms of total inventory value. In contrast, the vice president of supply chain needs to see inventory days, not just value, and should see it broken down into raw material, work-in-process, and finished goods inventories since each indicates something different about the possible root causes of any problems. We often get requests from companies to review and provide feedback on their metrics. Sometimes it’s possible to tell who decided what metrics to collect simply by looking at them. One company we worked with sent a list of five metrics, and all five were finance-related supply chain metrics such as days sales outstanding and manufacturing operating cost. Clearly the CFO had mandated that these metrics be used. However, although the metrics were very useful for him, they were not as useful for the supply chain folks who needed to identify the potential root causes and solutions of any problems they were having. Keep it small: avoid the "mushroom effect." It’s important to keep each portfolio to a manageable size for all the reasons noted earlier. The more metrics in the portfolio, the harder it is to consistently collect current, valid data that you can really use. Moreover, the larger the number of metrics, the harder it is to fi gure out what it all means even if you do manage to collect the data; the potential for confusion, lack of focus, and getting mired in the details all increase exponentially. What we’ve seen is that, without a concerted effort to minimize them, the number of metrics has a tendency to proliferate like mushrooms in the dark. Everyone has his or her favorites and often very good reasons for wanting to include or exclude certain metrics. Clarifying what you’re trying to do with the metrics, as noted above, will help guide decisions by providing objective criteria for what metrics should be included. Take the example of fi rst-pass yield. This is a very useful manufacturing metric, and it may be important to include it in a portfolio of deep-dive manufacturing metrics that aims to identify the root cause of any time or cost issues in the plants. But it’s not as important to include it in a portfolio of supply chain metrics that aims to look across the supply chain at a higher level and to identify and delve into potential root-cause areas. Many companies start their metric-defi nition effort by gathering all the metrics that people are currently measuring in the organization. They then trying to rationalize them and take out the metrics that they don’t need. We find that this method doesn’t work very well. It’s diffi cult to throw out metrics that are already in. Once a metric is in, people are worried about what they will lose by not collecting it, and there’s always a good reason to keep just one more metric. What works better is to start with an empty portfolio and put metrics in, using your clearly defi ned objectives as a guide. Finally, be sure that the guidelines you use to choose metrics help you focus from the outside in—that is, from the customer in, rather than from the inside of your company out. The goal is to create a portfolio that measures the performance of your supply chain as your customer experiences it. Address the Basics: Balanced, Cross Functional, Practical. These are the basics of good supply chain metric development. The portfolio should be balanced from the perspective of cost, quality, time, and effectiveness (and regulatory if applicable). Having a balanced portfolio will help ensure, for example, that you’re not measuring cost at the expense of service. Your portfolio should also be cross-functional to avoid "siloed" behavior that may optimize a particular function at the expense of suboptimizing the whole. For example, if logistics is holding shipments back so that they can consolidate loads to keep transportation costs down, you need to have metrics in your portfolio that will allow you to clearly see the impact of this on customer service metrics such as the perfect order. Finally, make sure the metrics you choose are practical. How easy is it to get the data required for the entire portfolio of metrics? If your organization is early in its measurement maturity, it’s best to be realistic about what can be accomplished. For example, some companies that have never measured the perfect order start with fill rates or ontime shipments because they can more easily get at those metrics. Over time, they then shift to a fuller measure of the perfect order. Align execution and strategy. Ensure that the metrics you choose don’t mask and do drive the correct behavior. One global company we worked with, for example, measured pockets of local inventory in different regions instead of global days of supply. By doing this, they created undesirable hoarding behavior. Another global company rolled up the performance of five divisions, which masked the fact that one of the divisions was carrying the other four on certain metrics. Understand the interdependencies. It’s important to clearly and explicitly understand the relationships among the metrics in your portfolio. Each metric does not sit in a vacuum. Similar to the supply chain processes they reflect, the metrics are interdependent, and certain metrics drive others. For example, we have found in our research that demandforecast accuracy is highly correlated with perfect-order performance: Companies that have better demand-forecast accuracy also tend to have better perfect-order performance. At the same time, perfect-order performance tends to be inversely correlated with performance on reducing supply chain cost; most of the companies we benchmark make a tradeoff between perfect-order performance and cost. For example, many companies hold higher inventories (increasing cost) to keep their perfect-order performance up. Clearly outlining these interdependencies addresses two important issues. First, it helps identify the metrics that matter, which in turn makes it easier to discard the less critical metrics and keep the portfolio small. Second, it serves to illuminate the path for root-cause analysis, which means that it will be easier to figure out what to do with the data once you’ve collected it. Rather than getting overwhelmed by a potpourri of numbers, you will have a structure in place with which to analyze results. AMR Research’s "Hierarchy of Supply Chain Metrics," for example, is a useful tool for understanding the relationships (see Exhibit 2). The hierarchy is a three-tiered framework that gives managers a progressively more granular view of performance. The top tier presents a 50,000-foot view of the overall health of the supply chain and the high-level trade-offs. The middle tier uses a composite cash-flow metric that provides an initial diagnostic tool, and the third tier uses a variety of metrics that support effective root-cause analysis. (For more information, see "The Hierarchy of Supply Chain Metrics" from the September 2004 issue of Supply Chain Management Review.) It’s worth noting that we often find a disconnect here between concept and practice. Everyone conceptually understands that metrics are interdependent. However, because companies are still largely organized by function, it’s easy to develop tunnel vision and focus on just the metrics that relate to one function or another, losing sight of the interactions among them. Balance the need for standards versus customization. One of the issues we see companies struggle with—particularly companies that are implementing a standard metrics portfolio across multiple divisions—is how to balance the need for standardization versus the need for customization. Often each division operates in very different markets and geographies, and each runs very different supply chains. At the same time, there are basic metrics that apply to any supply chain, no matter what environment it operates in. Every supply chain should aim to deliver a profitable perfect order, and as such, every supply chain should measure demand-forecast accuracy, the perfect order, total supply chain costs, and cash-to-cash cycle time (which includes inventory). What’s important here is to recognize where there are similarities and differences across supply chains and to put rules in place to address them. One option is to mandate that certain portions of the portfolio will remain standard, while other limited portions will accommodate some customization. Some of the companies we work with are implementing our hierarchy of supply chain metrics as their portfolio of metrics. In a few cases, they’ve kept the top two levels of the hierarchy standard but have allowed some customization in the third level. Another technique that works in some cases is to keep the metric label the same but allow some modifi cation to the definition where absolutely necessary, keeping in mind the conceptual intention behind the metric. Take the example of the request-to-quote metric. This metric measures the time it takes to respond to a request from a customer with a commitment (amount of product that can be provided and when). In a material-release environment where customers are not making individual requests but are rather pulling against previously defined commitments, this metric would seem to be inapplicable. However, it’s important to keep in mind the spirit of the metric, which is to measure the time it takes to respond to a customer request. Even in a material-release environment, there are occasionally requests that exceed contracted ranges, and the metric can be used to track the time it takes to respond to those requests. Metric Implementation Best Practices Develop a Metrics Strategy and Time Frame. The first step to take before implementing a set of metrics is to outline your strategy and time frame. This may sound obvious, but most companies don’t typically put the words "metrics" and "strategy" together. Implementing a measurement process is an organizational change, often of great magnitude, and as such it warrants a plan. The strategy should draw a line between the metrics defi nition phase and the implementation phase. It should identify the evolutionary stages of the metrics portfolio, or the points at which the metrics will be allowed to change. For many companies, implementing the metrics they want to collect all at once would be too diffi cult to accomplish in one step. For example, one company we worked with decided to implement the perfect-order metric. However, at that point in time, they were only measuring on-time shipments. Their strategy The first step to take before implementing a set of metrics is to outline your strategy and the time frame for executing it. This may sound obvious, but most companies don’t typically put the words "metrics" and "strategy" together. Implementing a measurement process is an organizational change, often of great magnitude, and as such it warrants a plan. The strategy should draw a line between the metrics definition phase and the implementation phase. It should identify the evolutionary stages of the metrics portfolio, or the points at which the metrics will be allowed to change. For many companies, implementing the metrics they want to collect all at once would be too diffi cult to accomplish in one step. For example, one company we worked with decided to implement the perfect-order metric. However, at that point in time, they were only measuring on-time shipments. Their strategy therefore outlined that in the first year, they would measure on-time shipments; in year two, they would expand to measure on-time delivery; and in year three, they would further expand to the full perfect-order measurement. The issue of constantly changing metrics noted above is a symptom of lacking a metrics strategy that clearly states when metrics definition is over and defi nes when metrics may or may not be changed. Define Scope. The implementation strategy also needs to outline the measurement scope. Most companies have more than one supply chain, and the definition of what constitutes a supply chain is often independent of a company’s organizational structure. For example, consider a food and beverage consumer products manufacturer that makes both dry and refrigerated goods. While the different product lines might reside in one business unit, they are very distinct supply chains and need to be measured separately. Pay Attention to Roles, Responsibilities, Structure, and Process. Make sure you have the structure and processes in place or at your disposal with which to act on the results of your measurement exercise. The goal of measuring is not only to identify problem areas but also to fix them. That requires a team of resources with processes in place and the authority to enact change. Otherwise, the results will simply turn into shelfware. It’s also important to choose the right resource to manage the data collection effort. The actual data will obviously come from multiple people in the organization who are responsible for different areas (for example, order management, production, and procurement). The person responsible for coordinating the effort must have influence in the organization and be well-respected by his/her peers in order to be successful in mobilizing their effort. Manage the Culture. Finally, one of the most important nissues to pay attention to is the measurement culture of your organization. The culture around measurement is separate from, but related to, the larger corporate culture. Is your existing culture one which will embrace the results of a measurement exercise, or will people be defensive? Will people get mired in the details of the metric calculations, or will they be able to step back and see the big picture? The way people respond to the results is tied to how the results will be used and, even more importantly, how people believe the results will be used. If people believe the results will be used in a way that negatively affects them—for example, their bonuses will be reduced, they will be blamed for any problems that are uncovered and their performance ratings will be negatively affected, or simply that they will be perceived as not doing their jobs—they will naturally resist and be defensive. Here is where senior management plays an important role. People take their cues from the words, attitude, and, most importantly, behavior of their senior managers. Executives need to clearly state that the purpose of measuring is to continuously improve rather than to place blame. They then need to make sure that their actions are consistent with that statement. Doing so will go a long way toward making a measurement program successful. In this way, each of the solutions listed above will help companies surmount the obstacles to world-class measurement as summarized in Exhibit 3. Ongoing measurement is key The growing interest in implementing a solid measurement program for the supply chain suggests that more companies are aware of the strategic importance of metrics. A worldclass, ongoing supply chain measurement capability allows companies to accomplish three distinct goals: 1. Set targets: Having hard data that tells you where you are versus others will help you determine what are realistic and desirable performance levels to target. 2. Achieve targets: The best companies use the data not only to tell them where they stand. By looking at the connections among the metrics, they use the data to help them analyze the root cause(s) of any problem areas that are uncovered and thereby achieve their targets. 3. Stay excellent: It’s not just about where your performance is today. Excelling at measurement gives you the capability to constantly improve. Without that, even the best company is at risk of being leapfrogged by existing or new competitors. In the end, excelling at supply chain measurement is crucial not just because it allows you to set targets to get to best, but because it also helps you get there and stay there.