We built the Competitive Interaction Analysis (CIA)® platform to work with any combination of geographies, time periods, and products. However, this flexibility has often haunted us when we freely offer to utilize any combination of products for a client-driven definition of the “category” and competitive set. When pressed for our opinion of the category, we generally ask a simple question: what is you market share? When the client replies seven or 18 percent (or whatever), we follow up with a second question: seven or 18 percent, or whatever percent of what? The “what” response is our recommended category definition. A lot of the time this gets us to where we need to be. Other times, it isn’t that easy.
I worked with Sally Martin at IRI in the nineties. She runs an excellent website called CPG Data Insights that addressed the significance of this question a few years ago. I agree with her argument that the right way to define a category is based on two key elements. The first is our concept of the wide-angle view and the shopper perspective. The shopper-centric category definition includes all the products that meet a similar consumer need during a usage occasion. She states that this is pretty easy for a product like toothpaste, but more difficult for popcorn. The second notion is how retailer partners define the category. The location of the products in the store is a major driver and supersedes the shopper perspective. For example, refrigerated orange juice from concentrate and frozen orange juice might be interchangeable to shoppers. However, they may be broken into two separate categories due to there being two different retailer buyers.
A recent modelling engagement went off the rails when we agreed to simultaneously produce results for a narrow and traditional category definition as well as a far broader one across multiple traditional categories. The resulting broader competitive set was five times the narrow one. I explained that greatly increasing the “category scope” would introduce new expected competitors. I also said that it would be extremely unrealistic to assume the same SKU by SKU transferred demand in the traditional or the incrementality categories. Unfortunately, this concept was not embraced. This was only made worse by how the client insisted on interpreting “relative” differences.
Unfortunately, the reliance on relative metrics as opposed to absolute metrics is a constant struggle for us at Middlegame. When we tell a client that a product with 200MM total unit sales has an incrementality of 20MM units, they immediately convert this absolute metric to a relative one, i.e. 20MM ÷ 200MM = ten percent incrementality. We then regularly see presentations comparing this ten percent to a “superior” result of 20 percent. But, the 20 percent potentially was a 10MM incrementality result for a product with 50MM total units. The 20 percent and ten percent are just not directly comparable. This is equally true when the debate is between the same product in different competitive sets.
The product in question was approximately 1,000MM units. The narrow model suggested that approximately 900MM of those units transferred within the competitive set while the remaining 100MM were considered incrementality. In the broader model, 300MM transferred to the same products from the narrow set (fair share index of 188), 500MM flowed to the new competitors (fair share index of 78), and 200MM were incremental. The client immediately converted these to absolute values, then to relative measures and cried foul. How could the broader model present 20 percent incrementality versus only ten percent in the narrow model? This was the classic telephoto view versus the wide-angle view. When we look at this from the shopper perspective, the product had gone from a ten percent share the narrow category to a two percent share of the broader definition. Introducing a competitive set that was 5X in size really mattered. If we look at the results in this context, the original 100MM was really 10% x 2% = 0.2% and the new 200MM was really 20% X 2% = 0.4%. This was like comparing the ratios of one in 500 versus two in 500. I admit that you could be disappointed if you expect the broader model to be the one and the narrow model to be the two. But is this precision realistic? These are statistically the same number.
I learned two things from the experience. The first is that, even though the data processing and model calibration engine never changes on the CIA® platform, it is far too optimistic to ever think that adding new products to the competitive set would generate similar results. The models fight hard to find interactions when we tell them they are meant to be there—even if they are super small—by defining the competitive set. The second is that relative metrics can destroy a conversation unless everyone is in the same frame of reference. That is sadly impossible if the discussion is about different models. Analytics need to be grounded in absolute measurement. Relative metrics should only be introduced after the absolute measurement is completely understood.
Middlegame is the only ROMI consultancy of its kind that offers a holistic view of the implications of resource allocation and investment in the marketplace. Our approach to scenario-planning differs from other marketing analytics providers by addressing the anticipated outcome for every SKU (your portfolio and your competitors’) in every channel. Similar to the pieces in chess, each stakeholder can now evaluate the trade-offs of potential choices and collectively apply them to create win-win results.