As I teach evidence-guided product development I encounter two types of reactions. Most people realize that their company is caught in a vicious cycle of plan-and-execute that heavily relies on opinions, consensus, and HiPPO. But when presented with the alternative — using research, experimentation, building product through discovery and delivery — various types of objections crop up. Today I want to discuss a particularly pernicious one: “We’re already doing it” and give you a tool to deal with it.
I’m hearing “we’re already doing it” (or it’s ugly cousin: “Nothing new here”) mostly from senior product managers, usually director level and up. These folk, who are often the hippos in the room, tend to believe that they’re already doing the right things in terms of injecting evidence into decision-making. Sometimes they are right, and there’s not much I can teach them (in fact often I can learn from their experiences and techniques). In other cases it takes just a little bit of digging and asking other people, to find that there’s quite a big gap between the intent and the reality. I can tell you that when I was a senior PM, I too didn’t realize I was over-relying on opinions and consensus.
The GIST Scorecard
To make the discussion less academic, I created a simple self-test tool that allows product organizations to assess how truly evidence-guided they are (you can download it here as a spreadsheet.) It’s called the GIST Scorecard because it’s based on my GIST model — Goals, Ideas, Steps, Tasks — the four areas of change.
How to use: fill out the cells marked in green with a score of 0%-100% showing how much you’re practicing this particular thing. Then you can average in each category — goals, ideas, steps, tasks — and average again across categories to get your final score.
Go ahead, try it now. It just takes a minute.
You may notice some terms — assessment, confidence, impact — that require further definition. Those are explained in my book, and in my articles, but I think you can fill the form even without knowing these formal definitions.
Naturally GIST is not the only evidence-guided model out there, but I’d argue that a ) these questions transcend frameworks and models b) they cover the most critical areas that cause us most to succeed or fail.
How to use the results
No one gets a perfect score in this test. Every company has some gaps, and sometimes that’s perfectly fine. But I find most companies have very deep issues that cause them to go down the wrong routes and waste a lot of time and resources. The scores should help you see where your biggest gaps are.
The good news is that you don’t have to close all the gaps at the same time. If, say, you realize that you’re building too much and experimenting too little, you can start at the Steps layer (in the language of GIST Steps is a general term for any type of assessment, fact finding, or experimentation activity that helps us validate the assumptions in an idea). You can start adopting earlier, cheaper forms of validation, and structuring your projects around discovery. If your main challenge is in the Goals layer you can start by creating a model using the north star metric and metrics trees. You can tackle the most important gaps first, even if it’s out of order.
The scorecard however should not be used as a guide on how to close the gaps. You’ll notice that most of the items are about output, and expressed in a very general way. You can measure progress using the tool, but I’d use other metrics for goals, for example number of ideas tested per quarter, or time spent on planning. The real purpose of the tool is to drive discussion: how well are we doing? Where should we improve first? What can we do about it? You can find answers to this last part in my book, that covers the GIST model, and in many other good books and resources out there. It’s also likely that your organization already has some processes or frameworks in place, which form a starting point. Sometimes improving those to work as they should is all you need to do. But often the change will go deeper and broader.
Some senior product leaders that used the GIST Scorecard had told me that this, more than any of my articles and resources, helped helped them realize that their org needs to improve. So it’s worth using it in your company as a way to trigger the important discussion about evidence, experimentation, product discovery, and their place in your product development processes.
Photo by Evgeni Tcherkasski
