This is an excerpt from my book, Evidence-Guided: Creating High Impact Products in the Face of Uncertainty. While the chapter references the GIST Model (Goals, Ideas, Steps, Tasks), the methods described can be used broadly.
Switching to an evidence-guided mode of work is rarely easy. The people developing the product and influencing it have to change the way they work and think, often in counterintuitive ways. Your managers, stakeholders, and team members may resist the change, and for good reasons. They are entrusted with important responsibilities and they have things to protect—revenue, budgets, customer relationships, product quality… People’s reputations and careers are also on the line.
In my experience most people will be supportive of a model like GIST once they undergo training, and they understand what the change entails, and how it is going to help the company and their area of responsibility. Still there’s a big difference between supporting a big change in principle and actually going through with it. You should expect to run into adoption hurdles and objections. In the first part of this chapter we’ll go over the most common challenges, including mistrust, lack of time, and lack of resources, and discuss ideas how to overcome them.
Even when everyone is onboard, the transition may run slow and be hard to complete. Some organizations take a very long time to implement GIST, and lose momentum midway. Others try to do too much too fast and get overwhelmed and fatigued. Yet others pick and choose, eventually creating just a slightly more sophisticated version of the old system. Personnel changes and reorgs bring new managers and employees into the fold and can cause regressions and setbacks.
The second part of this chapter covers change patterns that will smooth out the transition and give you a better chance of success. To make things more concrete I’ll talk specifically about adopting the GIST model, but the guidelines and principles apply broadly to other changes you might strive to drive.
[some text skipped]
Driving the Change
Gradual Rollout—Start Where the Pain Is
Some companies are in a rush to implement GIST all at once. I often recommend against this. Too many things will need to change in parallel and the ramp-up curve may be too steep. If you have a big gap to close, I suggest staging the adoption. GIST is a modular system—you can implement certain layers first and others later. So start by looking at the four layers of GIST and consider where the biggest challenges in the company lie today.
- In companies where the direction is unclear or constantly changing, where there are big challenges in alignment, or where focus on users and customers is low, I recommend starting by implementing the Goals layer of GIST: the North Star Metric, outcome roadmaps, and outcomes-based objectives and key results.
- In organizations that spend lots of time debating ideas, where ideas are chosen based on opinions, consensus, or weak evidence, and where there’s a track record of investment in bad ideas, I recommend introducing the Idea layer first—idea banks and ICE scores, often coupled with beefing up user/market/tech research.
- In companies where projects are long and expensive, where little or no testing and learning takes place during the project, and where results are generally disappointing, I suggest adopting the Steps layer first. Build learning milestones and customer feedback loops into projects, utilize the AFTER model, analyze results objectively, and be bold about pivoting or even stopping projects that don’t work.
- If, in your organization, development teams seem submerged in their own world and disconnected from the needs of users and the business, engineering and design projects take an inordinate amount of time, or you have to create mountains of specifications and user stories, I would suggest using some of the techniques we saw in Chapter 5, especially the GIST board or a similar project tracking tool that gives the team a broader view and makes it an active contributor. This will entail introducing some experimentation (steps) too.
Obviously, the four layers of GIST are all connected, but to get started, a North Star Metric is the only bit of plumbing you must have (and is therefore the first thing to figure out). The NSM will enable you to rank ideas, evaluate steps, and build early GIST boards where sometimes the only goal is to grow the NSM.
Set Ambitious-yet-Realistic Goals
No one can turn into a marathon runner overnight or shed 30 pounds within a week. Success comes from setting ambitious-yet-realistic targets and sticking to them. The same is true for your adoption of evidence-guided thinking. During the transition period you’ll need to work on two types of outcomes at the same time:
- Business/user outcomes (like the ones we saw in Chapter 2)
- Self-improvement outcomes that describe the change you want to create at the end of every quarter and year
What are these outcomes? We can get an idea by looking at the core operation of GIST:
Based on this picture, these metrics can help you calibrate your progress:
- The total number of ideas evaluated per quarter (using, at minimum, ICE analysis and goals alignment)
- Number of ideas tested per quarter
- The number of ideas released per quarter
- Total number of tests and experiments conducted per month
- Percent of steps that generated learning (i.e., where we were able to rescore the idea and/or generate useful insights based on the evidence collected)
- Percent of ideas launched at least with a medium Confidence level (per the Confidence Meter)
- Percent of ideas released that generate measurable outcome improvements
The first four metrics are aligned with Linus Pauling’s observation that the way to have good ideas is to test many ideas. However, these are outputs, and may be meaningless unless we’re able to learn and create outcomes, which is what the latter three metrics measure. These seven core metrics should give you the main outcomes to work toward. If you’re only testing three ideas per quarter, push yourself to test ten in the next one. If only 10% of ideas hit medium confidence prior to launch, try to reach 20%, then 30%, and keep improving.
Other things you may wish to measure:
- The ratio of outcomes-based goals out of all goals
- The ratio of teams that develop their own goals
- Ratio of employees reporting that they are working toward clear outcomes, and have the necessary context to understand why these outcomes are important
- The ratio of ideas that stem from outside research rather than inside ideation
And all of these metrics should ideally go down:
- Time spent on planning
- Number of sales escalation or management mandates per quarter
- The number of output-based incentives
- % ideas slowed down due to dependencies or processes
- Average time to launch an idea
You may wonder what you should aim for. There’s no fixed number I can give you, and the goal should be to continuously improve (more on this later) rather than to hit a certain fixed goal and stop. Still, if after six months of using GIST, you only manage to evaluate on average five ideas per month, and test one, you probably have some way to go before you start seeing the benefits.
I would start measuring the key metrics right now, before you’ve implemented GIST, so you’ll have a baseline to compare against. The current numbers may be depressingly low, but this is normal. Use this fact as evidence in your discussions with management and stakeholders for why you should adopt GIST.
Make Incremental Improvements
Many big changes come from a series of incremental improvements rather than one big leap. In Chapter 7 I told you how between 2008 and 2011 Hewlett Packard’s LaserJet Firmware division, consisting of 400 employees across multiple offices, had managed to rewrite its platform, introduce automated testing, and launch continuous integrations through a series of four-week iterations each with its own target set of outcomes (for example “P1 issues open < 1 week”) [1]. The book Toyota Kata recounts a similar pattern of continuous self-improvement that stands at the heart of the famous Toyota Production System (TPS)—the principle known as Kaizen.
The incremental improvement approach can work for adoption of GIST as well. Instead of waiting for the end of the quarter to measure progress, the product organization can set quicker cycles of self-improvement in which it tries to boost idea evaluation, experimentation, team autonomy, or anything else we identified as a key goal. The shorter cycles make the change concrete and less intimidating. Instead of trying to climb the entire mountain, we choose a much lower target we can potentially achieve by next month. Just the act of trying (and often failing) provides real evidence, helps surface core issues, and motivates good discussions. Over time self-improvement becomes a habit of the product organization or even the entire company. With continuous iteration and learning, success is bound to arrive, first slowly, but then at an accelerated rate. When it does it should be shared and celebrated broadly.
[1] As reported in the book Lean Enterprise: : How High Performance Organizations Innovate at Scale by Jez Humble, Joanne Molesky, and Barry O’Reilly
Image credit: Alessandro Simonetta on Unsplash
