These days everyone seems to be talking about transformation, product-based development, and discovery. And yet, I find that many people support these concepts in principle, but can’t truly believe they can work in their companies. They see them as theory, something very futuristic, hard to do, and abstract.
I disagree, but instead of giving you more theory I’ll use an example story. It’s fictional, but inspired by real-life events.
Blue corp is an e-commerce company selling computers and other hardware to small-medium businesses. It has been in business for well over a decade, but is struggling to grow in the face of a very competitive market. One end-of-quarter the business intelligence team circulates an important finding: only 10% of first-time purchasers ever make a second purchase. This statistic becomes a topic of much discussion within the leadership team. Someone brings up the idea of a loyalty program, a common practice in the industry, but one that Blue corp lacks. A quick back-of-the-envelope calculation suggests that offering first-time purchasers who join the program 10-15% discounts on their next two purchases will boost second purchases and more than double customer-lifetime value. The decision is made to pursue the loyalty program with high urgency.
Next, let’s see two possible follow up scenarios.
Scenario #1 – The Big Launch
The project is assigned to a product team. An OKR is drafted and planning kicks into high gear. The team is positive it can launch the loyalty program within a quarter. The first milestone is an MVP to be tested with 5% of customers in two-months’ time. Everyone drops what they’re doing and starts working on the new project. The engineering team decides to create a general-purpose promotion system that will support this specific program and any future one. The PM and UX designer define detailed scenarios and special cases, and design new UI to address them all. The marketing team plans major promotions to run across the site, in the app, in email, and in social media.
To cut a long story short, the team misses its MVP target date by two months. Finally they run the MVP (which isn’t minimal in any way) in an A/B test for two weeks. This is not enough time to see changes in second-purchase behaviour which usually occur in cycles of months. However there is an uptick in account signups, likely because of the bold banners that the test group now sees. It’s concluded that the experiment is a success and the team goes on proceeds with final polishes and a launch. The project took nearly 6 months to complete, but everyone is pleased to see it out. A few people are promoted on the back of this success.
Six months later the CEO asks about the impact of the new loyalty program. The business intelligence team comes back with the data. So far there’s no measurable difference, but this could be due to seasonality. They recommend waiting a few more months. Eight months later the topic comes up again. A data analysis shows that there was no discernible improvement in repeat purchases after the launch of the loyalty program. It does seem, however, that the loyalty program is costing the company money because of the discounts. The leadership team discusses the findings and concludes that the program is not well communicated to new customers. The marketing team gets the task to fix the issue, but it is repeatedly deprioritized in favor of more urgent things, and is eventually forgotten. As the years pass and people come and go, the loyalty program becomes a permanent part of the product. People assume it’s necessary, and there’s never any attempt to challenge it.
Join thousands of of product people who receive my newsletter to get articles like this (plus eBooks, templates and other resources) in your inbox.
Scenario #2 – Discovery
We rewind the clock to the time the team gets the loyalty program mission. While other team members work on other things, a sub-team maps the assumptions in the idea, and then goes on to do the following:
- A user researcher interviews 14 digital equipment purchasers that match the target audience. They’re asked how they pick a vendor to buy from, and what their experience with loyalty programs has been. The results are interesting. The buyers are looking for good prices, but even more for good service, before, during and after the purchase. They also appreciate vendors that offer good, unbiased information that helps them make purchase decisions. Loyalty programs are not in anyone’s top reasons to pick a vendor. The buyers vaguely remember having signed up for those, but they can’t remember where and with what benefits. Generally they consider loyalty programs a marketing ploy better suited for airlines and credit cards.
- The team launches an A/B experiment showing a loyalty program banner to 1% of the site’s visitors and comparing their behavior to a control 1% that don’t see it. The variant group showed a slight improvement in account creation, but they didn’t spend any more money. The team sends to those customers that signed up an email with two discount coupon codes. In the email they also include a survey. They get about 40 responses. While everyone appreciates the discounts, few have firm plans to use them, and some explain that this “nothing special” because they’re always able to find the products they’re after at a discount somewhere.
- The PM carries out a competitive analysis. She creates accounts on the sites of 15 leading competitors. Only four have loyalty programs. Those that do don’t seem to promote them very much and the user interface looks like it hasn’t been updated in years. She also notices that the upfront discounts somehow make these vendors feel less trustworthy.
Five weeks in, the findings are shared with management, with a recommendation to stop the loyalty program project. The team concludes it’s unlikely to drive growth in second purchases. They share six other ideas for boosting second purchases they were able to uncover during their research.
Here, we come to another fork in the road — management’s reaction. I’ll describe two potential sub-scenarios.
We learn in-depth how to implement product discovery in my Lean Product Management workshops. Secure your ticket now for the next public workshop or contact me to organize an in-house workshop for your team. For more info see itamargiladcom/workshops.
Sub-scenario #2.1 – Just Do It
Senior leadership is surprised and unhappy with this development. They expected execution, not research. The CEO is livid — he already presented the loyalty program to the board as a key initiative, coupled with optimistic growth projections. He goes on to give the CPO something of a public dress-down for what he calls “a waste of time and resources” (within two months the CPO is gone to another company; over the next 12 months some of the most talented PMs, engineers and designers follow suit). The team is told in no uncertain terms to build the loyalty program as envisioned. They proceeds to execute scenario #1.
Sub-scenario #2.2 – Learning and Course-Correction
The leadership team discusses the evidence shared and the recommendations. They agree to park the discount-based loyalty program. In the next all-hands meeting the CEO commends the team for its work and encourages others to learn from this example. The managers still see increasing second purchases as a key goal. They create a new cross-functional team composed of representatives from engineering, design, product management, marketing and sales, to own this goal.
The team goes on to conduct a series of experiments based on the initial research. Most ideas don’t pan out. Some create incremental improvements and are fully-built and launched. Finally the team strikes gold. They uncover a market segment that is poorly served at the moment — design studios that rely on powerful computers to do 3D modeling, photo and video editing, and other design work. These SMBs are willing to pay a premium for high-end computers and peripherals and often have a special affinity to Apple products. The buyers currently have to spend much time researching hardware options and comparing prices, and often have to buy from a variety of vendors.
The team experiments by creating a sub-site dedicated to design studios that offers subject-specific information, special hardware and software bundles, and dedicated support. The response is very positive. While minimal and unpolished, within months the studio sub-site becomes a major contributor to visits, purchases, and revenue. As the company doubles-down on its studio-specific offerings and adds a community dedicated to these buyers, word-of-mouth brings in new studios as well as independent creators who share many of the needs. Within a year Blue corp becomes the go-to destination for creators and studios seeking hardware. Growth accelerates sharply. Eventually the executive team decides to pivot the entire company and focus entirely on this lucrative market segment.
Which Company Do You Want to Work For?
Evidence-based product development is not rocket science. Leadership (at all levels) is about choosing goals, not ideas. Execution is about progressively moving towards the goals by considering multiple ideas, testing and learning. This method is far more effective when you face uncertainty (and we all do, even if you’re in a business that has been around for decades). It’s also far quicker and more efficient. Just as important, investing in learning uncovers opportunities that can take you to new places.
It’s an unintuitive change. It feels risky, even wrong. It’s tempting to half-commit, to invent ways to marry the old with the new. But that just gives you the worst of it — more process, but none of the benefits. To reap the rewards of product discovery you need to believe, commit, and let go of the old ways. That’s the true meaning of transformation.
Photo by Siora Photography on Unsplash
