Beyond Productivity: Gen-AI’s Deeper Use Cases

By now you must have heard an awful lot about how gen-AI will “10X” tech workers’ productivity and massively cut development costs. This promise is music to the ears of executives and tech-eutopians, but currently is not backed by any rigorous research (most of the evidence comes from coding competitions and reported usage — mostly from AI companies). But there may be even a bigger elephant in the room. In reality most tech companies see meddling results not because they are coding too slowly, but because they’re betting on the wrong things. As I tried to demonstrate in this analysis, an improvement in idea success/failure ratio can massively help business and user outcomes, while ramping up productivity alone may dig you deeper into the hole. 

In this article I will discuss some ways gen-AI can help you improve company outcomes. As the technology is still nascent, these ideas are quite speculative, but based on discussions with people and companies working on them they seem plausible and promising (which is why I started incorporating AI into my product management workshops). 

Cognitive Offload

There’s no shortage of product frameworks to help you achieve better results, but they all suffer from the same fundamental flaw: they require people to stop their flow of work and think. Activities like analyzing data, developing hypotheses, identifying opportunities, projecting impact, and making evidence-guided decisions are time-consuming and cognitively-hard. In my experience, companies and people are reluctant to spend the time and effort required. Instead they fallback to easy heuristics: What feels right? What do the executives think? What’s the consensus? What did the customers ask for? These mental shortcuts may save time and reduce cognitive load, but can also lead to very sketchy decisions. 

I found gen-AI to be very helpful as a thinking aid, and on some cognitive tasks it outperforms most humans. For example LLMs are better at developing back-of-the-envelope models for impact estimations than 90% of PMs that attend my workshops. They’re also helpful at building metrics trees, inventing objectives and key results, brainstorming opportunities, mapping assumptions, creating business models, and any number of other product practices that require deep thinking. I can see a future where AI aids us with deep cognitive tasks (analysis, logic, estimations), just as calculators and spreadsheets are helping us with arithmetic calculations.  

Does that mean you can trust LLMs to do all the hard thinking? Absolutely not. Like with all things AI, the output may be generic, naive, or plain wrong. Quality heavily depends on the context you provide and how well you prompt. It’s paramount to combine human intelligence (HI) and artificial intelligence. In my workshops we see that on some tasks it’s good to let the AI take the first stab and then review and refine, while on others it’s better for humans to first do the process, and then have AI critique or fill in missing gaps (on some things it’s better not involve AI at all). Either way AI helps do the mental heavy lifting, removes friction, and saves time — a huge benefit for thinking-shy orgs. 

Collective Memory

A less-talked about feature of gen-AI is its remarkable ability to interpret natural language. LLMs not only understand the words, but also the meaning behind them as well as higher-level concepts such as irony and humor. This means that we can now ingest and process spoken and written data of any type:

• External: customer calls, user feedback, survey answers, competitor pages, industry research reports, social media messages, … 
• Internal: documents, emails, Slack messages, meeting transcripts, user research  

Imagine having an AI-powered system that can process all this qualitative data, synthesize it into its core messages, label, and categorize those, and store them in a central data warehouse, alongside quantitative data. For the first time, qualitative data will become an equal-rights citizen. The potential is massive — here are some examples: 

• Analytics++ — Charts, reports, and data lookups will show both what people are doing and why they’re doing it 
• Information lookup — A customer once mentioned a desired feature in a call? We got some interesting insights in user research two years ago? A recent industry study includes some useful charts? Now all of these will be at your fingertips, and can be pulled in real-time while you’re in a meeting or writing an email or a document. 
• Hypothesis validation — you may ask the system what evidence exists in support of a particular product idea or set of hypotheses, and what’s the level of confidence. 
• Fact checking — In internal memos and discussions information can easily be taken out of context or shared partially, leading to wrong conclusions. AI can play the role of the impartial fact-checker, suggesting corrections, adding context and linking to the original source of the data. 
• A knowledgebase of learning — Historical data may show us what worked and what failed, what we learned from experiments and from research. AI can keep those persistent in the orgs collective memory. 

A big caveat is that Gen-AI is not designed for accurate data storage and retrieval. LLMs notably suffer from confabulations (aka hallucinations) and may produce false information to fill in gaps in their “memory” (humans do too, but we seem to be better at knowing when we don’t know). So gen-AI itself is likely not going to be the storage mechanism. There are also likely privacy and legal concerns to resolve about processing and storing everything people say and write. 

Knowledgeable Advisor

Lack of knowhow is often holding organizations back. I regularly see misuse of OKRs, incorrect idea prioritization with ICE, bad A/B experimentation practices, and more. Unless your company is big and rich it’s hard to build all the expertise in-house, and often there’s a lot of nuance that comes only from experience. The in-house expert may also lack political power to stop executives from subverting the process. Outside consultants and coaches can help, but they’re expensive and can’t be there 100% of the time. 

It’s possible that AI-based systems will step in to guide organizations, and correct execution mistakes. Already many tech professionals are turning to AI for advice and guidance, however in my experience, foundation LLMs such as the ones developed by OpenAI, Google, and Anthropic give generic advice based on what the Internet thinks is the best practice, which often misses the mark. Some product experts are experimenting with developing fine-tuned AI versions of themselves, but this idea is still in its infancy, and I’m not aware of any successful implementation yet. 

For better or for worse, AI is likely to become a source of org expertise. Still, human intelligence will be needed to adapt the advice to the context and realities of the company, and to update the best practices. Good human consultants/coaches/advisors will still add important insights and value, but may find ways to use AI to extend their presence. 

Culture Analytics

Org culture is a mostly hidden function that plays a major part in success and failure. Good product companies strive to make culture explicit and dynamic; they set targets, run experiments, and measure progress (culture as a product). However, the main tool to measure culture today is employee surveys. With the Collective Memory data system mentioned above, we may be able to do true culture analytics. For example:

• Average number of people required to approve decisions 
• Number of top-down directives this quarter
• What % of goals are about business results vs. user value 
• % time spent in meetings

While these may seem less important than business metrics, I assure you the two are tightly linked, and companies will do well to start measuring and setting meta-goals for culture metrics just as they do for business metrics. 

Experimentation Aid

This is the one most people are talking about. With AI “vibe coding” we can develop prototypes to use in experiments quicker and cheaper. However, in my experience most teams don’t experiment enough not because of the cost of development, but because they’re not motivated to experiment (often due to company culture), they lack knowhow, or they can’t measure the results. So while the ability of a PM to develop a prototype with no coding expertise is great, improvements to culture, knowhow, and analytics may make much more of an impact. 

Final Thoughts

Everyone seems to be consumed with the prospect of AI doing what humans already know how to do, just faster. For me, the much more interesting prospect lies in AI helping organizations do what they struggle with — think deeper, make better decisions, collect and analyze data of all kinds, implement frameworks and processes correctly, and improve org culture. Beyond initial anecdotes I have no proof that AI is capable of doing these things, but I’m much more excited about the potential than I am by AI coding. I feel we’ll do well to focus as much attention on these important directions as we do on boosting productivity.