Why Doing Product Evals Made Me a Better PM
Read this if you’re trying to build taste, sharpen your product sense, & stand out in a world where the traditional PM role is getting outsourced to AI. Plus, a prompt to help you run your first eval
Hi 👋! You are reading one of my free articles - I hope it helps you level up your PM skills. Want to go deeper? My premium subscribers get exclusive frameworks and strategies I don’t share anywhere else - content that has helped hundreds of product managers land better roles and make bigger impact.
Ready to accelerate your product career? Subscribe now for premium access and join a community of exceptional PMs gaining an edge every week.
Now, let’s dive into today’s insights…
When I first became a PM years ago, the widely acclaimed mark of a great PM was output. The number of features shipped. The velocity of the team. The perfect spec, the sharp deck, the tightly run meeting.
That was the approach many of my peers took to product management.
But over time, after building across massive platforms, hundreds of millions of users, leading teams, mentoring sharp PMs, and sitting across the table from hundreds of candidates, I noticed something that changed everything for me. I observed that the most effective PMs aren’t just efficient. They are largely observant and perceptive. They see what others miss. They anticipate user behavior. They reverse-engineer intent and can notice the asymmetry, which helps them form a sharp point of view and bet on it.
Without a doubt, it wasn’t shipping velocity that set them apart. It was sound judgment. Taste. A strong conviction of “why this” and “why now”.
That’s when I started writing Evals.
What you will learn from this article:
If you are new to Evals, or wondering why I have made them a cornerstone of this newsletter (and now, my coaching practice), I will discuss the following to help you distill my thoughts:
Why I started writing Evals in the first place.
How they helped me 10x my product sense.
Why I now ask for Evals in senior PM interviews.
How AI is changing the PM skill-stack and why Evals are future-proof.
And how you can use Evals to level up, build signal, and get noticed.
Let’s start from the beginning.
The Origin Story: From Strategic Execution to Sharper Judgment
I have spent years deep in the weeds, shipping complex products, scaling systems, leading teams, navigating ambiguity, and driving results. Strategy wasn’t a buzzword for me; it was the job. Execution wasn’t the final step; it was the rhythm. A proper song and dance.
But even with all that experience, something kept pulling at me. I noticed that the PMs who stood out the most, the ones who shaped organizational direction, earned trust fastest, and consistently made the right bets, were more than just good executors. They had incredible clarity of perception.
These individuals could look beyond the surface of a product. They could feel when something was off, even when metrics looked fine. They could reverse engineer a company’s bet just by scrolling through a feature. I do not think their edge was in how much they knew, but more in how sharply they saw.
That moment changed how I saw growth, both mine and the teams I was leading.
So I started deliberately building the muscle- One product at a time. One Eval at a time.
Why Evals Are a Cheat Code
A good Eval flexes four key muscles at once:
Product sense: spotting patterns, UX gaps, and strategic intent.
User empathy: stepping into the shoes of your target users.
Strategic thinking: seeing the “why” behind the build.
Storytelling: clearly articulating your point of view.
And unlike theoretical frameworks or PM books, Evals are a direct result of having multiple reps in the real world. They are anchored on experience, very specific and evergreen.
When I started writing these and sharing them privately with friends, I didn’t expect much. Then someone asked me to share the one I did on Threads, and that’s when it took off. A few hiring managers on LinkedIn reached out to reuse the framework in interviews. People weren’t just reading the Evals, they were practicing them.
AI changed the game, but not the fundamentals
In a world where everyone can ship faster, thanks to AI agents, code copilots, and automated research, what starts to matter more is direction. As a result, I believe Evals are how you train your anchor.
Anyone can ask an LLM to “write a spec for a shopping cart.” But only a sharp PM can say: here is why this feature matters now, for this user, in this market, and here is what success looks like. Heck, even spell out the risks, the real market patterns, and the gaps.
Often, I get questions about how to improve “taste” as a product manager. I think evals will teach you to look at products with a critical lens. To train your taste and to form a view before ChatGPT gives you one.
This is what great PMs do. Not just in AI. Not just in big tech. But anywhere vision and judgement are scarce and critically required.
Evals in the Wild: Hiring, Coaching, and Getting Noticed
I now ask senior PM candidates to do a Product Eval as part of interviews. Not to see if they are right, but to see how they think. How they break down ambiguity. What they notice, and what they ignore.
I have also prominently been using Evals in coaching sessions, helping PMs diagnose what’s working in their own products, and sharpening the way they talk about it. And I’ve seen people land new roles just by sharing a sharp Eval publicly. It’s one of the clearest ways to show you have signal and not just noise.
If You Want to Get Better at Product, Start Here
So, here’s your prompt:
Pick a product you use every day.
Write down what’s working, what’s not, and what you’d do differently.
Make guesses. Be wrong. Practice seeing.
I need you to ignore every template. Resist every opportunity to have a set of prompts. You just need curiosity and a willingness to think like the team behind the screen.
And if you want to do it alongside other sharp PMs and builders?
Join the Eval Lab (aka: my Substack Chat)
I have opened the Substack chat to all subscribers.
It’s a space to:
Share your own Evals
The workshop product takes
Learn by seeing how others dissect the same product
I’ll be jumping in, sharing highlights, and featuring the best ones in future posts.
If you’ve been reading quietly, now’s a good time to raise your hand.
The best way to sharpen your product sense… is to use it.
ICYMI:If you’ve read this far, you probably now understand how strongly I believe product evals can set PMs apart in an increasingly AI-driven world. These past issues are here to help you further sharpen your product sense and give you a solid starting point to run your first eval with confidence. Dive in, write your evals, share publicly and tag me when you do.
I figured, what better product to run my first in-depth eval on than the very tool being used to power most of today’s product work? In this piece, I break down what ChatGPT gets right, where it falls short, and what PMs can take away from it. 👇🏾
If you’re a PM, running product evals is one of the best ways to prep for exploratory interviews. This piece takes a close look at the Threads app, showing where product thinking shows up clearly, where it falls short, and what you can learn from it. It also includes a 7-step framework to help you run your first eval. Read it here.👇🏾
