This post is the summary of a talk I gave throughout 2017 at a couple of product management conferences. I was on stage at Working Products 2017, Productized Conference 2017 and Product Management Festival 2017.
This post will touch on the learning points of my talk. Head over to SlideShare for the full presentation and visit the overview of my public speaking activities for the video recordings.
What’s wrong with MVPs in the first place
The most significant misconceptions about MVPs I’ve encountered throughout my career are:
- They’re crappy versions of all planned features…
- …but are good to live on after launch anyway
- They’re the cheapest way to validate critical hypotheses
Let’s try to rephrase the meaning of an MVP a bit:
“An MVP is about building the most critical value proposition […] to further prove your product idea’s potential and product market fit and shipping it in the best possible quality […]. It is not about building slimmed down or extremely compromised versions of all your features […].”
So, what are the actual costs which come with MVPs? There two kinds of expenses here: The cost of waiting (caused by you) and the cost of building (produced by your team).
Mainly the cost of building is caused by a rushed transition of product discovery activities into the delivery track. This is because of management most often (only) cares about the output from the discovery track of product teams.
Another problem we see here is the moment when real validation happens when building and launching MVPs.
Why it’s essential to validate ideas before you start building
There’s a simple reason why you need to prove hypotheses and ideas before starting to deploy your team against it. And this is due to a shift in how resources within companies (of every size) are allocated:
The old way of thinking about resource allocation:
“This sounds like a great feature idea based on what I think to spend time, people and money on!”
Compared to a new way of thinking about it:
“Where’s the proof that this is a good opportunity to spend time, people and money on?”
The four most significant validation mistakes I see people making trying to follow that paradigm shift:
- Falling for Confirmation Bias
- Setting up only one hypothesis
- Only validating indirectly
- Picking the wrong method for the right questions
So, which methods are there at all to tackle validation? In the end, it boils down to two critical areas of validation: Qualitative and quantitative validation.
Qualitative (or also attitudinal) validation answers questions like ‘What do People need?’ And ‘How are people solving a problem?’.
Quantitative (or also behavioral) validation on the other and answers questions like ‘Do people want the product?’ or ‘Which design works better?’.
But there’s another complexity factor coming into play for your validation efforts: How close your most crucial hypothesis is to an upsell moment. The moment of truth when people have to enter their credit card data to pay for your product.
How to validate hypotheses in an upsell context
You can’t use qualitative methods to validate upsell hypotheses because purchasing something is so profoundly psychological to users (‘Do I want to spend $10 on this instead of Netflix?’) that it’s impossible to simulate that.
Also, there are circumstances in a qualitative validation situation; you just can’t control or bring any closer to reality. Examples are personal sympathy and the missing need to think about trade-offs.
The worst you can do is to ask ‘Would you buy that?’. So, when qualitative methods are off the table, what to do instead? Easy: Just use quantitative frameworks like fake e-mail testing, a concierge MVP or good old smoke tests.
Here’s what you should pay attention to when you’re validating in an upsell context:
- Don’t use qualitative methods to validate purchase-related hypotheses.
- Don’t confuse validated engagement with real willingness to pay. Just because people use your product, you can’t be confident that they will pay for it down the road.
- Fake the payment moment to make it as real as possible.
- Prepare vouchers and pre-order discounts as compensation. 😉