MVPs are about prioritized Depth, not crappy Width

Even though product people now already start arguing about whether to focus on the most riskiest assumptions instead of a viable product, the good old term MVP is still a good conversation starter. Especially within companies which may not be exactly ‘cutting edge’ when it comes to product development, the intent to launch earlier and ‘leaner’ poses a huge mindset shift.

The biggest push back I often hear about the MVP approach of a new product is the assumption that basically a shittier version of the overall vision gets shipped. This fundamentally wrong misconception then leads to huge concerns about the negative impact of a broken product for the brand which then leads to a ‘build everything and build it perfect’ attitude before any iteration of a product idea gets into the hands of actual users.

In order to condense the actual intent of an MVP in the context of this wrong perspective on the subject, I liked the following conclusion the best:
An MVP is about building the most critical value proposition (meaning feature) to prove your product idea’s potential first and shipping it in the best possible quality to your target users. It is not about building slimmed down or extremely compromised versions of all your feature ideas and bringing those to market.



While probably every product manager out there enters compromises as part of her regular product iterations, those are not meant to decrease a product’s quality, but are the result of necessary trade-off decisions to make the biggest impact within the (given) constraints of time or resources.

A great visualisation of what the difference between a shitty version of an overall product vision and qualitative sufficient iterations of key value propositions can be found at the top of this post.(intellectual property of Spotify).

Hopefully this helps you the next time you are in the midst of a discussion with stakeholders (or even yourself) about what it means to build an MVP after the general intent of a project was expressed and aligned.
The best food for thought from the image above is from my perspective, that an MVP doesn’t even necessarily need to be in the same format or channel of the overall vision. For example, a manually curated newsletter could be the MVP of an app-based community platform (yes, that’s exactly the story of Product Hunt).

What’s your elevator pitch stating what an MVP is supposed to be and what it’s not?

5 A/B Split Testing Mistakes Product Managers are responsible for

A/B Split Testing is the most powerful tool of a product manager for getting from gut feeling and HiPPO driven product decisions to actual progress based on measurable outcomes.
But while it’s comparably easy to asses a tool and implement your first testing setup, there are a couple of meta mistakes I see being made from time to time among product managers (and which I’ve made myself).

Here’s my personal selection and how to avoid them:

Sharing results too early with stakeholders

While ‘too soon’ is measured in two dimensions (more on that below), let’s just says there’s a certain point in time during which you just should not talk about the preliminary results of a running A/B test. Especially when you’re testing around a critical business KPI there are just too many dreams and visions emerging from (mostly non-technical) stakeholders – even though the test result doesn’t stand on solid ground.

In the past, I shared ongoing progress of a test we’ve implemented with my engineering team, in order to make their efforts implementing it transparent as possible. But when it came to upper management and marketing, I learned to wait a lot longer before I communicated a result. Best case is, you get approvals from two expert perspectives: A mathematical one (e.g. Digital Analyst/BI Manager) and a practical one (e.g. Director of Product or Product Management buddy).

A/B Testing for the sake of it

While A/B testing is something to aim for because of the above mentioned reasons, you shouldn’t run any testing idea which is floating through your regular stakeholder alignment meeting. While the costs of implementing and cleaning an A/B test up are important, it’s more about the environment you want to test in.

Do you actually have enough traffic flowing through your area if product in order to generate significant results within 4 weeks? Do you have the freedom to test variants which are differentiated enough from each other? Do you have the commitment from upper management to also implement the ‘unpopular’ version and they just let you A/B test in the first place because it’s a ‘modern approach’?

Solely relying on Significance Calculators

As I wrote earlier, the mathematical expert view on your test setup and results is a crucial one. But don’t let numbers alone influence your decision when a test is finished. You’ll run into scenarios in which a significance calculator will spit out a significant result after two days or so. But seriously, two days? Common sense should tell you to at least run a test through every day of the week twice in order to balance-out ‘seasonal’ impacts.
Also, significance calculators don’t know every twist and angle of your product and its dependencies like you do. So only you will be able to spot outliers and potential bugs influencing your testing setup.



 

Confusing A/B Test results with user feedback

‘Version A won, so our users like this one more!’ While this assumption doesn’t seem far-fetched, it’s quite misleading. It’s the perfect example why quantitative and qualitative testing (and research) need to go hand in hand for a complete picture.
Even though A/B testing tells you what users are doing (like clicking 5 times more likely on the yellow button instead of the red one), it doesn’t tell you why people are doing it. It’s the unfortunate discrepancy between what people say they do and what they actually do.

So, be happy about the measurable uplift your latest winning version has caused, but make sure to check back it’s mid- to long-term impact on your product by conducting some 1:1 user interviews digging into what users actually made them click more. E.g. in the worst case you just locked the users tighter into a checkout process and they had no other choice then to proceed. Again: Correct action to impact your target KPI, but it’s helpful to know the context around it.

Focussing on the wrong KPI to influence

Speaking of KPIs: When setting up your test, it’s important to only work towards an increase or decrease (e.g. when working on a churn topic) of the very metric you can directly influence with a test. Especially when implementing a test within a multi step funnel, you can never be sure to impact the very last checkout metric when only testing improvements within the address data step.
It’s intriguing to assume that all conditions before or after your area of testing stay the same, but then again you perform an A/B test to stop assuming, right?

What were you’re most impactful when designing, implementing and analysing A/B tests? Personally, my biggest learnings not only originated in practical doing but also tons of support from my former colleagues Patrick, Timo and Gudrun.

Why you always need a Plan B when you’re validating

As a posterboy Product Manager, you of course always include a ‘Validation’ phase in your outlined product discovery. Being lean-as-fuck, you aim for a fake e-mail test or a simple teaser in your existing product in order to validate customer interest of your killer feature idea. But wait, you’re not going to meet the conditions you need in order to apply your kick-ass validation idea? Puh…then I guess it’s straight do development, right?
Well, even though this story sounds a bit over the top, it provides me with the perfect trigger, why you should always have a plan b (in fact, multiple plans) in place when it comes how to validate during a product discovery.

While validation in general is broadly used, I personally like to differentiate two types of validation: Validating the user need and validating your problem-solution fit. Both kinds of validation feature opportunities to either use quantitative or qualitative methods.

So here’s some food for thought about to get creative when your default methods can’t be applied:



  • When you’re building a new product, is there a similar product sharing some characteristics with your idea live?
  • If so, are any analytics insights available?
  • If not, is quantitative data available?
  • If not, can you talk to the users of this existing products now or send them a survey?
  • When you dive right into execution, make sure to bring contrasty versions of your solutions to your usability testings.
  • If you can’t reach the users from within your target group face-to-face, reach out via e-mail and combine questions with wireframes of your solutions (make sure to randomize the order of your displayed solutions).
  • In case you don’t have access to users from within your target group at all (likely in the context of b2b or internal products), try to recruit users with common characteristics and talk to them face-to-face.

Even if you now roll your eyes and have heard some of these approaches before, I hope to at least inspire you to come up with creative alternatives to validation again and again and again…

The best Alignment Tools for Product Managers

Generating and keeping Alignment across all levels of an organization – from C level until the single engineer – is from my perspective one of the most important (and often times most challenging) for a Product Manager.
And while often times suggested simple change of habits like ‘talk more to each other’ or ‘keep a confluence page about what has been agreed’ are right, they don’t necessarily provide the right level of confidence to a Product Manager to strive at his mission.

That’s why I picked 4 alignment tools for Product Managers for you to grab for your next product strategy session or product discovery kick-off. Going too much into detail about how they’re used would blow the frame of this format:

Auftragsklärung

Luckily, I have been around at XING when this framework was developed. It is a mix of the influence from Stephen Bungay and Christian Becker and has been evolving through internal forces since then. Check out the full guide right here (original link is currently broken), watch my fellow Product Tank Hamburg Co-Organizer Marc speak about it attend the upcoming workshop around Auftragsklärung facilitated by Christian Becker in April at MTP Engage in Hamburg.

Intermission by Intercom

A simple but effective one pager developed from the guys at Intercom. They managed to integrate all the stuff I like the most around product development (right now): problems first, defining specific measures of success, job stories and a clear product scope.
Would love to see some writing on the actual usage from the product teams (and stakeholders) at Intercom in the future…



The Product Tree

The best thing about this format is it’s strong visual aspect. It’s maybe not as sophisticated as the other frameworks above, but it’s focus on feature prioritization is its biggest strength at the same time. It ruthlessly forces everybody at stake to develop a shared understand about what to actually build next.
In addition, its format is perfect for keeping the aligned vision constantly visible in the team space.

‘Working Backwards’ by Amazon

While there’s no official material on this framework, it’s widely confirmed and well-known in the product manager community. It’s the result of Jeff Bezos’ approach to cut subjectivity and fluff of product development. The result is a document which shouldn’t exceed a page and a half in lenght which needs to be provided to meeting participants before e.g. a kick-off meeting for tackling a new product.
Never used it in ‘production’ but will definitely do in the near future.

Which format, framework or result of common sende are you using right now to create and follow-through on alignment within your organization?

3 Common Misconceptions about User Testing

Meanwhile, a lot of company’s building (digital) products have embraced (or at least accepted) the positive impact user testing can have on the product development process. But just because you somehow involve users into your processes, doesn’t mean you’re doing it right.
Personally, I had the luck to get great teaching into the art of user testing from my former colleagues Britta and Anne. And while you can and have to divide user testing into its separate domains (research, deep interviewing, usability testing), I want to point out 3 very common misconceptions which I stumble upon in early discussions.

Focus Groups are the shit!

Nope, they’re not. They’re actually one of the worst ways to get feedback on your product or getting to real user struggles (right behind building assumptions just on your own behavior). When it comes to pointing out failures of a product or negative struggles, people are her animals. They’ll follow the most common opinion in the room and will be too embarrassed to be the first ones acknowledging a mistake or flaw.
Only personal situations and a comfy and trustworthy environment will bring you closer to see the real behavior of people and hear about their true struggles.

If you want to have the room full of like-minded customers of yours as part of your product creation process, I instead recommend the format of Co-Creation Sessions.

Personas are a great way to figure out my users needs

While personas are already quite an improvement for user-centered products, I still think they have a huge flaw: They focus on the wrong things and distract you!

Building up on what the smart guys behind Inside Intercom have written about why Job Stories beat the format of User Stories, I want to apply to the format of personas. Conducting the corner stones of a ‘traditional’ persona (age, hobbies, family status, hopes & dreams) takes way too much time (or is purely based on imaginative stereotypes) and doesn’t give you any tangible entry points for figuring out around which problem your product should be built.
Instead, you start chasing constructed touch points in a user journey far off a users core problem (you’re able to solve). Instead, you should almost entirely focus on the context and core problem which will inevitably lead to a certain ‘job‘ people hire a product/workflow for. After figuring those two things out, the demographic data of your target group becomes highly irrelevant.

If you want to apply a persona-style approach at the beginning of your product discovery anyways, I’m a strong advocate of Proto-Personas. This format will keep your efforts focussed on demographics reduced to a minimum and will include some reality checks.



We’ll just ask them what they want and how much they’re willing to pay for it

While I can’t stand the iPhone comparison when it comes to user research (‘Steve Jobs never asked anybody, whether someone wants an iPhone), there’s a certain point in it: Users are not able to articulate the right solution to their problems. This is why you are a tasked with building products which serve their needs. But in order to get there, you have to entirely focus on problems and not potential solutions, when talking to users before even sketching a first wire frame. Don’t ask them what they want (even if it itches you to shout this question at them after they’ve neglected 6 of your fancy prototypes) and rather focus on what makes them happy or grumpy.
And while you’re at least able to derive some valid qualitative data from talking to users about problems, you won’t be able to when it comes to monetization. Sure, you’ll get close to figure out a person’s most important product attributes which makes them paying (e.g. quantity over quality or cross device availability), but whether they’ll pay in the very moment for your product needs to be determined outside of that interview room (I’ll talk about quantitative validation at a later point).

Which misconceptions about user testing are you giving you a hard time to get out of people’s heads?

Frequently asked product questions when building native mobile apps

Since last week’s column was the first with real ‘hands-on’ advice, I recognized that a lot of you out there are battling with quite some operational questions around building native mobile apps.
Based on the feedback I received, I wanted to answer some of your more popular questions directly in this week’s newsletter. Again, I hope that you find value in it and keep the questions coming!

iOS or Android first?
Totally depends on your target group – start with the OS which is most popular amongst your most relevant group of users (business decision). In general, I highly recommend shipping an Android and iOS version in the end, but starting with an offset of at least one or two sprints.
This way, you’ll only make mistakes regarding general logic/usability and backend connection once.


What’s iteration 0/the first spike in app development?
Very technical, but I always try to get to a first iteration of tangible user value right from the beginning. Other approaches are feasibility checks for the apps most challenging areas (technical aspects). Again: totally valid and probably a better fit for incredibly technical-focussed projects, but I personally like to shoot for the basis of an app from user perspective (e.g. login)


How to come up with a nice app icon?
Get a great Visual Designer and provide her distinct cornerstones regarding your branding and the core job you want to communicate with the app. Great design resources for creating app icons on various platforms can be found right here.


Does an app’s JTBD need to be reflected in its name?
Speaking of jobs. For answering this question, you have to distinguish between two scenarios: apps with an already strong brand and upcoming new apps (which is likely what you out there have as well). In general I always recommend to set an app name based on a users search behavior for completing a task/job. This is not only relevant for the App Store itself, but also for a quick start of your app later on when it’s accessed via e.g. Spotlight (iOS).
So, while nobody searches for ‘self destructing messages social network’ but goes for ‘Snapchat’ instead, you’re much more likely to get into a users relevant set when including ‘gas station finder’ next to your brand name (a pretty common job and App Store search term).


How to conduct remote user testing?
Personally, I’m a huge fan of invision for assembling mobile prototypes. A neat workaround for walking someone through it would be a mix of those tools (Mac/iOS focussed):

Let me know, how this setup works out for you. A decent and more straightforward solution for Android can also be found checking out lookback.

How to get better at asking for Push Notification Permissions

Inspired by a (German) video my friend Marcel published a couple of days ago, I’d like to talk about push notifications this week. While Marcel focussed on how apps sometimes carelessly misuse the precious push notification permission, I’d like to focus on one step prior to that: asking the user for the permission to send push notifications at all.

Thereby, I want to focus on 4 important aspects of getting this opt in, I often times see just being done incredibly wrong or are just huge opportunities most app creators miss out on.

Timing

You know what sucks? Getting presented with a pop up asking me for a system-wide permission before I even had the chance to get a glimpse of an app’s content. Seriously, this not only reflects bad user experience decisions but also engineering ones. Just because the app has the technical capability of sending pushes, you shouldn’t use the default timing for asking for permission right after the app has been downloaded for the first time.

Instead, bring up the permission after a moment during which the user wants to get actual value from your app and which can then be enhanced by a push notification. For example when a video on YouTube is favorited and you get asked wether you want to get reminded when new videos from that channel get published or not.

Use custom opt-ins for more chances

While you can often times use a ton of marketing mechanics in order to make a bad user experience kind of ‘unhappen’, you can’t do so with push notification permissions. Especially when you’re using the standard iOS dialogue for getting the opt-in, a ‘no’ in that very moment can’t be reversed. You won’t get a second chance to ask again – Even if you considered my tip on timing from above.

This is why I highly recommend building your own ‘pre permission’ dialogue. This doesn’t have to be fancy, but by putting (really low) effort into your own pop up, you don’t miss that one shot to get the opt in should you ask the user in the wrong moment. Instead, it’s up to your (ideally backend-steered) logic when to ask again and again. And only if you pre qualified a users intent for receiving pushes from your custom dialogue, you let her access the official iOS one. You’ll be amazed how much higher your opt in conversion rates will be.



Tracking

Speaking of conversion rates, another disadvantage of the standard iOS dialogues is the lack of analytics within that dialogue. The only rough estimations you can derive from a standard permission implementation is if the ratio of sent out push notifications (a number which should be present in your backend) increases in parallel to the number of new acquired users. A pretty muddy estimation game.

Having a custom ‘pre permission’ dialogue (I kind of start to like this term) in place also let’s you track the interaction with it in more detail – e.g. within certain user groups. You still don’t get the final numbers from the iOS system dialogue, but you’ll learn much more about when and how to ask for it.

Design

Even though I personally consider timing the much more important aspect of getting a push notification opt in, the actual design of you asking for it plays a huge role as well.

And by design I not only mean the actual Visual Design but also the copy. Customizing the push notification permission gives you the opportunity to deploy multiple variants of how to actually ask for it at the same time. AB/Multi-variant testing is an incredible opportunity by itself – But deploying it for figuring out how to unlock one of the strongest engagement mechanics of your mobile app is HUGE! Regarding the actual pixels I personally think you should just carry-through your product’s identity (like Candy Crush is clearly doing blow).

Candy Crush Push Notification Permissions

Nothing too fancy, but I think a clear differentiation from the then followed default dialogue is a plus.

So, how is your app currently asking for push notification permissions? I hope you can derive some concrete improvements for your own work. Let me know what you’ve found out!

Investing in your skills pays off in the long run

I recently had to run a two day workshop in order to assess new product visions and plan the needed discovery phases accordingly. Due to some scheduling fuck-ups on my side, I hadn’t been really able to put in the preparation time for the workshop I usually intend to have.
So I expected to have a rather mediocre workshop itself, not matching my usual aspiration for a fun thing like that.

Contrary to my expectations, the workshop went pretty well. We had a great time, produced exactly the output we needed and were after all pretty aligned on the outcome.
Despite being happy about the results, I couldn’t help to think about why it worked that well, lacking my usual load of preparation time and thoughts. I almost felt guilty that I was ‘lucky’ and the scenario, day and time just played for me and led to the great results.

After putting some thoughts in it, I definitely think it was something very simple which led to the situation: experience.
By doing stakeholder-heavy product work for 6 years and being in challenging environments managing high-scale products, I can now benefit from this year-long learning and refinement of my skills. This is what allows me to now being able to run workshops or discovery phases with fewer upfront effort.

When I arrived at that conclusion, a neat comparison came to my mind which was often times articulated by my former swim coach back in the days. As you can imagine, putting a lot of mileage into swim training with a high share of endurance work isn’t exactly what you crave for as a short distance specialist (like I was).
Observing the training plans of older world-class sprinters, which focussed much more on intensity and had an incredibly low number of endurance sessions, we then raised the question towards our coaches why couldn’t train the same way.

The answer was simple, yet unsatisfying for me back at that time. Those athletes did put a lot of effort into laying endurance foundations when they were younger (e.g. during junior years) and could now still benefit from this high level they’ve built up for themselves. So they only had to put in quite few endurance effort right now in order to simply maintain this threshold.

It’s basically the same process which I described observing my own actions – I can now benefit from the hard work and investments I put into my abilities the years prior to now. Other examples are my efforts to ramp-up in coding, design and even online marketing to now being in a much better position to orchestrate those disciplines to form a great overall product.

The most crucial thing when arriving at this insight is though, to determine what the skills are you need to invest in today, in order to benefit from in a couple of years. For me, it’s for example public speaking and leadership – What are yours?

Why I don’t believe in ‘Idea Backlogs’

Hello and Happy New Year (just in case we haven’t seen each other yet in 2017). I hope you enjoyed the holidays and have been able to take some time off to recharge for whatever is next on your agenda.

Especially the beginning of the year we find ourselves in the middle of quarterly or even yearly planning discussions regarding which product initiatives to tackle next.
Often times, the few items which find an actual slot on the short term roadmap are the essence of a so called ‘long list’ of ideas. But what happens with all the other brainstormed output? They get thrown into a so called ‘idea backlog’, right next to that great idea you had last year and tried to bring into development. Especially when it has been a close call regarding prioritization, it feels like a good excuse for all participants to not entirely dismiss an idea but to ‘save it for later’.

Let me tell you what: Don’t do that. If an idea is just not good enough to outperform ‘competitors’ in the very moment and is so unbelievably full of value for you and your key stakeholders that you immediately want to dedicate your best resources to it right now, it won’t be any better the next time you revisit that list of broken dreams in your excel file.
Instead, I highly recommend to make a clear cut below the very top product ideas which find their ways into execution and just start a new ideation process from a blank sheet the next time a roadmap slot is up for discussion.
And if an idea is truly so good that it’ll appear again and again on your lists, it’ll find its way into production mode when the time is right. Trust me.

Let me give you another example for this way of thinking: I  did the same thing when 2016 ended with all my blog post ideas and ‘books to read’ resolutions. When they were just not that good that I wanted to immediately sit down and dedicate writing or reading time to them which I otherwise probably would have spent working out or watching YouTube, I deleted them. No more drafts and plans to ‘read this book very soon’.
When a topic or book is really really relevant to me, it’ll somehow resurface somewhere to me and will then get tackled.

What’s in your (personal or work-related) idea backlog? And how long have some of those items been in there? See, just kick them out and just execute on the next things which is driving you for real.

This article appeared first on my e-mail list for product people, ux designers and entrepreneurs.

My Personal Review of 2016

Can you believe it’s already the end of 2016? In a couple of days, we’re about to celebrate 2017 and boy has this year been a ride…Let’s take a (non-chronological) look at what’s behind:

  • First and foremost, I became a dad. Describing this experience would break all formatting space of this newsletter. Which is why I’ll just summarize it this way: WOW!
  • l decided to leave my job at XING after working with a bunch of incredibly talented people after 2,5 years. It wasn’t an easy decision but…
  • …it led me to join ORBIT! So far, I can’t believe how much we’ve been able to learn and achieve as a team in such a short period of time. I can’t wait for what we’re gonna pull off in the future!
  • On the same note, I not only gained awesome people in a professional sense, but also found incredibly like-minded people in Sven, Johann and Jan on a personal level to have fun with – Thanks for having me, guys!
  • Related to that, I actually found myself in the longest period of not working since I basically started my internship at eprofessional in 2010 – And instead spending an incredible amount of quality time with the little one and my better half.
  • I actually found quite a nice format and rhythm for this newsletter and which also is seemingly rewarded with an increasing number of subscribers and engagement. Considering that I’m sending a regular-ish newsletter since 2 years or so, it’s nice to see this is developing into something sustainable.
  • We decided to leave our trusted central neighborhood in Hamburg and moved a bit more to the north of this lovely city. While space and accessibility were our main triggers, having more nature around us and offering Jannis a more playful environment to grow up in were nice ‘side effects’.
  • With the help of Tim, I’ve been able to pick-up a regular schedule for Productish. Pretty happy with our format and the topics we’ve discussed so far.
  • I managed to publish my by far most successful blog post on Product Discovery. Which also boosted my motivation for more creative writing around product management a lot.
  • I decided to get back into public speaking about product-related topics and submitted a proposal for the ‘Working Products‘ in Hamburg – Maybe you wanna join me? Let’s see where this can get from there…
  • I had my first actual cycling vacation with Robert in the very north of Germany. We had a lot of fun, despite unsteady weather conditions. I really hope to repeat that in warmer conditions and with Friedrich by our side over the course of 2017.

See you in 2017. Let’s make it a good one!