So many teams have a list of projects laid out on a roadmap sometimes months or years out, without a clear idea of how success is measured. Are they being measured based on the number of projects completed? Getting them done “on time”? High quality? Team utilization? Are any of these things helping meet the company objectives?
When did we stop experimenting and start believing we were always right?
Why are we spending so much money building things that may or may not have any real value? How are we even determining what we build?
We have spent years calling ourselves Lean or Agile, as we optimize the delivery of the highest priority items in our backlogs. That is making the big assumption that we’re building the right things. What we are probably doing, though, is building the wrong things, faster.
Building The Right Thing
Before we move onto the next idea that will “make us a million bucks” or that’s sat long enough on a wishlist that it has nagged it’s way to priority, let’s get our acts in gear and I ask ourselves a few questions:
- What are our goals?
- What are our hypotheses around how to hit those goals?
- What are the tiniest experiments we can do to prove or disprove our assumptions around our hypotheses?
- How can we validate our experiments with our customers?
- What’s the next big hypothesis?
- Rinse and repeat.
Picture this: a team has the quarterly goal to increase user adoption of the mobile platform and enable a segment of users to become completely independent of the website, doing all business via the mobile app. In collaboration with Product, Marketing, UX and Customer Service, they start with a brainstorming session looking at where the numbers are now and come up with a few hypotheses on where barriers to entry are for users who fall out.
They determine what are the riskiest assumptions made, then come up with small experiments that will quickly help them determine if their assumptions are right or wrong. Once they determine what they should build from those experiments, they get that out in front of customers as quickly as possible to continue learning. Only after they truly see how and if the customers are using the product do they flesh out past the bare bones.
Sounds fantastic, right? So why aren’t we doing it?
Building Only What We Need
If you are already doing the above, congratulations! You might be building the right thing, but you’re likely still building WAAAAY too much. Too much of a good thing is still too much. When do you stop?
You have taken that idea and broken it down into the Epics or Features that make up the high-level backbone of your project. You’ve prioritized those by value and you may even have broken down the top of the backlog into roughly sprintable stories. You are ready to bring in the team and fill them in on your vision and walk through the stories that will be their guides for the next few months to build out your gorgeous product. Heck, you might even have trimmed it down to an MVP. But is it really what your users want? Will they really use it? Will it solve their problems? How do you know?
You’ve likely seen the above graphic of the findings of a Standish Group Study. They discovered that 64% of software features are rarely or never used. 64%! That is a lot of wasted time. That is a lot of wasted money. That is 64% more value we could have been adding somewhere else. Why do we keep building when we aren’t adding value?
Jim York did an Open Space session at Scrum Gathering earlier this year about Awesome Product Ownership. He challenged us to apply the Pareto Principle (also known as the 80/20 Rule) to a typical backlog. The logic goes as follows. You get 80% of the value from 20% of the effort. As you move down the backlog you apply the 80/20 Rule to what’s left. The first 20% of the backlog nets you 80% of your business value. That’s the easy calculation. Now it’s math time!
After knocking out that first 20%, 20% of the remaining 80% is 16%. This 16% of effort delivers 16% more value (80% of the remaining 20% = 16%). The next jump is another 12.8% effort delivering only 3.2% more value. That leaves 0.8% of the value of the backlog items taking up almost 50% of the effort. Where should we stop building? After that initial 20%? Into the next 16%?
If we compare the backlog to the segments of the Standish report below, we see a correlation. Always and Often represent 20% of the features, while Sometimes makes up the next 16%. So we can deliver 96% of the value by building only the top third of our backlog. WHY ARE WE STILL BUILDING EVERYTHING AND THE KITCHEN SINK!?!?!
When the business value dips below the level of effort percentage, we should stop. Yes, this requires ruthless prioritization. It requires talking to your users, testing hypotheses via experiments with validated learning and really figuring out what they want.
It is also not good enough to just apply the 80/20 rule to features or stories. You will need to dig into the stories themselves and apply an 80/20 on the acceptance criteria. York asked us who on the team would know what the most valuable thing to build in each story would be. It took a minute, but then someone had the “ah-ha” moment and exclaimed, “The QA Analyst!” “What is it?”, Jim questioned. “The Happy Path!!”
The cheapest, most valuable thing to build is the Happy Path. If you aren’t familiar with the term, it is the path the user takes through the product when everything is going as planned. This is such a high percentage of the time, yet we build out the 13 other paths they could take complete with error handling. Why??! This adds a ton of time and complexity to the code and doesn’t necessarily get us back the ROI. Some alternate paths might be necessary to avoid crashes or error conditions for “productization” sake, but the key is to assess which alternate paths deliver the most business value, and discard or delay the others. Perhaps apply the 20% rule again, this time on the Alternate paths as well. We are drowning in a sea of “what ifs” and our users are suffering because of it.
We need to stop spending so much money building so much crap. Set goals, hypothesize how we will hit those goals, run the smallest experiment we can to prove or disprove our assumptions, build the happy path, and then stop. As we learn we can choose to continue investing in the current project or allow ourselves to move on to the next experiment.