How Do You Measure Business Value?

Over time there have been many different ways I have seen agile projects or scrum teams measure value. The most important criterion is that it needs to be fit for purpose and usable, easily repeatable throughout the project life-cycle and teams cadences as well as relevant to the project resources, business stakeholders and end customer/user alike.

Challenges often present early in establishing value scales as team members challenge what is important to measure and how to measure different value criteria. The best approach, as is the case in almost all agile team formations, is to get the team agreement early to “just get started“. Start simple and then refine as you go. Create an initial prototype even if you cannot agree on the value ratings and learn what needs improvement by using the prototype, reviewing feedback and improve the process or model as you go. If you are starting to look at business value once a project has kicked off a good way to test out your prototype is to use historical data such as delivered user stories, put them through your prototype and then compare the results with the stories that may have been deprioritised during backlog refinement.

The benefits of starting with the simple model, even if it is as simple as suggesting a single numerical scale of 1 to 10 (or use Fibonacci as you would for story points 1,2,3,5,8,13,20), is that everyone can easily follow how to use it as it is familiar. The downside of this is that the team will often debate heavily around what the difference is between one persons 7 compared to another persons 7 etc. This is common, and an opportunity to have that exact conversation in the context of the item being discussed (lets say it is a story being rated for business value). This will give a real life example of how the prototype is being used and will generate insights on how the process might be incrementally improved. At the very least it starts to establish team thoughts on what a “7” actually is and relative value estimation starts to evolve.

So what do you do when peeps cant agree in a relative estimation model?

The answer in my experience is you need to move to a less subjective model. My advice is always to introduce change in consumable bite size pieces, iteratively. Don’t jump straight to a model that locks people down to a value scale that is dictated by directly quantifiable data values unless this was easily achieved at conception of the prototype. This can lead to teams feeling like they need to conduct high levels of detailed analysis up front around justifying business benefit at a story level and is not repeatable or sustainable.

One of my favorite models to use as a starting point in a baseline measuring system is the WSJF model as outlined in the Scaled Agile Framework (SAFe). WSJF is a mouthful of an acronym but stands for Weighted Shortest Job First, essentially this model compares the value (SAFe term being Cost of Delay) with the effort to complete the delivery of this value. The goal is to select the highest value reward with the lowest effort to complete. Another way of looking at this is lowest effort vs. highest return, therefore these product backlog increments (PBI’s) should go first.

Lets look at this concept firstly at it’s most basic, single backlog, single value stream level. In a fit for purpose approach the criteria can change to what ever the customer wants to measure as being valuable. In the original SAFe framework the criteria are User-Business Value, Time Criticality and Risk Reduction and/or Opportunity Enablement. The sum of these criteria then get divided by the effort or story points to come up with a figure that can be used at a glance to compare items across the backlog. For example:

In commercial business environments it is easy enough to put quantifiable value set behind the scale and therefore introduce less subjectivity. Opportunity Enablement in a Fibonacci scale of 1 to 20 for example can be linked to a dollar value (e.g. 1= <$50k and 20= >$5m).

What happens if your project is less attached to quantifiable measures and outputs?

You may find yourself working on more qualitative driven projects which are harder for the stakeholders to quantify benefit, however you still need to have conversations around business or customer value. You can still use a multi criteria approach based on the value drivers for that project but you may need to start with a less descriptive scale increments and adopt more of a low, medium, high value rating system.

In a recent project the product owners short listed a bunch of value measures and then prioritised these down to a set of 4 criteria being Customer Experience (CX), User Experience (UX), Time Criticality (TC) and Risk Reduction (RR). To set the scale for each they used terms that meant something to them or their customers but still keeping the scale simple so that it was easy to use…whilst maintaining relevancy.

So what happens next?

Backlog prioritisation sees the Product Ownership team, guided by the Product Owner & Business SME, evaluate each PBI (in this case User Story) relative to each other across each value category. This requires a starting point for relativity. SAFe suggests the team should identify the lowest value rated PBI in each category and give it a 1, thereafter every rating has a comparison point. Just as good is getting the team to elect the PBI they feel they know the most about and rate all categories for that Story. Then the story and all of the rated categories becomes the reference point for business value rating.

The figure above assumes that the team has also been involved in story point estimation. The business value is then created as a sum of the 4 categories. In this example the logical sequence of priority would be Story C, Story A and then Story B. Story C has the highest business value and lowest story points and therefore meets the highest reward vs lowest effort criteria. As we have strayed from true WSJF its interesting to note the WSJF on Story C would be 12 (36/3=12). Stories A & B have the same business value but Story A is only 5 story points so takes pole position. To reinforce this selection Story A = a WSJF of 6.4 and B = 2.5.

A good friend of mine and I were solving a particular problem for a customer and we were looking at ways to improve their business value rating system. For the purpose of this tale I am going to call him Andy, as that is his name. We were reviewing the value ratings of epics across the portfolio and observed that each product owner had very strong views that their epics scheduled next on the roadmap were all of high business vale. That could be because they were higher on each individual product owners backlog and as such of highest importance to that product owner, and fair enough right? But this introduced priority conflicts that were hard to defend with a single business value figure as they were only reviewing the business value sum for each others epics not the category breakdown. If the maximum business value number was 100%, everyone’s epics were 80% or higher.

Now for some fun… Its time to call the doctor!

To combat this we needed to come up with something that would expose value over effort, avoid concealing the rating of each category from only reviewing the sum and use something that would generate enthusiasm in its use. And the business value Blood Pressure chart was born.

Based on the concept of blood pressure measurements being articulated as 120 over 80, if you’re lucky ;), we wanted to carry this concept through to plotting your results on a hospital chart. In this example the reading was 36 over 3.

With the Value & Effort exposed they can easily be plotted on the blood pressure chart. The blood pressure chart is based on a version of a prioritisation matrix.

This type of chart or approach is not new but giving it a theme adds to engagement and collaboration. The line dividing the quadrant will need to move based on your scale and rating averages.

Its always good to review the process, the measures and outcomes in retrospectives to ensure they are as accurate as the initial vision and re-calibrate.

It is good for transparency to have the chart made visible in the team area and not hidden away in a digital location where its not accessible to the team and wider stakeholders.

This method has been very successful when managing multiple value streams under a program or portfolio or a single project, and it’s fully scaleable when required. Finding ways to gain consensus across multiple Product Owners and project objectives can be tricky to say the least. Whether you have an org structure that has a Group Product Ownership structure or separate product owners needing to collaborate on priority, a common business value system is hugely beneficial in generating the right conversations and transparency in an effort to remove information masking by more subjective methods.

What do you use to measure business value?

What has worked well or not so well?

Please share or provide thoughts on this article in the comments.

Cheers Jas