Here are some study notes that I created whilst learning more about Evidence-Based Management from Scrum.org.
These are shared as a revision resource intended to summarise the key elements from The Evidence-Based Management Guide.
Complex problems defy easy solutions. Organisations work toward their goals in a series of small steps, inspecting the results of each step, and adapting their next actions based on what they learn along the way.
Here are the 3 elements of Evidence-Based Management (EBM):
- Goals & States
- Measures
- Empiricism
Goals & States
- Strategic Goal (could be a Product Vision) – Large and far away, with many uncertainties along the path.
- Intermediate Goals (could be a Product Goal) – Achieving Intermediate goals indicates an organisation is on the path to reaching its Strategic Goal. The path to the Intermediate Goal is often still somewhat uncertain, but not completely unknown.
- Immediate Tactical Goals (could be a Sprint Goal) – Critical near-term objectives toward which a team or group of teams will work that help toward achieving Intermediate Goals.
- Current State (Now) – Where the organisation is relative to the Strategic Goal at the present time.
- Starting State (Starting point) – Where the organisation is relative to the Strategic Goal when it starts its journey.
To achieve a Strategic Goal, organisations should form hypotheses and run experiments intended to advance them toward their Intermediate Goal.
Examples Of Goals:
- Strategic Goal – Eradicate the effects of COVID-19
- Intermediate Goal (Product Goal) – The successful completion of a vaccine trial.
- Immediate Tactical Goals (Sprint Goal) – Stage 2 of vaccine trail complete. DNA sequenced.
Understanding What Is Valuable
Broadly speaking, measures fall into 3 categories:
- Activities – Work that people do. E.g write code, design a graphic.
- Outputs – These are things that people produce/ E.g. A report, a new release of software.
- Outcomes – Desirable things that a customer or user of a product experiences. E.g. Being able to earn or save more money than before
The problem most organisations face, which is often reflected in the things they measure, is that measuring activities and outputs is easy, while measuring outcomes is difficult. However, delivering valuable outcomes to customers is essential if organisations are to reach their goals. For example, working more hours (activities) and delivering more features (outputs) does not necessarily lead to improved customer experiences (outcomes).
Measures
- Key Value Areas (KVA) – A set of perspectives on value and the organisation’s ability to deliver value.
- Current Value (CV) – Value. The value that the product delivers TODAY. Customers, employees and investors.
- Unrealized Value (UV) – Potential. The potential future value that could be realized if we met the needs of all potential customers or users. The satisfaction gap between a desired outcome and the current experience.
- Time-to-Market (T2M) – Responsiveness. How quickly you can improve CV. The ability to quickly deliver new capabilities, services, or products.
- Ability to Innovate (A2I) – Effectiveness. Ability to deliver new capabilities that might better meet customer needs E.g. Low levels of technical debt, happy employees.
Why Is Evidence-Based Management Important
Organizations can spend a lot of money implementing features only to find that customers don’t share the company’s opinion on their value. Beliefs in what is valuable are assumptions until they are validated by customers.
The first step toward a Strategic Goal (that relates to Unrealized Value (UV)) is understanding your Current State. To understand where you need to improve, you may also need to understand your effectiveness (A2I), and your responsiveness (T2M).
The Experiment Loop
The Experiment Loop helps organisations move from their Current State toward their Next Target Goal, and ultimately their Strategic Goal, by taking small, measured steps, called experiments, using explicit hypotheses.
- Hypothesis – A proposed explanation for some observation that has not yet been proven (or disproven).
- Experiment – Make the change you think will help you to improve and gather data to support or refute your hypothesis.
- Inspect – Did the change you made improve your results based on the measurements you have made?
- Adapt – Adapt your goals or your approach based on what you learned.
Both your goals and your improvement experiments will likely evolve as you learn more about customers, competitors, and your organisation’s capabilities. Goals can change because of outside events, and your tactics to reach your goals may need to be reconsidered and revised.
Explicitly forming hypotheses, measuring results, and inspecting and adapting goals based on those results are implicit parts of an agile approach. Making this work explicit and transparent is what EBM adds to the organisational improvement process.
Key Value Measures (KVMs)
To encourage adaptability, EBM defines no specific Key Value Measures (KVMs). The appendix of the EBM Guide provides many examples.
Evidence-Based Portfolio Management (EBM-PM)
Evidence-Based Portfolio Management (EBM-PM) applies lean and agile principles to the challenge of investing to derive the greatest business benefit. It uses rapid delivery of small increments of value to run experiments, gather information, and adapt investment decisions based on evidence from the market. EBM-PM Principles:
- Experimenting Beats Comprehensive Planning
- Making Smaller Bets Reduces Avoidable Loss
- Working on Fewer Things at a Time Finishes Them Faster
--
Hi, my name is Simon Kneafsey and I am a Professional Scrum Trainer with Scrum.org & TheScrumMaster.co.uk. I am on a mission to simplify Scrum for a million people. I have helped over 10,000 people so far and I can help you too.
Learn more at TheScrumMaster.co.uk and signup for our newsletter with 80,000+ practitioners.
