Self Managing Defects Escaping into Prod

Is your team becoming less predictable Sprint over Sprint?  Does your team feel like too many defects are reaching production, but they aren’t sure what to do?    I have worked with multiple teams who have encountered similar issues in the past.  If this situation sounds familiar, this blog shows how to self-manage this problem.  Let's investigate.

Setup

This Scrum Team has two-week Sprints and then releases monthly.  The team says that they are following the Scrum framework. They have been cranking out Product Backlog Items (PBIs) at a consistent rate and the Product Owner is happy, with delivery as are the stakeholders.  The team has started using Kanban metrics to measure their progress.  Metrics like Cycle time, and Service Level Expectation.  The team has an expectation that when they begin a user story, they can complete it within 3 days.  The team has an initial Definition of Done that looks like the following:

Definition of Done

1. Product Owner must approve PBI
2. All tests pass
3. Code is reviewed by a different team member before it is committed.

ISSUE

About 2 months ago, the team noticed an uptick in the number of production defects they had to fix urgently.  Most of these defects, classified as severity 1, had to be resolved immediately.  Due to this, the team adopted the practice of swarming which enabled them to minimize the impact of their production defects on the customer.  But every time the team swarmed one of their production defects this prevented them from  working on the PBIs the team forecast to complete during Sprint Planning.  This has led to the team not getting all PBIs done in a Sprint, but they have still consistently met their Sprint Goal! 

VISUALIZE THE PROBLEM

The Scrum Master had recently read the Kanban Guide for Scrum Teams and thought visualizing the team’s true workflow might help.  She took the idea to the Scrum Team and they decided to implement a Kanban board so that they could visualize their flow during the Sprint.  They elected to add an “Urgent swim lane” to their board so that they would easily see the impact of the production defects.  Man, was this swim lane cluttered with production issues!  The Kanban board helped the team visualize what they were feeling.  

The Urgent issues are affecting their delivery of the features that stakeholders and the Product Owner like to see.  The team, unsure what to do next, talks about addressing this issue in their Retrospective.

RETROSPECTIVE

To prepare for the upcoming Retrospective, our intrepid scrum master investigates some new metrics from the DevOps Research and Assessment (DORA) survey.  The Scrum Master feels the 4 recommended DORA metrics might be helpful:

 

As the Scrum Master has been reviewing, she brings up the fact that Change Failure Rate (CFR) has been trending up, as she would expect.  The team reviews both CFR and Time to Restore Service (TRS).  The team notices that the TRS has also been trending up. 

For their Retrospective, the Scrum Master asks the team to use the DORA metrics to validate if the changes they make have the desired effect.  Using the "Open the Box" Retrospective, the team focuses on what they need to keep doing and what activities they need to add.  Within the Retrospective they identified 2 new actions to implement in the next Sprint. 

The Scrum Team decided to run an experiment for the next 3 sprints.  They added the following to their Definition of Done:.

• Each new feature will be unit tested with a minimum of 70% code coverage. 
• Each feature will be tested using automation in a QA environment before being approved for production.

As you can see from this hypothetical scenario, the team must use facts to help focus on the improvement action items in the retrospective.  And the team determines action items that support a hypothesis to improve.  Future Retrospectives (i.e. 2 sprints from now) should incorporate this hypothesis to see if the actions improved the situation.

Remember to use metrics to help the team self manage.  In this scenario, the Scrum Master found metrics that might help the team with the issue they wanted to fix.  The Scrum Master crafted a Retrospective using those metrics to energize the team towards action.  But ultimately the team came up with the action items for improvement.  And they will test their hypothesis of improvement.  For anyone on a Scrum Team looking to solve a similar problem, remember to utilize self management and metrics for the team. 

Eric Landes is a Principle DevOps Coach at AgileThought, where he guides large enterprise development teams in DevOps practices, quality practices within teams, behavior driven development, continuous testing and collaboration techniques.  As a Professional Scrum Trainer, Eric brings a developer mindset to the framework. For more information:   

 Upcoming classes -

Podcasts