Video Library

Four common software delivery bottlenecks

Dominica DeGrandis: Hi, I'm principal Flow advisor Dominica DeGrandis and in this tutorial i'm joined by VSM practice lead Carmen DeArdo to talk about how to identify common software delivery bottlenecks in Tasktop Viz.

Carmen DeArdo: Thanks Dominica i'm excited to talk about this. Finding bottlenecks is one of my favorite topics.

Degrandis: Mine too — in this tutorial we're going to talk about the importance of finding your bottlenecks, identifying and addressing four common bottlenecks, choosing the right bottleneck to address first, and then the importance of experimentation around bottlenecks. So when it comes to finding your bottlenecks, when you start using Tasktop Viz you gain visibility into the activities that flow across your value stream. And one of the things that you can start to see is after a period of baselining your data is just how fast you are delivering value as captured by Flow Time for each one of your flow work items. So that'd be Features, Defects, Risks and Depth. And one of the main goals with value streams is to achieve faster, more predictable workflows so you can meet the needs of your business that you identified for your product.

Finding your bottleneck, the part of the end-to-end workflow where work gets stuck, is kind of like having this helicopter view up above the highway where cars begin to back up. Because you're up high you have this big picture view — you can see where things are slowing down. You can start to make, you know, evidence-based decisions and changes to overcome the issue and see tangible measurable improvements in your flow time. And as a result, in your flow velocity, often value stream team members they intuitively understand where the issues are, but they lack the objective data to make this these evidence based changes for improvement. 

So Tasktop Viz can validate those suspicions with data and metrics, give you the evidence that allows you to open the door to this new understanding where you can address some of thecognitive biases that may be floating around in your organization And this data helpsnprovide the basis for experimentation and future investment.

DeArdo: The important thing to understand about bottlenecks is that you likely have a few places where work slows down. So the trick is to identify which one is the system's bottleneck — the one that that is worth tackling first. We'll talk about that towards the end of this tutorial. Your bottlenecks can appear anywhere in the flow work, from ideation to production. And today we're going to show you how to find them. We'll talk you through four common bottlenecks: what they are, how to see them in Viz and how to address them. The common bottlenecks are resource or expertise scarcity, lack of automation, process timings and dependencies.

DeGrandis: All right, so let's dive into the first one, Carmen: resource or expertise scarcity.

Resource or expertise scarcity

Dominica DeGrandis: Teams often wonder what they could accomplish if only they had more hands on deck. More people on the team, with the right skills who would be available at the right time when you needed them. The Bottleneck Analysis in Viz helps prove out some of these ideas by pointing to areas that slow down Flow, due to either insufficient capacity of people with the right skill set, expertise and domain knowledge. The more we scale, the more we need specialized people. And it's this growing division of labor in IT that results in the need for more expertise.

So let's begin with an example from the world of traffic flow. So imagine a two-lane road merging onto a one-lane bridge. Cars are forced to merge from two lanes into one lane to cross the bridge. Let’s say it takes a car about 30 seconds to cross the bridge, and its length accommodates six cars. If you created a workflow to describe the drive, you would include two workflow states for this portion of the road: a Wait state for ready to cross bridge, and an Active state for crossing the bridge. The crossing the bridge state would never have more than six concurrent cars in it, and it would only take about 30 seconds for each car to cross the bridge in what otherwise might be, say, a 45-minute journey.

Metrics on the crossing the bridge state might look pretty innocuous — only six concurrent work items and a duration of 30 seconds — however, what will point to this bridge as a bottleneck is the backup of cars in the prior state ready to cross the bridge. This preceding Wait state will have a higher count of cars, which will likely spend much longer than 30 seconds in it as they wait for their turn to cross the bridge. The high count and the long wait times in this Wait state point to a bottleneck in the Active state downstream crossing the bridge. 

Carmen DeArdo: So now let's apply this to the world of software. In a product value stream you might see this type of problem occurring when there's a limit on the amount of concurrent work that can be taken on during one of your Active states — the equivalent of the one lane bridge. For example, an engineering team has capacity to work on three features at a time, but the support team only has the capacity to troubleshoot three issues at a time. How will you see this in Viz? When there's a resource or expertise constraint slowing down Flow, the Viz Bottleneck Analysis will show high artifact counts and state durations in the preceding Wait state. Your understanding of the sequence of activities in your workflow will help you see that this is merely a symptom of a constrained skill set, or level of expertise in the Active state that follows.

So how do you solve it? Well let's go back to the bridge. There's no point investing in increasing traffic flow before the bridge. It would help cars get to the bridge faster but they're going to wait longer there to cross the bridge. The solution here would be add more lanes to the bridge to increase throughput. In software delivery, the solution to an expertise constraint is to ensure the right skills are available at the right time. Be sure to identify the precise set of skills in short supply. Hiring is one solution that might be appropriate, but you may also want to consider training or cross-training existing team members. The number of artifacts in the Wait state can help you estimate how many more trained specialists you may need.

DeGrandis: So let's summarize here. When a lack of resources or expertise results in long Wait times and work gets stuck, it'll manifest in high artifact counts and state durations that sit in the Wait state preceding the constrained Active state. To solve it, consider adding more skilled team members to the constrained Active state.

Lack of automation

Dominica DeGrandis: Now it's true that an expertise constraint can be solved by adding more experts — building those extra lanes if you will — however, there are cases where an expertise or a specialization constraint can be solved through other means like automation, self-service, or a process or policy change. Sometimes, increasing capacity might just slow things down further with the burden of onboarding new members because you have to train them and everything. So it's important to be aware of the different available options and when to apply them.

Carmen DeArdo: That's right Dominica. Now let's take a slightly different traffic flow example. We're all familiar with tollbooths on a highway. As we approach the tollbooth traffic starts to slow down because people have to interact with the tollbooth operator. As a result, cars can begin to back up as they wait their turn to go through the tollbooths. Now imagine there are 10 tollbooth lanes, but only two operators are currently working. So cars are forced to merge from 10 lanes into two lanes making the wait time even longer. That's a clear case of resource scarcity. But there are two ways you can solve it.

You can either add more tollbooth operators, or you can automate the tollbooth. Most tollbooths today have a fast pass system that allows people to just drive through the lane as it scans the tag mounted on the inside of the vehicle's windshield. You still might be forced to slow down, but much less so and the throughput is much higher preventing a backup. You've managed to automate something that once required a human interaction. This way you're not dependent on having a resource or expert available. You also don't need to slow down quite as much, which mitigates the long wait times.

DeGrandis: Nice. Let's apply that to the world of software now. In software it's often the case that a manual process or mandatory human interaction constrains Flow. For example, a common issue for value streams is getting new code tested in pre-production environments, because those environments are scarce and not self-service. As a result, there's new product capabilities or fixes to defects that pile up in the “ready-for-test” state. Just like in the previous example with the bridge, the constraint is in the subsequent Active state of “in test” where code changes are actively validated in a pre-production environment. But the pile up is really in “ready for test” where the code changes sit and wait until a test environment can be provisioned for them. 

How will you see this in Viz? in this example your Bottleneck Analysis will show high artifact counts and state durations in the preceding “ready-for-test” Wait state, which is a symptom of the constrained “in-test” Active state that follows. How do you solve it?  Let's look at the highway. Introducing fast pass lanes automates the human interaction with a tollbooth operator, reducing wait times and it improves your throughput. So if you automate test environment provisioning and allocation that will help improve flow for the value stream. Because it allows the teams to execute routine testing in pre-production environments as much as possible without relying on a central team to provision the environments because the central team's priorities may change. Your priority your team's priority might not be their highest priority.

DeArdo: In summary, when performing Flow or Bottleneck Analysis, remember that a resource or expertise scarcity or lack of automation is in an active state, but is first spotted in the preceding weight state where work piles up and ages.