Reliable Systems

I grew up as the third son in our family. When my oldest brother was a newly minted driver, like every new driver he was a little rough. And like any younger siblings, my other brother and I were kind and gentle in our commentary about it. This led him to declaring his first driving rule: No comments on his driving when he was driving. He was in command, and that was it.

One day soon thereafter, he was backing the car out of the garage with my other brother and I in it. Now, he didn’t normally park on the right side of the garage – that was where my dad’s car went. But by whatever fluke, there the big Ford station wagon was – on the right side of the garage. When backing out, you have to start turning right away because the driveway isn’t straight. In fact, you have to start turning left, meaning the front of the car goes to the right. Normally this was not a problem since there was plenty of clearance. But when starting on the right side of the garage, on the right is… the garage. As he started backing out, my brother and I quietly sat there, watched the side of the garage come right up and *bang* *crunch* we hit it. In the “after action review” that followed, my brother exclaimed “Why didn’t you tell me I was going to hit the garage?” You can imagine our response – “Because you had said never comment on your driving.”

At the time, I was smug in my righteousness. We had done exactly what he’d asked, we weren’t the driver and the driver is responsible for the car, so it was a big ‘not-my-problem’.

We were dead wrong.We were in a position to have prevented the problem, and we should have spoken up. Ever since, we’ve had the rule in our family that when riding in a car. The rule is to speak up if you see a problem without fear that the driver will be upset. The potential consequences of not calling a problem to the driver’s attention are too great.

How Do You Play the Blame Game?

The same story often plays out in the aftermath of a technology problem. Hang around a software development team long enough and you’re bound to hear a developer complain “Why didn’t QA find that defect? They should have found it before it shipped.” The difference between an experienced, healthy team and an amateur team is whether the developer is just venting or actually believes they are justified.

We often have a strong desire to try to reduce accountability for avoiding issues to a single party:

• QA is responsible for finding all defects in the software before it is released.
• IT Operations is responsible for keeping all of the servers running.
• The receptionist is responsible for ensuring we don’t run out of coffee.

Before looking at the contentious examples, look at just the last one. Say that you noticed that you pulled the next-to-last box of K-cups out of the supply cabinet. You’re not out of coffee yet – there are 24 individual servings in the box you pulled, one more box on the shelf. In most small companies, that’s at least a day’s worth of coffee. Would you tell the receptionist that you need more coffee? Or just assume that it will be taken care of? Say you then run out of coffee two days later, and everyone has to run out to Starbucks to feed their habit. Would you feel at all responsible for not speaking up when the problem was still avoidable?

You probably would have spoken up – the receptionist is a nice person, it’s an easy enough thing to do and you like your coffee.

Now look at the other two scenarios. The only real difference between them and running out of coffee is that these two will tend to be political and possibly even contractual. While you’d likely also speak up if you saw a defect in your company’s product before it was released or if you saw that a server was just about out of disk space, you wouldn’t want to accept any accountability after the fact if things went bad.

Here’s the elephant in the room: Your customers don’t care who was accountable for avoiding a problem. They care that the problem happened. They pay you for something that works (and has to work according to their definition of what works means). Anything else is just internal noise. If you want to drive your business forward – and really, if you don’t, you need to look to work somewhere else – this needs to be your motivator.

Formal vs. Practical Accountability

What if, instead of looking at issues as someone else’s problem, you followed these two principles?

• If you are in a position to prevent a problem, you are accountable for preventing it.
• If you are responsible for ensuring a problem doesn’t happen, you need to stay in a position to prevent it.

This means that many different people and groups may each be 100% accountable for a problem, because the most useful way to look at accountability is based on the ability and responsibility for preventing the problem. Why the most useful? Because the problem happened, that’s a matter of fact. While recriminations, blame, and shame may be cathartic or fun, they aren’t useful because they don’t further the goals of the team or the company. Put simply, your customers don’t care who’s at fault within your organization, just that you get the seriousness of the problem and you’re making it right. When debriefing your team, the ideal outcome is that everyone in the room sees how they could have prevented the problem, and takes on that they should have prevented the problem. From that, you then work into who was in the best place to prevent it – who could have seen it first, and addressed it while it was cheapest to address. You want to have everyone walk out with a balanced perspective of how they could have prevented it and how to identify when you’re in the best spot to prevent it.

A natural concern with this approach, particularly if it’s new to your organization, is that after action reviews are often a game of musical chairs – while there’s a superficial impression of honesty and openness, the true goal is to not be left without a chair when the music stops. Far from a well-calculated political move, this is really an emotional and ego driven outcome. No one likes admitting they are wrong, and with practice people get very skilled at justifying their emotional responses with pseudo-intellectual reasoning – it is called rationalization.

The next time you’re in this situation, try being the first party to speak up about what you could have done to avoid the problem, and make sure you communicate sincere regret you didn’t catch it. If you are completely open in this – sticking just to what you could have done without any back handedness (that’s right – you can’t say “I couldn’t cover up his incompetence.” That doesn’t count.) you’ll be amazed at how quickly the mood in the room changes. Very quickly others will jump in with what they could have done. You’ve created an environment where people can speak the real fears that are on their mind without posturing.

Once you’ve established this environment, you need to be active in maintaining it. If someone jumps into the attack, speak up and redirect the conversation. This is true particularly if the attack isn’t directed at you. Keep listening to have the conversation stay in even tones and that each party is either talking about what their area could have done or is constructively helping the overall conversation.

Eventually, there will be a fundamentally sticky conversation about which party was in the best position to avoid the problem. At this point it’s going to come down to culture – if your culture is one that learns from mistakes, it will be a clear and short conversation. Depending on how strong the duck-and-cover instinct is in your shop, it can be very painful. In the end – speak up if your team is the one that should have the spotlight. Fear of accountability is often overstated. In practice, managers know that in the end they need people that will be accountable for what happened, and the experience can still be positive in the long term. Great managers actively hunt out people that are quick to learn from their mistakes and own them.