Why organisations misunderstand things

In August this year, McKinsey released an article called “Yes, you can measure software developer productivity”. This created a lot of reactions from people saying that McKinsey had completely missed the point of recording metrics, why we record some metrics, and why we don’t record others.

And I broadly agree with those commentators critical of these metrics (Dave Farley’s analysis is particularly good, and there’s one from our very own Andrew Gibson finding the good in the metrics). However, what has happened here is not limited to McKinsey, or even to metrics.

I have not met a developer without a story where they have joined a team that claimed to be doing agile but instead were doing something else, usually waterfall. This is another example of the same problem, an organisation missing the deeper reasoning.

So how does this happen?

In 1985, Edgar Schein published a book called “Organisational Culture and Leadership: A dynamic view”. In it, he hoped to explain how organisations decide what is and what isn’t a priority issue for them to fix. It provides a great model for understanding why organisations misunderstand.

He proposed a model of three layers from most visible to least. On the outside, we have Artefacts; below this, we have Values, and below that, we have Assumptions.

Artefacts are the things you see in your organisation day to day. This might be tickets in Jira, or your daily standup, or even the tests in your codebase. These are the easiest things to see around your office and are often tangible or visible to the casual observer.

Below this we have Values. These are harder to experience, they are the things that the creators of the artefacts value. A great example of when people have considered this and written it down is the Agile Manifesto.

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

Manifesto for Agile Software Development

By the way: think back to your experience of that organisation that missed the point of agile. Did they hold these values?

Under all of these are the fundamental assumptions needed to make these values work. These are even less visible than the values. In the case of the Agile Manifesto, the assumption is that developing software is as much about discovering the requirements as implementing them and that working in small iterative chunks will produce better results than other methods.

When McKinsey wrote this article, they fell into the trap that Schein’s model exposes. They have looked at the fantastic work of Amy C. Edmondson and Nicole Forsgren in researching how organisations create effectively and seen the Artefacts: metric frameworks and not the Values or even the Assumptions below them.

It isn’t that McKinsey stupid, or malicious, it’s that they only saw the most visible part of the work. So when they came to build on those metrics, adding their own, they were unable to iterate on that work. I think that’s a very natural misstep to make.

My reading of the values underlying these metrics is that while metrics are important, they are merely the outward manifestation of the underlying value of using scientific research to identify, on aggregate, what the best indicators of organisational success are.

However, even Amy Edmonson and Nicole Forsgren are making assumptions. One of those is that organisations, when exposed to metrics, will react to them in a logical way and that we can perform scientific research to discover what these metrics should be.

I suspect that both of these are true, but they are very much assumptions.

So, next time you are learning something new, try and dig a little deeper and see if you can discover the values and assumptions of the thing you are trying to learn, and if someone is struggling to pick up what you are trying to teach them, make sure you have signposted your assumptions and values as best you can — it’ll help.

I will end this on a tip: Think of an artefact that you use. If you were to ban yourself from using it, what could you do to give yourself the same effect? Now ban that too, and come up with another. Do this enough times and the common thread between them is what you value.

You can likewise use this approach for assumptions too. List all the situations you don’t think the value applies, the common thread is what your assumptions are.

I’d love to hear about your experiences of discovering things you valued that you didn’t expect.