You Can’t Measure Everything: Dealing with Uncertainty

As project professionals, we want to be part of making jobs successful.

What constitutes success? Well, I think we can all agree that it’s a combination of: constructing a quality product safely, ensuring it’s delivered on time and on budget, looking after the environment, and producing an end product that is fit for purpose for all stakeholders. 

So why do so many projects fail, and why does this seem to be increasing as time goes by? It’s something that I’ve been thinking about over the last few years, as I’m sure most of us in the industry have. 

There is no one problem. However, I believe there’s an overarching framework of thinking about a project using complexity theory that can open our eyes to what’s going on.

The current state of projects

I’ll use project costs as a basis for my explanations because it’s the most measurable, but these concepts can be applied to all the project outcomes listed above.

To talk about these projects, let’s take a journey from building an estimate, to designing the structure, to building the works. (Note: The below is an oversimplified explanation of how a project estimate, design and execution occurs for simplicity.)

When it comes to building an estimate for a project, we start by imagining the resources, time and cost that go into each component of the works. For example a set crew and production rate is “known” for drainage pipe install and quantities are fed into an estimate to give a lineal m price.  

If our project involves building a road, we’ll start by estimating the cost of design, followed by the cost of activities such as site clearing, asset relocation, earthworks, and pavement installation.

Generally, this is completed through first principles pricing and using market rates for resources.

For certain items or activities, we may use historic rates, but only for those that are replicable or repeatable.

As an industry, we tend to disregard previous projects’ costs as somewhat irrelevant. We put very little effort into capturing costs for completing different activities. If we do capture the costs, we won’t capture the corresponding productivities for the works, rendering the data only half-complete.

Each project is unique –  we aren’t making widgets in factories and it’s not as simple as recording what you made the last 1000 widgets for and multiplying this forward. We build structures in dynamic environments that are different from project to project. This seems to translate into a reluctance to capture meaningful project production data – we can’t quite see what we’d use it for.

Because we can’t simply replicate past project performance from job to job, we discount empirical data (actual data) completely and build estimates using our past experience and industry knowledge.

Once we have our “bottom up” estimate, we add these items together.

Cost of road = earthworks + drainage + pavement + asphalting + overhead + R&O.

We may, if we’re feeling diligent, add up any risk and opportunity values that we can think of, weigh these, and add them to our estimate. Some projects use statistical techniques such as Monte Carlo analysis also. These still rely on the quality of the inputs placed into the models.

With some differences, this is how we estimate our projects, from small to very large projects.

Generally, little empirical data is considered in this process.

How predictable are your projects?

Executing the work is where things get interesting. Projects involve a number of interdependent moving parts. Some jobs are simple, some are complicated and others are complex. The more interconnected elements, the more complex the project.

Complicated and Complex systems act very differently. This is a vital mental model that we need to truly understand projects and how to make them successful. It’s distinct to how we generally think of projects now.

The more complex a project is, the harder it is to truly understand cause and effect relationships. This is a feature of complexity – humans aren’t built to properly understand this point, for example, introducing species into new environments to solve a simple problem (rabbits, foxes, cane toads). Generally, the larger a project, the more nodes that are connected in causal chains and therefore, the more complex it is. Complexity increases the chance that random events that you never even thought about will affect your project performance.

Put another way, as your project increases in complexity, it becomes more like the stock market or the weather – it becomes less predictable.

So, is this a problem?

Well, it depends on whether those things that you can’t predict are more likely to have a positive or a negative effect on your project. Are you more likely to have lucky events or unlucky events occur?

Well, it turns out that, by far, you’re more likely to be unlucky. This is because the nature of unknowns increases the length of the project, which costs money, impacting budgets that tend to be quite lean in the first place.

This imbalance is defined as negative asymmetry. That is, random events that occur are more likely to have a negative impact on your project than a positive one.

Our estimate is generally built from the bottom up. In this way, it assumes the works are complicated and not complex. We’re assuming we’re building a washing machine, rather than dealing with the weather.

So, we have a bid process where we’re encouraged by competition to keep prices very lean with little contingency. We build the pricing using first principles, adding discrete parts together without considering the interactions between these parts. We also use very little empirical data because we know it can’t simply be applied to the process due to the unique nature of each project. 

Then, we set about building the project. We know that due to our concept of complexity, the more complex a project is (or the more complex we make it) the more interconnections there are between each part of a project. Therefore, it is more likely random events will occur that couldn’t reasonably be predicted at tender. Due to the nature of projects, these random events are much more likely to have a negative effect on a project than a positive one.

A tricky problem

Without understanding this concept, we have, as an industry, exacerbated its effects by centralising decision-making, adding unnecessary nodes to our projects, and not investing in the resources that will actually help projects perform. The more complex the project, the more this problem impacts performance.

In the name of mitigating risk and trying to meet budgets through austerity measures, we’ve actually increased our likelihood of not meeting our project goals because we’ve misunderstood the nature of the environment we’re playing in. The bigger the project, the more complex it’s likely to be, and the more we have to act differently.

This explains why traditional approaches work on small-scale projects that are relatively simple, but not on larger more complex projects. We aren’t building washing machines.

You might be thinking, ‘thanks Lach for dumping this issue on my desk without a solution. So what do we do?’. Well, by changing our mental model of projects there are ways in which we can deal with the uncertainty. I’ll outline a few here.

1. Move from a scarcity mindset to an investment mindset

We know that due to randomness and negative asymmetry, just trying to finish the works by doing everything very cheaply will not be enough to achieve project targets. We need to move our thinking as project professionals to a value creation mindset. What can we invest in the process that will save us time and money in the future? For example,  you may need to spend more in the budget now, but by doing so, it will save you more in the future – even if you can’t exactly measure it. 

This last point is a wicked problem, as we’ve become obsessed with measuring everything. By its very nature, reducing unknown unknowns can’t be measured as it prevents things from ever happening.

This is what I like to refer to as an investment mindset versus a scarcity mindset. I’ll dive deeper into how having an investment mindset can lead to better project outcomes in a future article, as I believe it’s a massive issue in why projects go over budget. 

2. Reduce complexity

One of the key things we can do is make our system less connected and our processes less dependent on one another, particularly through streamlining systems and onsite works. Every time we add a new process, it adds a node to the system. Anyone who has ever laid a multi bet on Grand Final day understands what this does to the likelihood of positive outcomes. We must decouple activities and system chains and make them as simple and effective as possible to get the best results.

3. Reverse the trend of centralised decision-making

Projects are dynamic and complex. Teams need to react quickly to mitigate risk. However, the trend towards centralisation of approvals and decision-making is having the opposite effect. It’s locking the negative effects in. We need a new, more nuanced approach to risk management that protects us, but allows project professionals to do their jobs at the coal face. This is essential for successful projects and a key tenet of dealing with complex systems as an organisation.

4. Invest in project management

Project management is not a cost to be minimised – it’s your best defence against randomness. It’s the number one thing you can invest in to get a return.

The lean manufacturing revolution that changed the world in the 1950s figured out the outsized returns in investing in project management to optimise systems. It’s time we did the same.

5. Collect empirical data from projects

We’ve avoided doing this properly for years, because without a detailed statistical analysis, it isn’t as useful as it would be in a closed system like manufacturing.

But within actual data, includes the effects of randomness that we can’t predict at the start of our project. If we collect enough of this information, particularly production and cost data, and categorise it properly, we can change the way projects are delivered forever.

By changing our mental model, we can better navigate the uncertainties that are a given on any project, big or small. We have to act, and think, differently if we want to meet our project goals.

If you’re interested in learning more about our project management philosophy, or if you’d like to chat about what it’s like to work at FSC get in touch.