Building data-driven organizations, Part 1: What it means to be data-driven
Decision-making and what really matters
Many companies claim to be data-driven, but not all of them are. Despite having the right intentions, they sometimes think about it the wrong way.
A common pattern is approaching being data-driven as a list of capabilities: setup a data warehouse, buy a BI platform, hire a data scientist. It’s a trap I call being data-driven on paper, because from the outside they look like all the right things to do. But behind the window dressing, companies fail to leverage their data to unlock real business value.
There’s a difference between being data-driven on paper and being data-driven in practice. To close the gap, we need to understand what it means to be data-driven from first principles.
Being data-driven is about decision-making
At its core, being data-driven boils down to a simple question:¹
Are we making decisions differently based on data?
Let’s break this down.
We care about “decisions” because decisions are how data gets operationalized within organizations. Whether made by humans based on business insights or machines based on algorithms, decisions help translate data most directly to business outcomes (revenue growth, cost savings, etc).
We care about “differently” because data should help people make different decisions than they would have made otherwise. Data represents information that is useful to decision-makers. If the presence or absence of data doesn’t affect the end decision, it’s not providing value.
And obviously, we care about data. One analogy to describe how it fits together is an experiment where we seek to drive incremental (differently) impact via the treatment (data) on the outcome variable (decision).
Show me all the decisions
In practice, companies make many decisions and many types of decisions. By applying this simple framework, we can start developing a holistic view of data-driven decision-making across the organization. Here are some dimensions I like to think about:
Coverage of decisions. How pervasive is data in decision-making processes across the organization? Are product decisions data-driven but marketing decisions not, or vice versa? While it’s natural to spend time with teams that are further along, often the biggest opportunities lie with teams that are not yet working with data at all.
Significance of outcomes. How important are the decisions where data is involved? It’s common for decision-makers to fall back on intuition when the stakes are high. For example, it’s one thing to let data determine the color of a button on a website, or how to optimize a small marketing channel. It’s entirely different to let data drive whether to invest resources to launch a new product, or how to allocate the company’s overall budget.
Magnitude of change. How different would the decision be otherwise? For example, it’s easy to use data to recommend a 5% price increase, if there’s a widely-held belief that the product is underpriced. Increasing prices would be the default decision anyway. It’d be harder to use data to convince decision-makers to keep prices the same. Yet some of the biggest wins come from challenging people to do the right thing, against their prior beliefs.
To build on the original question:
Are we making decisions differently based on data – consistently across the organization, where the stakes are high and the recommendations are unpopular?
Let’s talk about the hard problems
If we’re being honest to ourselves, the answer to the question is often no. We can all recall frustrating examples of decision-makers misinterpreting data points from finished work. We’ve witnessed senior leaders override experiment results due to personal bias. And we’ve seen the graveyards of completed analyses and models that went nowhere. Often organizational, political, and ideological obstacles stand in the way of progress. And even decision-making challenges we’ve put to rest end up resurfacing in another form as companies scale.
These are hard problems to solve. They also deserve great attention, because the companies that successfully overcome them are the ones who truly unlock the business value of being data-driven in practice. The ones that don’t pay an invisible, compounding tax on decision-making that reflects negatively in the long-term trajectory of their businesses.
To be clear, investing in data capabilities is important and it’s a topic I’ve written about in the past. But having the right technology and talent is ultimately a means to an end. We need to think about being data-driven in the right way, and that starts with having conversations about the decisions themselves.
¹ This framework evaluates whether decisions are driven by data, but does not discuss how decisions should be driven by data. This depends on many other factors and may be a future blogpost.