When Good Data Drives Bad Decisions

data-crystal-ballWe’ve all seen it.  Some junior executive falls in love with a particular bit of data. Statistics are calculated. Reports are generated. Mass distraction ensues as the rare commodities of time and attention are expended to “move the needle” on the object of the exec’s affection.  If we’re lucky there is little net impact to the experience of customers, employees, and other partners. Often there are missed opportunities for other, greater impacts. In the worst cases, the organization becomes so fixated on that innocent data point that they loose focus on serving customers, engaging employees, or maintaining necessary partnerships.

So what happened?  While even I don’t have the courage to delve into the various foundations of affection in business leadership, in this age of big data some patterns are beginning to emerge, at least on an empirical level, if not yet in the clearly defined, if a tad delusional, world of statistics.  Clearly there are times when better data supports better decision making, when insights emerge from data that are not accessible to mere mortal instinct. When data is employed to remove confirmation bias or open avenues for creativity, good things usually happen.

However, data is not an un-alloyed good. Like its bastard sibling, technology, data is not morally neutral, ferocious arguments to the contrary not withstanding.  Like technology, our data is infused with our history, culture, and context. The numbers on which we choose to focus, the presentation of those elevated numbers, and certainly their interpretation all introduce a level of subjectivity that is not immediately apparent when data gets wrapped in the royal purple of factual aristocracy.

So what’s a decision maker to do? The idea of data driven decision making has been elevated to the level of mystical belief, beyond question or analysis. Challenging the use of data in decision making is not a quest even Don Quixote would take on. As with any questionable article of faith in a given population, the most effective route is rarely head-on confrontation. Our articles of faith emerge from years of experience, distilled across generations, with at least a veneer of effective application. When data is driving bad decisions and priorities in your organization, stringing up the data is rarely a quick or effective remedy.  One must follow the data back through its interpretation, publication and calculation to the subjective foundations and assumptions that are the beginning of any data hegemony.

Where to look?  Well, here the empirical patterns begin to point a way.  There are some common and observable ways in which data gets miss-applied in decision making. By far the most common is when data collection and reporting replaces the decision making.  When a decision maker abdicates their accountability for judgment and puts raw data in that seat, bad things will eventual ensue.  Any decision maker who cannot give a convincing rational for how the data supports a given decision is probably engaging in this non-productive behavior.

Another early warning sign of data addiction is a persistent preference for data over on-going reality. Using data to get by bias and unnecessary ambiguity is one thing. Preferring a representation to the real thing is another. If data is consistently manipulated, even slightly, to paint a more positive picture (usually in the guise of “greater accuracy”) for example, look out. Some one is falling into the trap that better data always means better reality. Data is an echo of reality, what’s left after reality leaves the stage. Mistaking one for the other is rarely a path to joy.

A third pattern to look for is a preference for the dance of data to the observation of its impact. You know this drill.  Someone is endlessly amused by the mechanics of data collection, the intricacies of its calculation and the nuances of its reporting, while completely clueless to actual use or relevance.  If your organization never reflects on the effort and attention required to collect and present data as opposed to the actual use of that data, the organization is probably not getting the highest possible valuable out of a data orientation.

There are any number of other data faux pas and I suppose one could read this little rant as a polemic against data. Actually, I’m a true believer that better data can support better decision making across a broader population. However, like any social good, the positive outcomes require our engagement not only at an intellectual level, but also in the realms of our emotions, our attachments and our assumptions. Effective use of data isn’t some magic bypass around all that subjectivity, but it can provide us a useful map to aid in our navigating through it all.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s