By Jonathan H. Westover, PhD
Listen to this article:
Abstract: This article discusses potential pitfalls that can arise from data-driven decision making when not properly balanced with human judgment, strategic thinking, and organizational context. Drawing on the author's experience as a management consultant, key risks are examined such as the "tyranny of metrics" when KPIs distort goals, misinterpreting correlations as causation, overgeneralizing from limited data sets, confirmation bias within "filter bubbles," and overemphasizing quantitative data at the expense of human elements. Specific industry examples are provided to illustrate unintended consequences that have occurred. The article advocates for maintaining a multidisciplinary, questioning approach to analytics, constantly reevaluating metrics and models, supplementing internal data with external realities, and blending quantitative and qualitative inputs. Balancing these risks and enacting various safeguards can maximize the benefits of fact-based management while avoiding potential downsides.
As a management consultant, I've seen firsthand how data-driven decision-making can transform performance when done right. However, I've also witnessed how things can go awry when an overreliance on numbers replaces good old-fashioned judgment.
Today we will explore common ways data-driven decisions can derail if not kept in check by human wisdom, strategic thinking, and an understanding of organizational culture.
The Tyranny of Metrics
One danger sign I've noticed is when quantitative targets and KPIs start driving behavior in a way that was never intended (Davenport & Harris, 2007). For example, a hospital I once worked with set ambitious readmission rate goals to reduce costs. On the surface, this seemed like a smart use of data to improve quality of care. However, what actually happened was physicians started avoiding "riskier" patients to keep their numbers looking good. This had the perverse effect of harming access to care for those most in need.
As counterintuitive as it seems, an overreliance on metrics can undermine the very objectives they were put in place to achieve. The problem, as author Dylan Kapoor aptly terms it, is "The Tyranny of Metrics" - when quantifying performance distorts and displaces actual performance (Kapoor, 2018). To avoid this, leaders must closely monitor unintended consequences and periodically reevaluate whether metrics still align with strategic goals as conditions change. Data should supplement, not substitute, thoughtful human deliberation.
Misinterpreting Correlations as Causation
Another common mistake is assuming any correlation revealed in the data implies direct causation. As the old adage goes, correlation does not equal causation. For example, one tech company I advised noticed productivity clustered around certain demographic traits like age and education level. Executives wrongly inferred these attributes caused differences in output.
In reality, more nuanced factors were likely at play. Productivity is influenced by a complex web of interrelated forces - not defined by any single variable in isolation (Gillespie & Patrick, 2018). By misreading patterns as proof of direct cause-and-effect relationships, leaders risk implementing misguided initiatives that don't actually address root issues. Thoroughly vetting statistical links with contextual understanding prevents such faulty decision-making.
Overgeneralizing from Limited Data Sets
Large corporations amass vast troves of customer, employee, and operations data. But no matter how "big" one's data grows, it remains just a sample of a much larger, more complex reality (Davenport, 2014). Overreliance on internal records can fuel misleading overgeneralizations about markets, workforces, and the world at large if companies forget this fundamental limitation.
For instance, one retail chain was making poor inventory choices because its ordering algorithms only learned from years of its own purchasing data patterns, missing relevant trends elsewhere. A broader market perspective was needed to see how shifting demographics and emerging competitors were disrupting traditional sales cycles (Lee et al., 2019). Leaders must exercise prudent skepticism of conclusions drawn from any single data set and complement numbers with diverse external perspectives.
Filter Bubbles and Confirmation Bias
Related to the prior trap, when data teams only look within their own four walls, it facilitates what internet scholars call "filter bubbles" - instinctively seeking out information confirming one's initial assumptions while filtering dissent (Pariser, 2011). This risks missing "black swan" insights that challenge existing mindsets.
For example, during strategic planning for a big e-commerce firm, analytical workshops kept revisiting familiar hypotheses about growth drivers instead of daring to consider disruptive alternatives. More diversity of thought during data sessions may have unearthed fresh options instead of circular reasoning. Opening dialogue to informed contrarians can counterbalance the Confirmation biases numbers sometimes fuel.
Losing the Human Element
At the end of the day, organizations are about people - their skills, relationships, behaviors, and broader circumstances numbers can't fully capture (Pfeffer & Sutton, 2006). Over-centralizing decisions around clinical data patterns obscures these human realities that shape productivity, creativity, and job satisfaction.
As one software startup discovered, an AI tool for screening job candidates based on past hires was dismissing top talent because their backgrounds didn't numerically resemble previous profiles. Only by supplementing formulas with in-person interviews did they recognize strong "cultural fits" the algorithms were missing (D’Aveni, 2015). Data must serve but not supplant traditional management functions requiring social intuition and qualitative judgement.
Industry Examples
To bring these risks to life, consider how they have played out in specific industry settings:
A hospital network saw readmissions rise after overindexing metrics on discharge rates, to the detriment of coordination with community services (Prybil et al., 2014).
An insurance company's fraud detection system wrongfully flagged long-time clients as "high risk" due to insufficient representative training data on customer longevity (Way et al., 2019).
An electric utility underinvested in maintenance by overrelying on predictive failure models trained only on its own equipment records, failing to foresee changes in manufacturer best practices affecting reliability (Dietvorst et al., 2015).
As these real-world examples show, data-driven decisions demand as much art as science to avoid unintended fallout. Grounding analytics in context guards against common pitfalls while still realizing data's substantial upsides.
Managing the Risks
So given these potential downsides of data overreliance, what precautions can organizations take? Based on my experience, the keys are:
Maintain a diverse, interdisciplinary team when developing and validating models, not just technical “quants.”
Constantly reevaluate KPIs and metrics to avoid misaligned incentives over time.
Scrutinize correlations with subject matter experts before leaping to causal conclusions.
Benchmark internally-generated insights against external market realities whenever feasible.
Foster an environment encouraging “devil’s advocacy” to counteract confirmation biases.
Blend data-driven and experience-based decision-making, not replace one with the other.
Closely monitor implementation impacts and revise course as needed to avoid unintended consequences.
In the hands of a prudent, multidisciplinary leadership team, data truly holds immense power to sharpen strategy and maximize outcomes. But likewise, an excessively quantified view risks clouding more than it clarifies. With careful management of these challenges, the upsides of fact-based management far outweigh potential downsides.
Conclusion
In closing, used prudently as one input among many, data can meaningfully advance corporate objectives. However, as with any tool, overreliance or misapplication can backfire. By maintaining a balanced, multi-faceted decision framework attentive to human and contextual realities, leaders can sidestep common analytics pitfalls while still capitalizing on the transformative capabilities of fact-based insights. Numbers alone do not an effective strategy make - but paired with seasoned judgment, they can light the path to sustainable excellence. In an era when the very definition of “good management” is being rewritten by new technologies, maintaining this blend of art and science remains as important as ever.
References
Davenport, T. H., & Harris, J. G. (2007). Competing on analytics: The new science of winning. Harvard Business Review, 85(1), 98–107. https://hbr.org/2007/01/competing-on-analytics
Kapoor, D. (2018). The Tyranny of Metrics. Princeton University Press. https://press.princeton.edu/books/hardcover/9780691180276/the-tyranny-of-metrics
Gillespie, N., & Patrick, G. (2018). Why Metrics Matter: The Importance of Organizational Culture. Organizational Culture White Paper Series, 1-12. https://www.charleskochinstitute.org/wp-content/uploads/2021/02/Gillespie-Patrick-Organizational-Culture-White-Paper.pdf
Davenport, T. H. (2014, May 15). Big data at work: dispelling the myths, uncovering the opportunities. Harvard Business Review. https://hbr.org/2014/05/big-data-at-work-dispelling-the-myths-uncovering-the-opportunities
Lee, J., Lakshmanan, M. A., & Ng, V. (2019). Transfer learning via knowledge networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 3266–3273. https://doi.org/10.1609/aaai.v33i01.33013266
Pariser, E. (2011). The filter bubble: What the Internet is hiding from you. Penguin. https://www.penguinrandomhouse.com/books/290539/the-filter-bubble-by-eli-pariser/
Pfeffer, J., & Sutton, R. I. (2006). Evidence-based management. Harvard business review, 84(1), 62. https://hbr.org/2006/01/evidence-based-management
D’Aveni, R. (2015). The 3-D printing of organizations. Harvard Business Review, 93(10), 40–48. https://hbr.org/2015/10/the-3-d-printing-of-organizations
Prybil, L., Dent, D., Calves, N. E., Murray, G. F., & Kelly, M. (2014). Governance and management practices in high-performing community health centers. Effective Clinical Practice, 17(1), 29–37. https://doi.org/10.1001/jamainternmed.2013.12956
Way, A. E., Larreguy, H. A., & Gibson, J. L. (2019). Machine-learning IAT: Understanding implicit prejudice in the 21st century. Psicología Política, 59, 69-89. doi:10.5093/pp2019a8
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114. https://psycnet.apa.org/doi/10.1037/xge0000033
Jonathan H. Westover, PhD is Chief Academic & Learning Officer (HCI Academy); Chair/Professor, Organizational Leadership (UVU); OD Consultant (Human Capital Innovations). Read Jonathan Westover's executive profile here.
Suggested Citation: Westover, J. H. (2024). When Analysis Outruns Wisdom: Potential Pitfalls of Data-Driven Decision-Making, 12(1). doi.org/10.70175/hclreview.2020.12.1.2
