Point Lookout: a free weekly publication of Chaco Canyon Consulting
Volume 23, Issue 9;   March 1, 2023: Goodhart's Law and Reification

Goodhart's Law and Reification

by

Goodhart's Law, applied to organizations, is an observation about managing by metrics. When we make known the goals for our metrics, we risk having the metrics lose their ability to measure. The risk is elevated when we try to "measure" abstractions.
Charles Goodhart delivers the keynote speech in the 2012 Long Finance Spring Conference

Charles Goodhart delivers the keynote speech in the 2012 Long Finance Spring Conference at Bank of America Merrill Lynch. The talk was entitled "Procyclicality of Financial Regulation: And how to deal with it." The event was hosted in partnership with Gresham College. This is a portion of an image (cc) Attribution-ShareAlike 3.0 Unported by Jamesfranklingresham, courtesy Wikimedia.

Goodhart's Law isn't a law in the legal or scientific sense. Charles Goodhart put it this way in a 1975 paper: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." [Goodhart 1975] More plainly, when we express an organizational goal in terms of a metric, the metric loses its value as a measure of anything. Although Goodhart's interest was managing national economies, his observation turns out to apply also to managing organizations.

Goodhart's insight has only grown in importance since 1975, because metrics have become so central to the practice of managing organizations. Indeed, a search at Google for "managing by metrics" (without the quotes) returned 1.1 billion hits. By comparison, searching for "finding love" yields only about four times as many hits.

As an When we express an organizational goal
in terms of a metric, the metric loses its
value as a measure of anything
example of the implications of Goodhart's Law, suppose we measure the performance of our project management teams by tracking their budget overruns and schedule overruns. And suppose we use this data when we make decisions about promotions, salaries, and bonuses. Goodhart's Law implies that the value of this data as a measure of performance will gradually decline over time, and ultimately collapse. In this example, we would find that project planners would become so accurate in their projections of cost and schedule as to cast doubt on the measurements, because only the clairvoyant could possibly be so accurate.

Those who use metrics-based approaches for managing their organizations would do well to consider to what degree there is risk that Goodhart's Law might apply. To assess that risk, examine the factors that could make the metrics data misleading. One factor is what psychologists call the reification error.

The reification error

To reify, in its psychological sense, is to regard — and treat — an abstract entity as if it were a physical, concrete entity. [Levy 1997] [Brenner 2011] For example, after releasing a bowling ball, with the ball still rolling down the lane, a bowler might reach out a hand as if to push the ball more toward the "sweet spot" so as to increase the number of pins the ball can knock down. The bowler knows that releasing the ball marks the end of the bowler's influence. Still, it's satisfying to pretend otherwise. For many bowlers, this behavior is no pretense.

In the case of metrics, we engage in reification when we assume that we can "measure" something that has no physical manifestation. For example, it's impossible to measure the value of a software engineer's daily output. We can measure the time the engineer spent on a task, because we can measure time. We can measure the number of lines of code produced, because we can count. But the value of the engineer's output isn't measurable until we've subjected it to quality tests, the most basic of which is, "Does it work?" Even after testing, we might not know how adaptable the code is, how maintainable it is, or how difficult it will be for successive engineers to understand how it works. These factors, some of which are known as "ilities" or non-functional requirements (NFRs), aren't directly measurable.

NFRs can't be measured by examining the engineer's output, and some NFRs can't actually be measured at all. To presume that we can measure something as immeasurable as engineering output quality, reducing it to a number or to a set of numbers, is to commit the reification error. What we can measure are proxies for some aspects of some NFRs. But then the reification error leads us to believe that the proxy is the principal — that the measurement is equivalent to the attribute it supposedly stands for. In many situations the two are not equivalent.

How reification can undermine metrics

Reification in itself cannot compromise the utility of a metric as effectively as it can when the people involved in the processes being measured are aware of the goal value of the metric. In that case, awareness of the goal and the reification error conspire. Here's one illustration of this dynamic.

Consider a metric that purports to represent an organizational attribute that's subject to the reification error. Like engineering productivity, such attributes are necessarily abstractions. If the metric goal is widely known, the temptation to adjust the measurement protocol so as to produce favorable results can be extreme. And since the attribute has no physical manifestation, the variety of possible protocol adjustments is limited only by inventiveness. For example, someone might point out that because the existing protocol "overlooks important phenomena," we must adjust it to gain a more reliable prediction of future performance. Over time, following a stream of adjustments like that, the measurement protocol no longer produces data representative of anything.

Last words

Numeric data carries with it the air of objectivity. Because we are accustomed to respecting the reality of numeric measurements of physical entities, we sometimes overlook the flexibility of the relationship between numeric data and the abstractions that data supposedly represents. It is that flexibility that can create a risk of disaster when we manage an organization according to numeric data whose goal values are widely known.

That flexibility has many sources. Next time we'll explore how people engage in "gaming" the metrics — another mechanism that can undermine metrics when the metric goal values are widely known.  Next in this series Go to top Top  Next issue: Goodhart's Law and Gaming the Metrics  Next Issue

52 Tips for Leaders of Project-Oriented OrganizationsAre your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!

Footnotes

Comprehensive list of all citations from all editions of Point Lookout
[Goodhart 1975]
Charles Goodhart. "Problems of Monetary Management: The U.K. Experience", in Courakis, Anthony S. (ed.), Inflation, Depression, and Economic Policy in the West. Totowa, New Jersey: Barnes and Noble Books (1981), p. 116. Order from Amazon.com. Back
[Levy 1997]
David A. Levy. Tools of Critical Thinking: Metathoughts for Psychology. Boston: Allyn and Bacon, 1997. Order from Amazon.com. Back
[Brenner 2011]
Richard Brenner. "The Reification Error and Performance Management," Point Lookout blog, September 28, 2011. Available here. Back

Your comments are welcome

Would you like to see your comments posted here? rbrenIyeJIiAfnGdKlUXrner@ChacsxirZwZlENmHUNHioCanyon.comSend me your comments by email, or by Web form.

About Point Lookout

This article in its entirety was written by a 
          human being. No machine intelligence was involved in any way.Thank you for reading this article. I hope you enjoyed it and found it useful, and that you'll consider recommending it to a friend.

This article in its entirety was written by a human being. No machine intelligence was involved in any way.

Point Lookout is a free weekly email newsletter. Browse the archive of past issues. Subscribe for free.

Support Point Lookout by joining the Friends of Point Lookout, as an individual or as an organization.

Do you face a complex interpersonal situation? Send it in, anonymously if you like, and I'll give you my two cents.

Related articles

More articles on Personal, Team, and Organizational Effectiveness:

A squirrel running a cageRunning Your Personal Squirrel Cage
As Glen rounded the corner behind the old oak, entering the last mile of his morning run, he suddenly realized that he was thinking about picking up the dry cleaning tomorrow and changing his medical appointment. Physically, he was jogging in a park, but mentally, he was running in a squirrel cage. How does this happen? What can we do about it?
Gut bacteriaMitigating Outsourcing Risks: I
Outsourcing internal processes modifies the usual risk configuration of those processes, but it also creates a special class of risks that are peculiar to the outsourcing relationship. What are some of those risks and what can we do about them?
Winslow Homer's painting, BlackboardFill in the Blanks
When we conceal information about ourselves and our areas of responsibility, we make room for others to speculate. Speculation is rarely helpful. It's wise to fill in the blanks.
Barack Obama, 44th President of the United StatesSpeak for Influence
Among the factors that determine the influence of contributions in meetings are the content of the contribution and how it fits into the conversation. Most of the time, we focus too much on content and not enough on fit.
An outstanding example of the Utility Pole anti-patternThe Utility Pole Anti-Pattern: I
Organizational processes can get so complicated that nobody actually knows how they work. If getting something done takes too long, the organization can't lead its markets, or even catch up to the leaders. Why does this happen?

See also Personal, Team, and Organizational Effectiveness and Problem Solving and Creativity for more related articles.

Forthcoming issues of Point Lookout

What most of us think of when we think of checklistsComing February 28: Checklists: Conventional or Auditable
Checklists help us remember the steps of complex procedures, and the order in which we must execute them. The simplest form is the conventional checklist. But when we need a record of what we've done, we need an auditable checklist. Available here and by RSS on February 28.
Adolf Hitler greets Neville Chamberlain at the beginning of the Bad Godesberg meeting on 24 September 1938And on March 6: Six More Insights About Workplace Bullying
Some of the lore about dealing with bullies at work isn't just wrong — it's harmful. It's harmful in the sense that applying it intensifies the bullying. Here are six insights that might help when devising strategies for dealing with bullies at work. Example: Letting yourself be bullied is not a thing. Available here and by RSS on March 6.

Coaching services

I offer email and telephone coaching at both corporate and individual rates. Contact Rick for details at rbrenIyeJIiAfnGdKlUXrner@ChacsxirZwZlENmHUNHioCanyon.com or (650) 787-6475, or toll-free in the continental US at (866) 378-5470.

Get the ebook!

Past issues of Point Lookout are available in six ebooks:

Reprinting this article

Are you a writer, editor or publisher on deadline? Are you looking for an article that will get people talking and get compliments flying your way? You can have 500-1000 words in your inbox in one hour. License any article from this Web site. More info

Follow Rick

Send email or subscribe to one of my newsletters Follow me at LinkedIn Follow me at X, or share a tweet Subscribe to RSS feeds Subscribe to RSS feeds
The message of Point Lookout is unique. Help get the message out. Please donate to help keep Point Lookout available for free to everyone.
Technical Debt for Policymakers BlogMy blog, Technical Debt for Policymakers, offers resources, insights, and conversations of interest to policymakers who are concerned with managing technical debt within their organizations. Get the millstone of technical debt off the neck of your organization!
Go For It: Sometimes It's Easier If You RunBad boss, long commute, troubling ethical questions, hateful colleague? Learn what we can do when we love the work but not the job.
303 Tips for Virtual and Global TeamsLearn how to make your virtual global team sing.
101 Tips for Managing ChangeAre you managing a change effort that faces rampant cynicism, passive non-cooperation, or maybe even outright revolt?
101 Tips for Effective MeetingsLearn how to make meetings more productive — and more rare.
Exchange your "personal trade secrets" — the tips, tricks and techniques that make you an ace — with other aces, anonymously. Visit the Library of Personal Trade Secrets.
If your teams don't yet consistently achieve state-of-the-art teamwork, check out this catalog. Help is just a few clicks/taps away!
Ebooks, booklets and tip books on project management, conflict, writing email, effective meetings and more.