Point Lookout: a free weekly publication of Chaco Canyon Consulting
Volume 23, Issue 9;   March 1, 2023: Goodhart's Law and Reification

Goodhart's Law and Reification

by

Goodhart's Law, applied to organizations, is an observation about managing by metrics. When we make known the goals for our metrics, we risk having the metrics lose their ability to measure. The risk is elevated when we try to "measure" abstractions.
Charles Goodhart delivers the keynote speech in the 2012 Long Finance Spring Conference

Charles Goodhart delivers the keynote speech in the 2012 Long Finance Spring Conference at Bank of America Merrill Lynch. The talk was entitled "Procyclicality of Financial Regulation: And how to deal with it." The event was hosted in partnership with Gresham College. This is a portion of an image (cc) Attribution-ShareAlike 3.0 Unported by Jamesfranklingresham, courtesy Wikimedia.

Goodhart's Law isn't a law in the legal or scientific sense. Charles Goodhart put it this way in a 1975 paper: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." [Goodhart 1975] More plainly, when we express an organizational goal in terms of a metric, the metric loses its value as a measure of anything. Although Goodhart's interest was managing national economies, his observation turns out to apply also to managing organizations.

Goodhart's insight has only grown in importance since 1975, because metrics have become so central to the practice of managing organizations. Indeed, a search at Google for "managing by metrics" (without the quotes) returned 1.1 billion hits. By comparison, searching for "finding love" yields only about four times as many hits.

As an When we express an organizational goal
in terms of a metric, the metric loses its
value as a measure of anything
example of the implications of Goodhart's Law, suppose we measure the performance of our project management teams by tracking their budget overruns and schedule overruns. And suppose we use this data when we make decisions about promotions, salaries, and bonuses. Goodhart's Law implies that the value of this data as a measure of performance will gradually decline over time, and ultimately collapse. In this example, we would find that project planners would become so accurate in their projections of cost and schedule as to cast doubt on the measurements, because only the clairvoyant could possibly be so accurate.

Those who use metrics-based approaches for managing their organizations would do well to consider to what degree there is risk that Goodhart's Law might apply. To assess that risk, examine the factors that could make the metrics data misleading. One factor is what psychologists call the reification error.

The reification error

To reify, in its psychological sense, is to regard — and treat — an abstract entity as if it were a physical, concrete entity. [Levy 1997] [Brenner 2011] For example, after releasing a bowling ball, with the ball still rolling down the lane, a bowler might reach out a hand as if to push the ball more toward the "sweet spot" so as to increase the number of pins the ball can knock down. The bowler knows that releasing the ball marks the end of the bowler's influence. Still, it's satisfying to pretend otherwise. For many bowlers, this behavior is no pretense.

In the case of metrics, we engage in reification when we assume that we can "measure" something that has no physical manifestation. For example, it's impossible to measure the value of a software engineer's daily output. We can measure the time the engineer spent on a task, because we can measure time. We can measure the number of lines of code produced, because we can count. But the value of the engineer's output isn't measurable until we've subjected it to quality tests, the most basic of which is, "Does it work?" Even after testing, we might not know how adaptable the code is, how maintainable it is, or how difficult it will be for successive engineers to understand how it works. These factors, some of which are known as "ilities" or non-functional requirements (NFRs), aren't directly measurable.

NFRs can't be measured by examining the engineer's output, and some NFRs can't actually be measured at all. To presume that we can measure something as immeasurable as engineering output quality, reducing it to a number or to a set of numbers, is to commit the reification error. What we can measure are proxies for some aspects of some NFRs. But then the reification error leads us to believe that the proxy is the principal — that the measurement is equivalent to the attribute it supposedly stands for. In many situations the two are not equivalent.

How reification can undermine metrics

Reification in itself cannot compromise the utility of a metric as effectively as it can when the people involved in the processes being measured are aware of the goal value of the metric. In that case, awareness of the goal and the reification error conspire. Here's one illustration of this dynamic.

Consider a metric that purports to represent an organizational attribute that's subject to the reification error. Like engineering productivity, such attributes are necessarily abstractions. If the metric goal is widely known, the temptation to adjust the measurement protocol so as to produce favorable results can be extreme. And since the attribute has no physical manifestation, the variety of possible protocol adjustments is limited only by inventiveness. For example, someone might point out that because the existing protocol "overlooks important phenomena," we must adjust it to gain a more reliable prediction of future performance. Over time, following a stream of adjustments like that, the measurement protocol no longer produces data representative of anything.

Last words

Numeric data carries with it the air of objectivity. Because we are accustomed to respecting the reality of numeric measurements of physical entities, we sometimes overlook the flexibility of the relationship between numeric data and the abstractions that data supposedly represents. It is that flexibility that can create a risk of disaster when we manage an organization according to numeric data whose goal values are widely known.

That flexibility has many sources. Next time we'll explore how people engage in "gaming" the metrics — another mechanism that can undermine metrics when the metric goal values are widely known.  Next in this series Go to top Top  Next issue: Goodhart's Law and Gaming the Metrics  Next Issue

52 Tips for Leaders of Project-Oriented OrganizationsAre your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!

Footnotes

Comprehensive list of all citations from all editions of Point Lookout
[Goodhart 1975]
Charles Goodhart. "Problems of Monetary Management: The U.K. Experience", in Courakis, Anthony S. (ed.), Inflation, Depression, and Economic Policy in the West. Totowa, New Jersey: Barnes and Noble Books (1981), p. 116. Order from Amazon.com. Back
[Levy 1997]
David A. Levy. Tools of Critical Thinking: Metathoughts for Psychology. Boston: Allyn and Bacon, 1997. Order from Amazon.com. Back
[Brenner 2011]
Richard Brenner. "The Reification Error and Performance Management," Point Lookout blog, September 28, 2011. Available here. Back

Your comments are welcome

Would you like to see your comments posted here? rbrendPtoGuFOkTSMQOzxner@ChacEgGqaylUnkmwIkkwoCanyon.comSend me your comments by email, or by Web form.

About Point Lookout

This article in its entirety was written by a 
          human being. No machine intelligence was involved in any way.Thank you for reading this article. I hope you enjoyed it and found it useful, and that you'll consider recommending it to a friend.

This article in its entirety was written by a human being. No machine intelligence was involved in any way.

Point Lookout is a free weekly email newsletter. Browse the archive of past issues. Subscribe for free.

Support Point Lookout by joining the Friends of Point Lookout, as an individual or as an organization.

Do you face a complex interpersonal situation? Send it in, anonymously if you like, and I'll give you my two cents.

Related articles

More articles on Personal, Team, and Organizational Effectiveness:

Ice cream barMake Space for Serendipity
Serendipity in project management is rare, in part, because we're under too much pressure to see it. If we can reduce the pressure, wonderful things happen.
A forest glenGames for Meetings: I
We spend a lot of time and emotional energy in meetings, much of it engaged in any of dozens of ritualized games. Here's Part I of a little catalog of some of our favorites, and what we can do about them.
Ancient stairs at ruins in CambodiaThe True Costs of Indirectness
Indirect communications are veiled, ambiguous, excessively diplomatic, or conveyed to people other than the actual target. We often use indirectness to avoid confrontation or to avoid dealing with conflict. It can be an expensive practice.
Perceptual illusions resulting from reificationThe Reification Error and Performance Management
Just as real concrete objects have attributes, so do abstract concepts, or constructs. But attempting to measure the attributes of constructs as if they were the attributes of real objects is an example of the reification error. In performance management, committing this error leads to unexpected and unwanted results.
Agricultural silosDisjoint Awareness: Assessment
When collaborators misunderstand each other's work and intentions, they're at risk of inadvertently interfering with each other. Three causes of misunderstandings are complexity, specialization, and rapid change.

See also Personal, Team, and Organizational Effectiveness and Problem Solving and Creativity for more related articles.

Forthcoming issues of Point Lookout

A meeting in a typical conference roomComing April 3: Recapping Factioned Meetings
A factioned meeting is one in which participants identify more closely with their factions, rather than with the meeting as a whole. Agreements reached in such meetings are at risk of instability as participants maneuver for advantage after the meeting. Available here and by RSS on April 3.
Franz Halder, German general and the chief of staff of the Army High Command (OKH) in Nazi Germany from 1938 until September 1942And on April 10: Managing Dunning-Kruger Risk
A cognitive bias called the Dunning-Kruger Effect can create risk for organizational missions that require expertise beyond the range of knowledge and experience of decision-makers. They might misjudge the organization's capacity to execute the mission successfully. They might even be unaware of the risk of so misjudging. Available here and by RSS on April 10.

Coaching services

I offer email and telephone coaching at both corporate and individual rates. Contact Rick for details at rbrendPtoGuFOkTSMQOzxner@ChacEgGqaylUnkmwIkkwoCanyon.com or (650) 787-6475, or toll-free in the continental US at (866) 378-5470.

Get the ebook!

Past issues of Point Lookout are available in six ebooks:

Reprinting this article

Are you a writer, editor or publisher on deadline? Are you looking for an article that will get people talking and get compliments flying your way? You can have 500-1000 words in your inbox in one hour. License any article from this Web site. More info

Follow Rick

Send email or subscribe to one of my newsletters Follow me at LinkedIn Follow me at X, or share a post Subscribe to RSS feeds Subscribe to RSS feeds
The message of Point Lookout is unique. Help get the message out. Please donate to help keep Point Lookout available for free to everyone.
Technical Debt for Policymakers BlogMy blog, Technical Debt for Policymakers, offers resources, insights, and conversations of interest to policymakers who are concerned with managing technical debt within their organizations. Get the millstone of technical debt off the neck of your organization!
Go For It: Sometimes It's Easier If You RunBad boss, long commute, troubling ethical questions, hateful colleague? Learn what we can do when we love the work but not the job.
303 Tips for Virtual and Global TeamsLearn how to make your virtual global team sing.
101 Tips for Managing ChangeAre you managing a change effort that faces rampant cynicism, passive non-cooperation, or maybe even outright revolt?
101 Tips for Effective MeetingsLearn how to make meetings more productive — and more rare.
Exchange your "personal trade secrets" — the tips, tricks and techniques that make you an ace — with other aces, anonymously. Visit the Library of Personal Trade Secrets.
If your teams don't yet consistently achieve state-of-the-art teamwork, check out this catalog. Help is just a few clicks/taps away!
Ebooks, booklets and tip books on project management, conflict, writing email, effective meetings and more.