Goodhart's Law and Reification

Goodhart's Law isn't a law in the legal or scientific sense. Charles Goodhart put it this way in a 1975 paper: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." [Goodhart 1975] More plainly, when we express an organizational goal in terms of a metric, the metric loses its value as a measure of anything. Although Goodhart's interest was managing national economies, his observation turns out to apply also to managing organizations.

Goodhart's insight has only grown in importance since 1975, because metrics have become so central to the practice of managing organizations. Indeed, a search at Google for "managing by metrics" (without the quotes) returned 1.1 billion hits. By comparison, searching for "finding love" yields only about four times as many hits.

As an When we express an organizational goal
in terms of a metric, the metric loses its
value as a measure of anythingexample of the implications of Goodhart's Law, suppose we measure the performance of our project management teams by tracking their budget overruns and schedule overruns. And suppose we use this data when we make decisions about promotions, salaries, and bonuses. Goodhart's Law implies that the value of this data as a measure of performance will gradually decline over time, and ultimately collapse. In this example, we would find that project planners would become so accurate in their projections of cost and schedule as to cast doubt on the measurements, because only the clairvoyant could possibly be so accurate.

Those who use metrics-based approaches for managing their organizations would do well to consider to what degree there is risk that Goodhart's Law might apply. To assess that risk, examine the factors that could make the metrics data misleading. One factor is what psychologists call the reification error.

The reification error

To reify, in its psychological sense, is to regard — and treat — an abstract entity as if it were a physical, concrete entity. [Levy 1997] [Brenner 2011] For example, after releasing a bowling ball, with the ball still rolling down the lane, a bowler might reach out a hand as if to push the ball more toward the "sweet spot" so as to increase the number of pins the ball can knock down. The bowler knows that releasing the ball marks the end of the bowler's influence. Still, it's satisfying to pretend otherwise. For many bowlers, this behavior is no pretense.

In the case of metrics, we engage in reification when we assume that we can "measure" something that has no physical manifestation. For example, it's impossible to measure the value of a software engineer's daily output. We can measure the time the engineer spent on a task, because we can measure time. We can measure the number of lines of code produced, because we can count. But the value of the engineer's output isn't measurable until we've subjected it to quality tests, the most basic of which is, "Does it work?" Even after testing, we might not know how adaptable the code is, how maintainable it is, or how difficult it will be for successive engineers to understand how it works. These factors, some of which are known as "ilities" or non-functional requirements (NFRs), aren't directly measurable.

NFRs can't be measured by examining the engineer's output, and some NFRs can't actually be measured at all. To presume that we can measure something as immeasurable as engineering output quality, reducing it to a number or to a set of numbers, is to commit the reification error. What we can measure are proxies for some aspects of some NFRs. But then the reification error leads us to believe that the proxy is the principal — that the measurement is equivalent to the attribute it supposedly stands for. In many situations the two are not equivalent.

How reification can undermine metrics

Reification in itself cannot compromise the utility of a metric as effectively as it can when the people involved in the processes being measured are aware of the goal value of the metric. In that case, awareness of the goal and the reification error conspire. Here's one illustration of this dynamic.

Consider a metric that purports to represent an organizational attribute that's subject to the reification error. Like engineering productivity, such attributes are necessarily abstractions. If the metric goal is widely known, the temptation to adjust the measurement protocol so as to produce favorable results can be extreme. And since the attribute has no physical manifestation, the variety of possible protocol adjustments is limited only by inventiveness. For example, someone might point out that because the existing protocol "overlooks important phenomena," we must adjust it to gain a more reliable prediction of future performance. Over time, following a stream of adjustments like that, the measurement protocol no longer produces data representative of anything.

Last words

Numeric data carries with it the air of objectivity. Because we are accustomed to respecting the reality of numeric measurements of physical entities, we sometimes overlook the flexibility of the relationship between numeric data and the abstractions that data supposedly represents. It is that flexibility that can create a risk of disaster when we manage an organization according to numeric data whose goal values are widely known.

That flexibility has many sources. Next time we'll explore how people engage in "gaming" the metrics — another mechanism that can undermine metrics when the metric goal values are widely known.   Next issue in this series   Top    Next Issue

Are your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!

Footnotes

Your comments are welcome

About Point Lookout

Thank you for reading this article. I hope you enjoyed it and found it useful, and that you'll consider recommending it to a friend.

This article in its entirety was written by a human being. No machine intelligence was involved in any way.

Point Lookout is a free weekly email newsletter. Browse the archive of past issues. Subscribe for free.

Support Point Lookout by joining the Friends of Point Lookout, as an individual or as an organization.

Do you face a complex interpersonal situation? Send it in, anonymously if you like, and I'll give you my two cents.

Related articles

More articles on Personal, Team, and Organizational Effectiveness:

Names and Faces

Most of us feel recognized, respected, and acknowledged when others use our names. And many of us have difficulty remembering the names of others, especially those we don't know well. How can we get better at connecting names and faces?

Mudfights

When we steer the discussion away from issues to attack the credibility, motives, or character of our debate partners, we often resort to a technique known as the ad hominem attack. It's unfair, it's unethical, and it leads to bad, expensive decisions that we'll probably regret.

The Shower Effect: Sudden Insights

Ever have a brilliant insight, a forehead-slapping moment? You think, "Now I get it!" or "Why didn't I think of this before?" What causes these moments? How can we make them happen sooner?

Finding Work in Tough Times: Strategy

If you're out of work and discouraged — or getting there — you're in great company. Better than ever before. Getting back to work starts with getting to work on finding work. Here's a collection of strategies for the job of finding work.

Contextual Causes of Conflict: I

When destructive conflict erupts, we usually hold responsible only the people directly involved. But the choices of others, and general circumstances, can be the real causes of destructive conflict.

See also Personal, Team, and Organizational Effectiveness and Personal, Team, and Organizational Effectiveness for more related articles.

Forthcoming issues of Point Lookout

Coming April 2: Mitigating the Trauma of Being Laid Off

Trauma is an emotional response to horrible events — accidents, crimes, disasters, physical abuse, emotional abuse, gross injustices — and layoffs. Layoff trauma is real. Employers know how to execute layoffs with compassion, but some act out of cruelty. Know how to defend yourself. Available here and by RSS on April 2.

And on April 9: Defining Workplace Bullying

When we set out to control the incidence of workplace bullying, problem number one is defining bullying behavior. We know much more about bullying in children than we do about adult bullying, and more about adult bullying than we know about workplace bullying. Available here and by RSS on April 9.

Coaching services

I offer email and telephone coaching at both corporate and individual rates. Contact Rick for details at rbrentSgXnAlNVWlhxNIJner@ChacAtZoEYrrmofzZnjPoCanyon.com or (650) 787-6475, or toll-free in the continental US at (866) 378-5470.

Get the ebook!

Past issues of Point Lookout are available in six ebooks:

• Get 2001-2 in Geese Don't Land on Twigs (PDF, )
• Get 2003-4 in Why Dogs Wag (PDF, )
• Get 2005-6 in Loopy Things We Do (PDF, )
• Get 2007-8 in Things We Believe That Maybe Aren't So True (PDF, )
• Get 2009-10 in The Questions Not Asked (PDF, )
• Get all of the first twelve years (2001-2012) in The Collected Issues of Point Lookout (PDF, )

Are you a writer, editor or publisher on deadline? Are you looking for an article that will get people talking and get compliments flying your way? You can have 500-1000 words in your inbox in one hour. License any article from this Web site. More info