Editing 2899: Goodhart's Law
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 36: | Line 36: | ||
While there is a temptation to game any metric, measurement is the main objective way of describing the success of an activity and assessing the effect of changes. "Data-driven" or "evidence-based" approaches are used to drive measurable improvements in various areas of society. | While there is a temptation to game any metric, measurement is the main objective way of describing the success of an activity and assessing the effect of changes. "Data-driven" or "evidence-based" approaches are used to drive measurable improvements in various areas of society. | ||
β | Discussions of Goodhart's Law have noted [https://commoncog.com/goodharts-law-not-useful/] that people may respond to a metric by either (1) improving the system, (2) distorting that system (examples below), or (3) distorting the data (e.g., governments publishing false or cherry-picked economic data). Channeling energy toward improvement requires an organization to make (1) more appealing (flexibility and culture) and the others less (transparency, culture, reduced pressure to meet unrealistic goals). Figuring out how to do that involves a slow and thoughtful process unlike White Hat's | + | Discussions of Goodhart's Law have noted [https://commoncog.com/goodharts-law-not-useful/] that people may respond to a metric by either (1) improving the system, (2) distorting that system (examples below), or (3) distorting the data (e.g., governments publishing false or cherry-picked economic data). Channeling energy toward improvement requires an organization to make (1) more appealing (flexibility and culture) and the others less (transparency, culture, reduced pressure to meet unrealistic goals). Figuring out how to do that involves a slow and thoughtful process unlike White Hat's unilateral jump to a new metric. |
===Additional examples of Goodhart's Law=== | ===Additional examples of Goodhart's Law=== |