Editing 1132: Frequentists vs. Bayesians
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
| Latest revision | Your text | ||
| Line 12: | Line 12: | ||
<blockquote>I seem to have stepped on a hornet’s nest, though, by adding “Frequentist” and “Bayesian” titles to the panels. This came as a surprise to me, in part because I actually added them as an afterthought, along with the final punchline. … The truth is, I genuinely didn’t realize Frequentists and Bayesians were actual camps of people—all of whom are now emailing me. I thought they were loosely-applied labels—perhaps just labels appropriated by the books I had happened to read recently—for the standard textbook approach we learned in science class versus an approach which more carefully incorporates the ideas of prior probabilities.</blockquote> | <blockquote>I seem to have stepped on a hornet’s nest, though, by adding “Frequentist” and “Bayesian” titles to the panels. This came as a surprise to me, in part because I actually added them as an afterthought, along with the final punchline. … The truth is, I genuinely didn’t realize Frequentists and Bayesians were actual camps of people—all of whom are now emailing me. I thought they were loosely-applied labels—perhaps just labels appropriated by the books I had happened to read recently—for the standard textbook approach we learned in science class versus an approach which more carefully incorporates the ideas of prior probabilities.</blockquote> | ||
| − | The " | + | The "frequentist" statistician is (mis)applying the common standard of "{{w|P-value|p}}<0.05". In a scientific study, a result is presumed to provide strong evidence if, given that the {{w|null hypothesis}}, a default position that the observations are unrelated (in this case, that the sun has ''not'' gone nova), there is less than a 5% chance that the result was merely random. (The null hypothesis was also referenced in [[892: Null Hypothesis]].) |
| − | Since the likelihood of rolling double sixes is below this 5% threshold, the "frequentist" decides (by this rule of thumb) to accept the detector's output as correct. The " | + | Since the likelihood of rolling double sixes is below this 5% threshold, the "frequentist" decides (by this rule of thumb) to accept the detector's output as correct. The "Bayesian" statistician has, instead, applied at least a small measure of probabilistic reasoning ({{w|Bayesian inference}}) to determine that the unlikeliness of the detector lying is greatly outweighed by the unlikeliness of the sun exploding. Therefore, he concludes that the sun has ''not'' exploded and the detector is lying. |
A real statistician (frequentist or Bayesian) would probably demand a lower ''p''-value before concluding that a test shows the Sun has exploded; physicists tend to use 5 sigma, or about 1 in 3.5 million, as the standard before declaring major results, like discovering new particles. This would be equivalent to rolling between eight and nine dice and getting all sixes, although this is still not "very good" compared to the actual expected likelihood of the Sun spontaneously going nova, as discussed below. | A real statistician (frequentist or Bayesian) would probably demand a lower ''p''-value before concluding that a test shows the Sun has exploded; physicists tend to use 5 sigma, or about 1 in 3.5 million, as the standard before declaring major results, like discovering new particles. This would be equivalent to rolling between eight and nine dice and getting all sixes, although this is still not "very good" compared to the actual expected likelihood of the Sun spontaneously going nova, as discussed below. | ||
The line, "Bet you $50 it hasn't", is a reference to the approach of a leading Bayesian scholar, {{w|Bruno de Finetti}}, who made extensive use of bets in his examples and thought experiments. See {{w|Coherence (philosophical gambling strategy)}} for more information on his work. In this case, however, the bet is also a joke because we would all be dead if the sun exploded. If the Bayesian wins the bet, he gets money, and if he loses, they'll both be dead before money can be paid. This underlines the absurdity of the premise and emphasizes the need to consider context when examining probability. | The line, "Bet you $50 it hasn't", is a reference to the approach of a leading Bayesian scholar, {{w|Bruno de Finetti}}, who made extensive use of bets in his examples and thought experiments. See {{w|Coherence (philosophical gambling strategy)}} for more information on his work. In this case, however, the bet is also a joke because we would all be dead if the sun exploded. If the Bayesian wins the bet, he gets money, and if he loses, they'll both be dead before money can be paid. This underlines the absurdity of the premise and emphasizes the need to consider context when examining probability. | ||
| − | |||
| − | |||
The title text refers to a classic series of logic puzzles known as {{w|Knights and Knaves#Fork in the road|Knights and Knaves}}, where there are two guards in front of two exit doors, one of which is real and the other leads to death. One guard is a liar and the other tells the truth. The visitor doesn't know which is which, and is allowed to ask one question to one guard. The solution is to ask either guard what the other one would say is the real exit, then choose the opposite. Two such guards were featured in the 1986 Jim Henson movie ''{{w|Labyrinth (1986 film)|Labyrinth}}'', hence the mention of "A LABYRINTH GUARD" here. A labyrinth was also mentioned in [[246: Labyrinth Puzzle]]. | The title text refers to a classic series of logic puzzles known as {{w|Knights and Knaves#Fork in the road|Knights and Knaves}}, where there are two guards in front of two exit doors, one of which is real and the other leads to death. One guard is a liar and the other tells the truth. The visitor doesn't know which is which, and is allowed to ask one question to one guard. The solution is to ask either guard what the other one would say is the real exit, then choose the opposite. Two such guards were featured in the 1986 Jim Henson movie ''{{w|Labyrinth (1986 film)|Labyrinth}}'', hence the mention of "A LABYRINTH GUARD" here. A labyrinth was also mentioned in [[246: Labyrinth Puzzle]]. | ||
| Line 31: | Line 29: | ||
There is no record of any star ever spontaneously exploding—they always show signs of deterioration long before their explosion—so the probability is near zero. For the sake of a number, though, consider that the sun's estimated lifespan is 10 billion years. Let's say the test is run every hour, twelve hours a day (at night time). This gives us a probability of the Sun exploding at one in 4.38×10<sup>13</sup>. Assuming this detector is otherwise reliable, when the detector reports a solar explosion, there are two possibilities: | There is no record of any star ever spontaneously exploding—they always show signs of deterioration long before their explosion—so the probability is near zero. For the sake of a number, though, consider that the sun's estimated lifespan is 10 billion years. Let's say the test is run every hour, twelve hours a day (at night time). This gives us a probability of the Sun exploding at one in 4.38×10<sup>13</sup>. Assuming this detector is otherwise reliable, when the detector reports a solar explosion, there are two possibilities: | ||
# The sun '''has''' exploded (one in 4.38×10<sup>13</sup>) and the detector '''is''' telling the truth (35 in 36). This event has a total probability of about 1/(4.38×10<sup>13</sup>) × 35/36 or about one in 4.50×10<sup>13</sup> | # The sun '''has''' exploded (one in 4.38×10<sup>13</sup>) and the detector '''is''' telling the truth (35 in 36). This event has a total probability of about 1/(4.38×10<sup>13</sup>) × 35/36 or about one in 4.50×10<sup>13</sup> | ||
| − | # The sun '''hasn't''' exploded (4.38×10<sup>13</sup> − 1 in 4.38×10<sup>13</sup>) and the detector '''is not''' telling the truth (1 in 36). This event has a total probability of about (4.38×10<sup>13</sup> − 1) / 4.38×10<sup>13</sup> × 1/36 or about one in 36 | + | # The sun '''hasn't''' exploded (4.38×10<sup>13</sup> − 1 in 4.38×10<sup>13</sup>) and the detector '''is not''' telling the truth (1 in 36). This event has a total probability of about (4.38×10<sup>13</sup> − 1) / 4.38×10<sup>13</sup> × 1/36 or about one in 36 |
Clearly the sun exploding is not the most likely option. Indeed, Bayes' theorem can be used to find the probability that the Sun has exploded, given a result of "yes" and the prior probability given above: | Clearly the sun exploding is not the most likely option. Indeed, Bayes' theorem can be used to find the probability that the Sun has exploded, given a result of "yes" and the prior probability given above: | ||
