Editing 2110: Error Bars

{{comic
| number    = 2110
| date      = February 11, 2019
| title     = Error Bars
| image     = error_bars.png
| titletext = ...an effect size of 1.68 (95% CI: 1.56 (95% CI: 1.52 (95% CI: 1.504 (95% CI: 1.494 (95% CI: 1.488 (95% CI: 1.485 (95% CI: 1.482 (95% CI: 1.481 (95% CI: 1.4799 (95% CI: 1.4791 (95% CI: 1.4784...
}}

==Explanation==


On statistical charts and graphs, it is common to include {{w|error bars}} showing the probable variation of the actual value from the value shown (or the possible error of the value shown). Since there is always uncertainty in any given measurement, the error bars help an observer evaluate how accurate the data shown is, or the implications if the true value is within the likely error, rather than the exact value shown. There are statistical methods for calculating error bars (they can show a {{w|standard deviation}}, a {{w|standard error}}, or a {{w|confidence interval}}) but the fact that there are multiple ways of calculating them - plus general unfamiliarity with statistical methods - means that people often misinterpret or misunderstand them.

As charts may be of data that has been mathematically processed, the known error from the recording process must also be mathematically processed in order to determine the likely error in the final result - a process called propagation of error. Different transformations of the data result in different transformations of the error, and the correctness of the transformations used can sometimes depend on the subtle differences in the distribution of the source data. At a loss as to how to correctly propagate all the possible sources of error, [[Randall]] instead puts error bars on the ends of his error bars to reflect his uncertainty in the original error bars. However, since his second error bar calculations are also suspect, he puts a third set of error bars on them.  This repeats {{w|ad infinitum}} (though only four levels are drawn), creating a fractal-like object.

In the title text, he says that the {{w|effect size}} of some variable being calculated is 1.68 and follows it with a 95% confidence interval, or CI (a range of possible values which, under repeated sampling, would contain a number within the interval 95% of the time), which would normally be represented by something like "1.68 (95% CI 1.56 - 1.80)." Since he is stating that those bounds are uncertain, he starts with "1.68 (95% CI 1.56" but then puts the 95% CI for that lower bound of the interval, "95% CI 1.52," followed by the lower bound for that value, "95% CI 1.504," and so on. He goes 11 layers deep before resorting to an ellipsis.

In real life, there is not enough data to compute an error bar on error bars. The data being measured have a sampling distribution, e.g. one might make ten measurements of something which come out to 1, 1, 1.1, 1, 1.4, 1, 1, 0.5, 1, and 1, suggesting it is probably close to 1, so there is a range of values that could likely be. However, properties such as the average and standard deviation do not themselves typically have ranges. If one is uncertain that one has computed these correctly, there is not enough data to compute one's own uncertainty in one's skills in any meaningful way; one can claim error bars on error bars, as in this example, but those are just guesses with no statistically useful backing. One way to make the nested error bars valid might be (if one had the time and money) to run the entire experiment ten times, calculating sigmas each time; then there would be a valid (although not necessarily useful) sigma on the sigmas. Then one would have to run the set of ten runs ten times for the next "level" of sigmas, etc. The difficulty of doing this entire process, especially when considering that Randall is only nesting error bars out of ignorance, makes this comic all the more absurd.

==Transcript==
:[A line graph with eight marks on the Y-axis and five marks on the X-axis.  The graph has four points represented by dots and connected by three lines between them.  Each dot has error bars coming out of the top and bottom of it.  The horizontal line delineating the end of each error bar has another set of smaller error bars attached to it.  These second error bars in turn have a still smaller third set of error bars attached to the end of them.  There is a final fourth set of very small error bars attached to the third set, for a total of 56 error bars]

:[Caption below the panels:]
:I don't know how to propagate error correctly, so I just put error bars on all my error bars.


{{comic discussion}}

[[Category:Statistics]]
[[Category:Line graphs]]