Editing 2323: Modeling Study

{{comic
| number    = 2323
| date      = June 22, 2020
| title     = Modeling Study
| image     = modeling_study.png
| titletext = You've got questions, we've got assumptions.
}}

==Explanation==

In this comic, a humorous comparison is drawn between two common types of scientific studies: {{w|empirical research}}, where an experiment is designed to test a scientific theory, and {{w|mathematical modeling}}, where mathematical formulations are produced to predict how physical systems behave under given circumstances. In empirical studies, hard questions about the limitations of existing theory tend to be addressed in the abstract, which is the brief summary of the paper that is presented at the beginning of most scientific articles. In modeling studies, assumptions based on existing theory are built into the model, and any problems associated with these assumptions tend to be discussed in the methods section, which outlines the design of an experiment in the case of an empirical study, or how the model was designed and the reasoning behind the choices made in the case of a modeling study.  In the empirical study, the proverbial "big red problem box" is stated up-front where everyone who finds the paper will read it, while in the modeling study, it's buried in the middle of the paper, where it's less likely to be read.

The caption opens like a typical statement in favor of modeling studies, "A mathematical model is a powerful tool for taking hard problems," but while a researcher who works with models might go on to say "...and breaking them down," or "...and studying them in ways that would be impractical for empirical studies," Randall concludes that they can't actually make hard problems any easier.  His title text, "You've got questions, we've got assumptions," plays on the slogan of the now-defunct electronics chain Radio Shack of "You've got questions, we've got answers" by pointing out that any answers provided are built on assumptions by the modelers.  In other words, {{w|garbage in, garbage out}}.

Randall doesn't call this a [[:Category:Tips|"tip"]], but it does fit in with his [[:Category:Science tip|science tip]] in [[2311: Confidence Interval]], namely, that "If your model is bad enough, the confidence intervals will fall outside the printable area."  Much as that tip suggests that a model's results can be made to look more impressive by hiding the error bounds outside the printed area of a graph, this comic strip suggests that acknowledgments of problems can be moved to less-trafficked parts of the paper by switching from empirical to modeling studies.

For a more concrete example, consider the 2020 pandemic of COVID-19.  Empirical studies measure things like infections, hospitalizations, and deaths, and the circumstances that lead to those events, and attempt to answer questions about how COVID-19 spreads, what measures are effective in preventing its transmission, what those measures' other costs and side effects are, and what therapies are effective in treating cases.  These are made difficult by gaps in testing capability, the imperfections of those tests which are available, and the fact that all of the conditions of society are interconnected and constantly changing -- there is no "control universe" or any way to go back and try different ideas.  Modeling studies offer the possibility to simulate thousands or millions of possible pandemics, to hopefully figure out those variables' effects in advance and offer guidance to governments and health workers, but without specific knowledge of COVID-19's properties. Especially in the early days of the pandemic, modelers must make assumptions about how COVID-19 spreads, kills, and is (or is not) treated.  For pandemics especially, which behave exponentially until they are brought under control (or the pathogen burns through its host population), even small changes in model assumptions can lead to orders of magnitude difference between equally-plausible projections (such as [https://www.newscientist.com/article/2238578-uk-has-enough-intensive-care-units-for-coronavirus-expert-predicts/ projected deaths falling from half a million to 20,000]; note that all of these outcomes are [https://www.abs.gov.au/websitedbs/a3121120.nsf/home/statistical+language+-+estimate+and+projection ''projections'' rather than ''predictions''], even though they are often presented as the latter in the press).  Even if all such projections are made earnestly, with the best available information, it can lead to distrust of the models and their results, especially if the models are presented to non-experts with too much certainty. Moreover, projections which attempt to project how many deaths will happen if nothing is done to stop an outbreak, and which are therefore more dire than those which simply assume that protective actions will be taken, are self-defeating ''specifically because'' they cause people to take the outbreak seriously and take protective actions.

==Transcript==

:[There are two columns.]

:[The column on the left is a piece of paper labeled "Empirical Study". The paper consists of the sections "Abstract", "Introduction", "Methods", "Results", and "Discussion". Each section consists of several horizontal lines meant to represent blocks of text. All sections except "Abstract" have gray, dulled text, while "Abstract" has fully black text to separate it from the other sections. In the middle of the "Abstract" section, there is a large red rectangle. Inside this rectangle is the word "Problem" in large red letters.]

:[The column on the right is a piece of paper labeled "Modeling Study". It consists of the same sections with the same highlighting, but the large red rectangle with the word "Problem" is in the "Methods" section instead of the "Abstract" section. Because of this, it is dulled to match the rest of the "Methods" section.]

:[There is a curvy black arrow pointing from the red box in the paper on the left to the red box in the paper on the right.]

:[Caption below the panel:]
:A mathematical model is a powerful tool for taking hard problems and moving them to the methods section.

{{comic discussion}}
[[Category:Science]]
[[Category:Math]]
[[Category:Scientific research]]
[[Category:Comics with color]]