explain xkcd - User contributions [en]

2726: Methodology Trial

2023-01-23T05:32:45Z

Daz10000:

{{comic
| number = 2726
| date = January 18, 2023
| title = Methodology Trial
| image = methodology_trial_2x.png
| imagesize = 339x459px
| noexpand = true
| titletext = If you think THAT'S unethical, you should see the stuff we approved via our Placebo IRB.
}}

==Explanation==
{{incomplete|Created by a PLACEBO RESEARCHER - Please change this comment when editing this page. Do NOT delete this tag too soon.}}

When testing the efficacy of a potential medical treatment, researchers compare subjects who received the treatment against subjects who received a {{w|placebo}}. Usually each subject does not know whether they received the treatment or placebo, and neither do the practitioners, until the end of the trial. This distinguishes the actual effects of the treatment from the effects of simply participating in a study. People who receive a placebo (or an ineffective treatment) often believe their treatment is working due to such causes as paying more attention to one's health or expecting to feel better. This misattribution of effect to a non-treatment is called the "placebo effect".

In this comic a team of researchers appears to have studied some medical treatment, using a placebo controlled test. They present their findings in which a particular subset of participants (out of at least four distinct groups) shows an apparently significant result. The graph shows that three groupings have results whose error-bars indicate that they might easily have zero (or neutral) true effects, if not negative ones. But, even at the lowest extent of the accepted uncertainty, the fourth stands out as definitively having some degree of positive effect (of whatever kind this particular graph is plotting).

However, it is revealed that the 'treatment' they were given was also a placebo. Their own study was the subject of a placebo controlled test conducted on their methodology. They were the placebo group, while a different team presumably used the exact same methodology to study the real treatment. Thus, all of this team's findings were due to the placebo effect, or else the trial size and scope allowed a purely statistical 'blip' to occur, instead of there being any real merit to the "treatment". This indicates that their methodology shouldn't be used for any real world applications. This may be a subtle dig at the recent aducanumab Alzheimer's drug trial controversy[https://en.wikipedia.org/wiki/Aducanumab] where post-hoc reanalysis of one subgroup of patients revealed a surprising result when the overall trial had otherwise failed.

The particular flaw in the methodology appears to be dividing too few subjects into too many sub-groups, allowing a chance cluster of anomalous results to overly influence an apparent result. The researcher did find significance in one sub-group, even though in reality there was no signal, just noise, since it was all placebo groups. This references the same p-hacking problem as [[882: Significant]]. Only in this case the researcher themself is the subject of the real trial.

If the non-placebo study had the exact same size and design (as it should have, in such a meta-study), it would cast doubt upon whether any similar-looking findings in London were significant. Especially if they also found that the same subgroup were again exhibiting the sole significant effect, which might reveal an inbuilt flaw in the procedure. On the other hand, it could just further show how likely any particular grouping was to falsely show a result; if all groups had apparently benefited, the chances are that most of them were correct, whether or not [[2268: Further Research is Needed]].

Treatments ''can'' be more effective on specific subgroups of the population; for example, an anti-cancer drug might only work against specific mutations that cause cancer. But any such result needs to have appropriate statistical significance and new subjects from that subgroup should be tested to ensure the result is repeated.

The title text points out how the experiment has almost certainly violated some set of ethical standards, because one researcher offers what he believes to be genuine treatment to a large number of participants only for a third party (the offscreen speaker) to replace all his medicine with placebos, ultimately deceiving the patients. The title text implies that it was approved by a genuine Institutional Review Board (IRB), the group which decides whether a proposed experiment is ethical to perform. However they also have a "placebo IRB", presumably made up of people who have no qualifications to make such judgements well, or perhaps not made up of people at all, but simply a mechanism for generating random decisions.

However, such a methodology trial using all placebos wouldn't necessarily be unethical. In addition to using a placebo, most studies are "double blind" meaning neither the patients nor the doctors/nurses treating them know who is getting the placebo and who is not; only the researchers conducting the study know. This is so doctors/nurses cannot inadvertently let the patients know who is getting real medicine (by acting with remorse around patients they know are not being treated, or being more cheerful with patients they know who are). It is considered perfectly ethical for doctors to give patients what they believe is medicine but is not (the placebo). This is because without the double blind procedure it may not be possible to identify real medicines from ones that have no effect, and the impact of preventing real medicine from being used by millions is greater than the deceit towards the small number receiving a placebo in the experiment. By extension it could be ethical to have the researcher conduct a trial with two placebos without knowing it. For instance if the London team and the team in the comic were finding beneficial effects in new drugs that other researchers found had no effect (or finding other drugs didn't work when others had evidence they did) then it may be worth investigating if their shared methodology has the flaw demonstrated in the article. That way regulatory agencies could exclude their flawed data when they make decisions on what drugs to approve, while the two teams could shift to a better methodology and return to contributing to medical science.

==Transcript==
{{incomplete transcript|Do NOT delete this tag too soon.}}
:[Cueball stands in front of a poster holding a pointer. The poster shows a scatter plot with four points and error bars, with one data point labeled "Subgroup" is marked with an asterisk and is placed somewhat higher up than the other three points.]
:Cueball: However, we see clear evidence that the treatment is more effective than the placebo for some subgroups.
:Off-panel voice: Thank you.
:Off-panel voice: However, we can now reveal that the '''''London''''' team was studying the real treatment. Both groups in your study got a placebo.
:Cueball: Aw, '''''maaan...'''''
:[Caption below panel]
:Researchers hate it when you do placebo controlled trials of their methodology.

{{comic discussion}}
[[Category:Comics featuring Cueball]]
[[Category:Scientific research]]

1471: Gut Fauna

2015-01-09T05:56:17Z

Daz10000: /* Explanation */

{{comic
| number = 1471
| date = January 9, 2015
| title = Gut Fauna
| image = gut_fauna.png
| titletext = I know it seems unpleasant, but of the two ways we typically transfer them, I promise this is the one you want.
}}

==Explanation==
{{incomplete|First draft, needs editing and expanding}}

The microbiome is the collection of "good" bacteria that reside in the human digestion tract and help digest certain foods. It is sometimes referred to as gut flora. The title Gut fauna is a play on words. In this comic, Cueball is visiting a doctor for some unknown problem. The doctor informs him that his gut macrobiome is out of balance, which Cueball responds to with confusion over whether or not she meant the microbiome or macrobiome. A macrobiome, instead of being composed of small organisms such as bacteria, is composed of larger organisms such as mammals. The phrase "gut fauna" would refer to any animals living inside a gut (as the word fauna refers to animals living in an ecosystem). Cueball is right to be worried by the doctor's reference to his macrobiome, as normal humans shouldn't have large animals living inside them. This fear is compounded when the doctor prescribes one wolf for Cueball to swallow, which is normally impossible for humans and would, at the very least, result in major interior damage when the wolf disagrees with being swallowed.

The title text suggests that the comic is not the worst situation that could have occurred, as the doctor refers to another "way" that the wolf could be administered. Typical drug routes include oral (by mouth), through injection, through the rectum, through the eye, or through absorption through the skin. Considering the wolf is intended to be placed in the stomach, the worse "other way" that the doctor is referring to is likely the rectal route, which is definitely less preferably than attempting to swallow a live wolf.

==Transcript==
{{incomplete transcript}}

{{comic discussion}}

1388: Subduction License

2014-06-30T18:30:59Z

Daz10000: /* Explanation */

{{comic
| number = 1388
| date = June 30, 2014
| title = Subduction License
| image = subduction_license.png
| titletext = 'Dude, why can't you just be a normal roommate?' 'Because I'm coming TOWARD you!'
}}

==Explanation==

In {{w|structural geology}}, {{w|subduction}} is the mechanism by which one {{w|tectonic plate}} disappears under another. This process usually creates a {{w|mountain range}} on the second tectonic plate (in particular, an {{w|island arc}} or a {{w|volcanic arc}}, though the volcanic aspect of the process is not shown in the comic).

In this comic, [[Beret Guy]] is very happy because he has just received his ''subduction license''. His roommate [[Cueball]] very reasonably asks him: ''Your what?''. But instead of answering him, Beret Guy begins to move towards him in their small room. It turns out that the license has literally enabled him to initiate subduction: As he moves towards Cueball, he slowly sinks under the floorboards of the room, and in this process he creates a small mountain range on the floor. In the end, much to Cueball's consternation, these mountains turn his desk and chair over. Cueball actually falls out of the frame in the final panel, where Beret Guy is already halfway down beneath the floor.

The title text plays on the double meaning of the word "normal", which Cueball means in the sense of "like most people, not strange," but which Beret Guy interprets in the geological sense. While subduction occurs when two plates crash into each other, a {{w|normal fault}} occurs when two plates are moving away from each other. Here, "normal" is used in the sense of "perpendicular," as the result of a normal fault is often that part of the crust moves vertically downward, forming a {{w|graben}}.

There appears to also be a play on the word 'seduction. The title texts texts refers to coming TOWARD you which is a play on coming ONTO you or to attempt to seduce someone.

A similarly atypical license was mentioned previously in [[410: Math Paper]]. Puns on geological terms (including types of faults) were previously made in [[1082: Geology]].

==Transcript==
:[Beret Guy is looking at some mail he has received while Cueball is at his computer desk at the other side of the room.]
:Beret Guy: Sweet! I finally got my subduction license!
:Cueball: Your what?

:[Beret Guy starts sinking into the ground, causing it to ripple.]
:Cueball: ...What are you doing?

:[Beret Guy sinks further, forming a miniature mountain range in front of him. Cueball is frantically trying to keep his computer steady as his desk tilts.]
:Cueball: ''Stop it! Stop it!''

:[Beret Guy is waist-deep, and snow caps have formed on the mountains. Cueball is falling backwards from his desk, and the monitor unplugs itself from his computer.]
:Cueball: ''AUGH!''

{{comic discussion}}

[[Category:Comics featuring Beret Guy]]
[[Category:Comics featuring Cueball]]