Talk:2533: Slope Hypothesis Testing

Explain xkcd: It's 'cause you're dumb.
Revision as of 14:30, 27 October 2021 by GreatWyrmGold (talk | contribs)
Jump to: navigation, search

In the line "Randall has repeatedly made comics about this hopeful error", should specific examples be provided? I know /882 is one, but I'm blanking on any others. 10:21, 26 October 2021 (UTC)

Definitely, otherwise it would not be very useful. -- 13:10, 26 October 2021 (UTC)
hi, I added the line. <thinking-out-loud removed in copyediting>. I think there was one where article titles that blatantly used poor statistical techniques were listed, not sure. 14:40, 26 October 2021 (UTC)

I love it. Fwacer (talk) 02:52, 26 October 2021 (UTC)

I imagine that the problem here is that the errors are not independent. I can't find anything else wrong with this, but I feel like there's something obvious I'm not seeing. They might revoke my statistics degree if I miss something big here, hehe.--Troy0 (talk) 03:06, 26 October 2021 (UTC)

The scores are clearly the one score they originally (sometime prior to the expanded test) received. Either that or multiple tests with the same exam questions without having given them enough feedback to change their answer-scheme at all. The volumes are probably a "good go at screaming" on demand, belying any obvious "test result -> thus intensity of scream" (what might be expected if the scream(s) of shock/joy/frustration were recorded immediately upon hearing a score).
What they have here is a 1D distribution of scream-ability/tendency (which was originally a single datum), arbitrarily set against test scores. (Could as easily have been against shoe-size, father's income-before-tax, a single dice-roll, etc.)
I initially assumed that the participants are screaming in response to learning their scores, so the relationship is not arbitrary -- the students with good scores sceam loudly with joy. Barmar (talk) 14:45, 26 October 2021 (UTC)
Perhaps it was that (except the screaming volume is clearly confounded by other factors, such as how loudly they normally scream) but asking them to "scream again" seems to show far more personal correlation than emotional attachment, for they are the same people but time has passed to reduce the spontenity of the response and their newer submissions are at best "try to scream like you did when you heard your score the first time". Which is problematic and not really a valid new response to add to the list. At best, it's a test of replay consistency (now unlinked from the original feeling). 15:57, 26 October 2021 (UTC)
Whether there was an original theory that grades correlated with intensity of vocalisation is perhaps a valid speculation, but clearly the design of the test is wrong. Too few datum points, in the first instance, and the wrong way to increase them when they find out their original failing.
The true solution is to recruit more subject. (And justify properly if it's intensity of spontaneous result-prompted evocations or merely general ability to be loud that is the quality the wish to measure. Either could be valid, but it's not obvious that the latter is indeed the one that they meant to measure.) 04:21, 26 October 2021 (UTC)
It's pretty straightforward. This is a simple linear regression, Y = α + βX + ε, where α and β are parameters and ε is a random variable (the error term). Their point estimations for α and β are correct. But their confidence intervals (and thus p-values) are wrong, because they are based on a false assumption. They constructed their intervals assuming ε was normally distributed, which it clearly is not. ε will always be approximately normally distributed if the central limit theorem applies, but it does not apply here. The central limit theorem requires that the samples be independent and identically distributed. Here, they are identically distributed, but they are not remotely independent. After all, the same people were selected over and over again. Therefore ε will probably not be randomly distributed (it isn't even close), and the confidence intervals (and so p-values) are wrong. 09:10, 26 October 2021 (UTC)
You seem to be the only person so far who's learned in academia why this is wrong. Is the current state of the article correct? 14:31, 26 October 2021 (UTC)
The scientific error isn't quite what people are saying it is. The issue here is not "reusing a single test score" or an issue with non-normality of errors, the issue is that the data are *nested* within participants and that isn't being accounted for. There are fairly standard ways of managing this, at least in the social science literature (and these ways are statistically valid), most commonly the use of multilevel modeling (also known as hierarchical linear modeling). This accounts for the correlated nature of the errors. Now, even using the right method they're not going to attain statistical significance, but at least they aren't making a statistical mistake.

I don't think the title text speakers are unidentified, I'm pretty sure it's a direct continuation of the dialogue in the last panel. Esogalt (talk) 04:11, 26 October 2021 (UTC)

I agree. the second speaker starts to say "I said, are you sure--", this is the start of Cueball's last line. I think this is intended to be Cueball and Megan trying to talk about the results while the students are still screaming. TomW1605 (talk) 06:45, 26 October 2021 (UTC)
It could also be the case that their hypothesis was true and they failed so badly at statistics, that their voices are inaudible now.
Because, since they didn't write that test, their score is zero. 14:29, 26 October 2021 (UTC)

Is there a polynomial that better fits this data? 14:29, 26 October 2021 (UTC)

Given three points, there's a circle that exactly fits them... ;) 15:57, 26 October 2021 (UTC)
Also, infinite number of polynomials which fits this data exactly. -- Hkmaly (talk) 23:06, 26 October 2021 (UTC)
But exactly one circle (assuming all mutually different, and not already upon a line), in an elegantly obvious manner. Therefore clearly the more individually significant statistical match for three datum points. :P 11:50, 27 October 2021 (UTC)

Speaking as the person who wrote the explanation for "slope hypothesis testing," we need someone to write a better explanation for it. It's just a placeholder meant to patch a gap until someone who knows what they're talking about (or what to google) can write a better one. GreatWyrmGold (talk) 14:30, 27 October 2021 (UTC)