[Verse 1] Grab a thousand strangers off the street at random Margin's plus-or-minus three, that's the tax you pay for truth Bigger ain't always better, it's the picking that's the fulcrum Random sample, tight and clean, beats a million skewed [Verse 2] Nineteen thirty-six, the Digest mailed out postcards To magazine subscribers, phone owners, country club list Landon in a landslide, that's what the headline boasted Roosevelt swept forty-six, the sample flat-out missed Gallup took a smaller crowd but picked it representative Proved the method beats the mass, precision beats the heap That's the lesson tattooed on every pollster's narrative Quality of sample is a promise you gotta keep [Chorus] Sample size, don't let it fool ya, three percent's the wiggle room Random beats enormous, that's the mnemonic, that's the tune Who you ask is everything, the bias hides inside the frame Literary Digest learned it — Gallup's small hand won the game [Verse 3] Regression's like a filter, holding variables still Age and income, education, locked while we test one thing Does the coefficient move the needle, does it pay the bill P-value asks "could this be luck?" — below point-oh-five, let it sing Statistical significance ain't proof carved out in stone It's just a flashlight checking if the pattern's fluke or real Correlation's not causation, that old rule we've always known Controlling for the noise is how we get a cleaner deal [Verse 4] Gerber and Green rolled through New Haven, clipboards knocking doors Door-to-door canvassing, they measured who showed up to vote Randomly assigned who got a visit, who got calls, who got mail-store flyers Face-to-face beat robocalls, that's the finding that they wrote [Chorus] Sample size, don't let it fool ya, three percent's the wiggle room Random beats enormous, that's the mnemonic, that's the tune Who you ask is everything, the bias hides inside the frame Literary Digest learned it — Gallup's small hand won the game [Bridge] Can't flip a coin for every policy, can't randomize the law So we hunt for natural experiments, a lottery, a draw Difference-in-differences, two groups before and after the change Subtract the trend that would've happened, isolate what's strange A border splits a county, one side got the raise, one stayed the same That's the quasi-random gift that lets us play the causal game [Verse 5] So when you see a headline screaming "Poll says this or that," Check the sample, check the method, check who got to speak Numbers don't lie outright but they'll happily go flat If the people counting cut a corner or two weak [Chorus] Sample size, don't let it fool ya, three percent's the wiggle room Random beats enormous, that's the mnemonic, that's the tune Who you ask is everything, the bias hides inside the frame Literary Digest learned it — Gallup's small hand won the game
← Thinking Like a Social Scientist | Cases and Comparisons →