
The subject of Joe Janesâ latest is close to the heart of many academic researchers: the threshold for âstatistical significanceâ â and the man who, in a âsurprisingly offhand manner,â set that mark for ages afterward at 5 percent, no more no less.
The man in question is English statistician and world-class evolutionary biologist , and the document is his 1925 book, âStatistical Methods for Research Workers.â It embraces the question: How certain of something do you have to be in order to say it is likely so; or as Janes writes, âHow much likelihood, what probability of a result being wrong, we should be willing to live with.â
Modern researchers, he said, would refer to the threshold as âas a p level of .05, a 5 percent probability that a research result doesnât indicate a real effect but rather comes from some random source.â
In the podcast series, Janes, an associate professor in the UW Information School, explores the origin and often evolving meaning of historical documents, both famous and less known. All the podcasts are available online through the iSchool website, and on , where the series has more than 250,000 downloads.
Janes, who has taught statistics and quantitative methods courses for many years, said he knows the topic so well he found the episode challenging to craft without overwhelming the listener or skipping âwhat I thought were important details.â
Still, even he found more to know about Fisher: âI didnât know all the fine details and particularly the rather offhand nature of how his musings wound up becoming hard and fast strictures in the emerging field of statistical analysis,â Janes said.
Crucially, Janes notes that âwhen you have a hard and fast rule, itâs hard and fast, so a study result that just makes it across the line, winding up with a p value of .0499, gets to be called âstatistically significant,â and one that falls just short, with .0501, doesnât.â Though he quickly adds that studies in the medical and pharmaceutical fields often have much more stringent thresholds, at 1 percent or even a tenth of a percent, because they involve the health and safety of the public.
Listen to the podcast to learn more about the man, as Janes writes, âwhose work and ideas youâve probably never heard of but who has had an effect on nearly every statistical study â and the way we understand the way we understand the world â for nearly a century.â