Inflation bias

Inflation bias, also referred to as “p-hacking” or “selective reporting”, occurs when researchers try multiple different analyses behind the scenes, and only publish those that produce a statistically significant result.

Some common practices that are actually p-hacking include:

  • Conducting analyses part way through data collection to determine whether to continue collecting data
  • Recording many variables, but only reporting a subset
  • Deciding whether to include or exclude outliers post-analysis
  • excluding, combining, or splitting treatment groups post-analysis
  • including or excluding covariates post-analysis
  • stopping data exploration if an analysis yields a significant p-value

References

Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015 Mar 13;13(3):e1002106. doi: 10.1371/journal.pbio.1002106. PMID: 25768323

Cite this article as: Justin Morgenstern, "Inflation bias", First10EM blog, June 8, 2021. Available at: https://first10em.com/ebm/inflation-bias/.
%d bloggers like this: