Inflation bias, also referred to as “p-hacking” or “selective reporting”, occurs when researchers try multiple different analyses behind the scenes, and only publish those that produce a statistically significant result.
Some common practices that are actually p-hacking include:
- Conducting analyses part way through data collection to determine whether to continue collecting data
- Recording many variables, but only reporting a subset
- Deciding whether to include or exclude outliers post-analysis
- excluding, combining, or splitting treatment groups post-analysis
- including or excluding covariates post-analysis
- stopping data exploration if an analysis yields a significant p-value
References
Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015 Mar 13;13(3):e1002106. doi: 10.1371/journal.pbio.1002106. PMID: 25768323
Morgenstern, J. Inflation bias, First10EM, June 8, 2021. Available at:
https://doi.org/10.51684/FIRS.80297