Inflation bias

Inflation bias, also referred to as “p-hacking” or “selective reporting”, occurs when researchers try multiple different analyses behind the scenes, and only publish those that produce a statistically significant result.

Some common practices that are actually p-hacking include:

• Conducting analyses part way through data collection to determine whether to continue collecting data
• Recording many variables, but only reporting a subset
• Deciding whether to include or exclude outliers post-analysis
• excluding, combining, or splitting treatment groups post-analysis
• including or excluding covariates post-analysis
• stopping data exploration if an analysis yields a significant p-value

References

Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015 Mar 13;13(3):e1002106. doi: 10.1371/journal.pbio.1002106. PMID: 25768323