P-Hacking

P-hacking, also known as data dredging or data snooping, is the misuse of data analysis to find patterns that can be presented as statistically significant, which dramatically increases and understates the true risk of a false positive. It is typically done by running many statistical tests on a data set and reporting only the ones that come back significant, often by exhaustively searching combinations of variables or subgroups for an apparent correlation or difference; because conventional significance testing accepts some baseline risk of a mistaken conclusion, testing enough hypotheses makes it virtually certain that some will be reported significant purely by chance, even though the result is misleading. The term was coined in a 2014 paper by the researchers behind the blog Data Colada, who have focused on uncovering exactly this problem within social science research, including psychology's own replication crisis; the practice is an example of the broader multiple comparisons problem, and when misused it is a questionable research practice that can undermine scientific integrity.