P-values – What does it mean to be Significant?

A p-value is the most common measurement of significance in statistics. It is a number between 0 and 1 that represents the probability of obtaining a particular result by random chance. As such a lower number is a better indication that differences in our data are real! Significance can be thought of as “Do we have enough data to make the claim?” As we collect more and more data, we can be more and more sure that our results are not because of random chance.

The smaller the p-value, the stronger the evidence against the null hypothesis and the more confident we can be in rejecting it.

In most biological studies, we use a P-value threshold of .05. Meaning, that at a p value of .05 we would have a false positive from random noise about 5% of the time. Any value equal to or lower than .05 tells us that our data is significant and we are able to reject our null hypothesis. This .05 value is linked directly to the common 95% confidence interval. Keep in mind that this threshold is arbitrary, and many statisticians recommend using a stricter threshold such as .01.

The p-value and significance is also the most misunderstood metric in statistics. As we’ll discover in our next lesson, significant results do NOT mean meaningful results. Rather, they tell us “we have enough data to detect a difference”.