p-Value – Statistics Basics

00:00 I'm seeking to reject that null hypothesis. Now imagine you're doing a test, a measurement, maybe comparing a treatment group versus a placebo group in your randomized controlled trial, and you get a result. And let's say you do that test again, you get a different result. You did that test 1 million billion number of times, you get 1 million billion different results. Sometimes you get the same result, sometimes you don't. If you're to plot the frequency with which certain results occur, you'll get a shape that looks like a bell curve or a normal curve. Now the P value is a value under that normal curve that tells you whether or not it's probable that the null hypothesis is true. More precisely, it's telling you the probability that you rejecting that null hypothesis was done incorrectly.

00:50 It’s a difficult concept to absorb but just rely upon this bit of rule; if your P value is less than a certain cut-off value, you can reliably reject the null hypothesis. What's that cut-off value? We call that value an alpha value and it is usually set at 0.05 or 0.01.

01:12 There is no the reason for that, that is just history, 0.05 is the most common one. So again, if your P value is less than 0.05, you can reliably reject the null hypothesis and conclude that you probably found something, you probably found an effect. For example, if we're testing whether the average heights of two different groups of children are different and we perform a test to do so, let’s say a T-test, which is the appropriate test in this case, and you find a P value of 0.02. 0.02 is less than 0, so we conclude that we can probably reject our null hypothesis. How likely is it that that rejection was done in error? Well that’s the P value, 0, 2%, there is a 2% chance that I rejected that null hypothesis in error, we say that's a good enough number. Now defining or interpreting a P value is problematic at the best of times, there is a convenient, though inaccurate interpretation that the P value is the probability that your result was due to chance alone. So in the previous example, there is a 2% chance that your result was all luck, that's not really what's going on here. More accurately the P value is a probability that your test incorrectly rejected the null, that's all it’s saying. Now if that's too confusing, just remember this very simple rule, if the P is low, the null hypothesis must go, you want a low P value, it means you found something of statistical importance. What is low? Low is a P less than alpha. What’s alpha? Usually 0.05, sometimes 0.01. Now there is a different way to express statistical