The P-value of a test of a null hypothesis is the probability that a the null hypothesis is true. b the null hypothesis is false. c assuming the null hypothesis is false, the test statistic will take a value at least as extreme as that actually observed. d assuming the null hypothesis is true, the test statistic will take a value at least as extreme as that actually observed. e the alternative hypothesis is true.

A good way to think about the null hypothesis is that it is a statement that is assumed to be true until we have found evidence to deem it false and conclude in favor of the alternative (other) hypothesis specified.

- normally null is H o and alternative is H 1

The evidence we find is through the test statistic that is calculated depending on what type of test you are conducting.

P-value:

"Assuming that the null hypothesis is true, the probability that you get a test statistic at least as extreme or more extreme than that which you observe (the test stat you calculate) if you were to perform the test multiple times."

Full Explanation:

A lot of people use significance levels to allocate a "r e j e c t i o n region" which is just an area under the curve beyond the value associated with 5% of the distribution on the tail end. If our test stat falls into this region we conclude the null hypothesis to be false.

eg. 5% significance level meaning that the p-value is given to be 5%. There is a statistic that corresponds with this 5% level and we call this the critical value.

- 5% is the unofficial benchmark indicating a small enough probability to r e j e c t the null hypothesis.

So you perform the test as usual, by calculating the test statistic under the null hypothesis (ie. if the null hypothesis is true this is the statistic we will observe accordingly). However, if the value of the test statistic is large enough, (larger than the critical value statistic) then it will fall into the r e j e c t i o n region. This means that there is enough evidence against the null hypothesis, and we can conclude that the null hypothesis is false in favor of the alternative hypothesis.

Note that the test statistic that we calculate also has a p-value (probability) associated with it. This is what we are referring to in the previous explanation; that if the test statistic is large enough it will have a very small probability associated with it. Meaning the probability of getting a test statistic as "extreme" (as large) as this one or "more extreme" is very small. This is called the observed p-value. So given that we have calculated it under the null hypothesis, the assumption can be deemed false. Thus, we state that we have enough evidence to do so and conclude in favor of the alternative hypothesis.

*read further if it is still unclear; if not skip to "Some things to remember"

It helps to picture the bell curve of the normal distribution. On the bottom, the values run from a negative to a positive and the peak is at zero (if it is a standard normal distribution). These are potential values of the test statistic.

The normal distribution has a function associated with it. To find the probability you find the area under the graph (this is never done manually for the normal distribution, we have tables for this). So the full area across the whole range is equal to 1. (probabilities run from zero to one). The r e j e c t i o n region is a point on the x axis and the p-value is the area under the curve that falls to the right of it.

Some things to remember:

The observed value is the test statistic that you have calculated.

You perform the test under the null hypothesis.

You have a distribution of values that follows a specific probability distribution, say the normal distribution. The test statistic you calculate will fall along that range (think of the bell curve of the normal distribution that shows both negative and positive values with the center being either zero for a standard normal distribution or the null hypothesis estimate).

When you calculate the test statistic it will fall along that range of values. Now, as a probability distribution, each statistic demarcates a specific probability.

The area under the distribution represents the probability. The probability below the entire curve is equal to one. So if your test statistic falls at a specific point , then your p-value will be the area under the curve that lies beyond that test statistic.

The p-value would be the area under the curve (indicating a probability) that lies beyond the value of the test statistic.

The test can be one sided or two sided:

If it's one sided the 5% (0.05) will be on the right hand side of the distribution.
If it's two sided the 5% (0.05) will be divided in two at each end of the symmetrical distribution (ie. 2.5% on each side).
the two methods: critical value method (r e j e c t i o n region) and the observed p-value method (the p-value of the t-stat you calculate).