You want to determine whether a new gasoline additive has an effect on gas mileage. If the known gas mileage for this specific class of cars is 25 miles per gallon (mpg), the hypotheses for this study are H0: μ = 25 and HA: μ ≠ 25.
The p-value indicates how well the sample evidence supports the rejection of H0. Usually, the smaller the p-value, the heavier the weight of the sample evidence for rejecting H0. More specifically, the p-value is the smallest value of α that results in the rejection of H0. For any value of α > p-value, you fail to reject H0, and for any value of α ≤ p-value, you reject H0.
The p-value is also the proportion of the test statistics from samples of the same size (taken from the same population) that would yield a more extreme value of the test statistic under the assumption that H0 is true. In our t-test example, the test statistic is a function of the mean, and the p-value is .026. This indicates that 2.6% of the samples of size 35, drawn from the population where μ = 25, will produce a mean that provides as strong (or stronger) evidence as the current sample that μ is not equal to 25. Ask yourself which is more likely: that μ = 25 and you just happened to select a very unusual sample; or that μ is not equal to 25?
The p-value is traditionally compared to α values of less than .05 or .01, depending on the field of study. Check journal topics in your field for acceptable values.
In our example, let’s assume a .05 value of α. The p-value of .026 indicates that the mean miles per gallon of all cars of this type (not only the mean of the 35 cars in the study) is probably not equal to 25. A more statistically correct way to state this is “at a significance level of .05, the mean miles per gallon seems to be significantly different from 25.”
Using p-values, then, is simple if you know two key facts: the values of α that are acceptable in your field, and the null and alternative hypotheses for the tests you are using.