When you do a hypothesis test, two types of errors are possible: type I and type II. The risks of these two errors are inversely related and determined by the level of significance and the power for the test. Therefore, you should determine which error has more severe consequences for your situation before you define their risks.
The probability of rejecting the null hypothesis when it is false is equal to 1–β. This value is the power of the test.
Null Hypothesis | ||
Decision | True | False |
Fail to reject | Correct Decision (probability = 1 - α) | Type II Error - fail to reject the null when it is false (probability = β) |
Reject | Type I Error - rejecting the null when it is true (probability = α) | Correct Decision (probability = 1 - β) |
To understand the interrelationship between type I and type II error, and to determine which error has more severe consequences for your situation, consider the following example.
Null hypothesis (H_{0}): μ_{1}= μ_{2}
The two medications are equally effective.
Alternative hypothesis (H_{1}): μ_{1}≠ μ_{2}
The two medications are not equally effective.
A type I error occurs if the researcher rejects the null hypothesis and concludes that the two medications are different when, in fact, they are not. If the medications have the same effectiveness, the researcher may not consider this error too severe because the patients still benefit from the same level of effectiveness regardless of which medicine they take. However, if a type II error occurs, the researcher fails to reject the null hypothesis when it should be rejected. That is, the researcher concludes that the medications are the same when, in fact, they are different. This error is potentially life-threatening if the less-effective medication is sold to the public instead of the more effective one.
As you conduct your hypothesis tests, consider the risks of making type I and type II errors. If the consequences of making one type of error are more severe or costly than making the other type of error, then choose a level of significance and a power for the test that will reflect the relative severity of those consequences.