Stat > Power and Sample Size > One-Way ANOVA > Options
Significance
level
Usually, a significance level (denoted as α or alpha) of 0.05 works well. A significance level of 0.05 indicates that the risk of concluding that a difference exists—when, actually, no difference exists—is 5%. It also indicates that the power of the test is 0.05 when there is no difference.
Choose a higher significance level, such as 0.10, to be more certain that you detect any difference that possibly exists. For example, a quality engineer compares the stability of new ball bearings with the stability of current bearings. The engineer must be highly certain that the new ball bearings are stable because unstable ball bearings could cause a disaster. Therefore, the engineer chooses a significance level of 0.10 to be more certain of detecting any possible difference in the stability of the ball bearings.
Choose a lower significance level, such as 0.01, to be more certain that you detect only a difference that actually exists. For example, a scientist at a pharmaceutical company must be very certain that a claim that the company's new drug significantly reduces symptoms is true. The scientist chooses a significance level of 0.01 to be more certain that any significant difference in symptoms does exist.
