In This Topic
Power for difference
The first line of output indicates how the hypotheses are specified for the equivalence test.
"Power for difference" indicates that the hypotheses are specified in terms of the difference between the mean of the test population and the mean of the reference population (Test mean – reference mean).
Power for ratio
The first line of output indicates how the hypotheses are specified for the equivalence test.
"Power for ratio" indicates that the hypotheses are specified in terms of the ratio between the mean of the test population and the mean of the reference population by log transformation (Test mean / reference mean).
Null hypothesis and alternative hypothesis
- Null hypothesis
- Minitab tests one or both of the following null hypotheses, depending on the alternative hypothesis that you selected:
- The difference (or ratio) between the mean of the test population and the mean of the reference population is greater than or equal to the upper equivalence limit.
- The difference (or ratio) between the mean of the test population and the mean of the reference population is less than or equal to the lower equivalence limit.
- Alternative hypothesis
- The alternative hypothesis states one or both of the following:
- The difference (or ratio) between the mean of the test population and the mean of the reference population is less than the upper equivalence limit.
- The difference (or ratio) between the mean of the test population and the mean of the reference population is greater than the lower equivalence limit.
Interpretation
Use the null and alternative hypotheses to verify that the equivalence criteria are correct and that you have selected the appropriate alternative hypothesis to test.
Method
| Power for difference: | Test mean - reference mean |
|---|---|
| Null hypothesis: | Difference ≤ -0.425 or Difference ≥ 0.425 |
| Alternative hypothesis: | -0.425 < Difference < 0.425 |
| α level: | 0.05 |
In these results, Minitab tests two null hypotheses:
- The difference between the mean of the test population and the mean of the reference population is less than or equal to the lower equivalence limit of −0.425.
- The difference between the mean of the test population and the mean of the reference population is greater than or equal to the upper equivalence limit of 0.425.
α (alpha)
The significance level (denoted by alpha or α) is the maximum acceptable level of risk for rejecting the null hypothesis when the null hypothesis is true (type I error). For example, if you perform an equivalence test using the default hypotheses, an α of 0.05 indicates a 5% risk of claiming equivalence when the difference between the test mean and the reference mean does not actually fall within the equivalence limits.
The α-level for an equivalence test also determines the confidence level for the confidence interval. The confidence level is (1 – α) x 100% by default. If you use the alternative method of calculating the confidence interval, the confidence level is (1 – 2α) x 100%.
Interpretation
Use the significance level to minimize the power value of the test when the null hypothesis (H0) is true. Higher values for the significance level give the test more power, but also increase the chance of making a type I error, which is rejecting the null hypothesis when it is true.
Assumed within-subject standard deviation
The within-subject standard deviation is the standard deviation of multiple response values from the same participant. This measure of standard deviation estimates the magnitude of the random error in the response measurements from the same participant after eliminating treatment effects, period effects, and other systematic effects. Higher values indicate greater variability in the response values of each participant.
Interpretation
If you perform a power analysis for an equivalence test about the difference, you must enter the within-subject standard deviation. This assumed within-subject standard deviation is an estimate of the population standard deviation.
Minitab uses the assumed within-subject standard deviation to calculate the power of the test. Higher values of the within-subject standard deviation indicate more variation or "noise" in the data, which decreases the statistical power of a test.
Assumed within-subject coefficient of variation
The within-subject coefficient of variation is the coefficient of variation of multiple response values from the same participant. The coefficient of variation is equal to the standard deviation divided by the mean.
This measure of variation estimates the magnitude of the random error in the response measurements from the same participant after eliminating treatment effects, period effects, and other systematic effects. Higher values indicate greater variability in the response values of each participant.
Interpretation
If you perform a power analysis for an equivalence test about the ratio, you must enter the within-subject coefficient of variation. This assumed within-subject coefficient of variation is an estimate of the population coefficient of variation.
Minitab uses the assumed within-subject coefficient of variation to calculate the power of the test. Higher values of the within-subject coefficient of variation indicate more variation or "noise" in the data, which decreases the statistical power of a test.
Difference
This value represents the difference between the mean of the test population and the mean of the reference population.
Note
The definitions and interpretation in this topic apply to a standard equivalence test that uses the default alternative hypothesis (Lower limit < test mean - reference mean < upper limit).
Interpretation
If you enter the sample size and the power of the test, then Minitab calculates the difference that the test can accommodate at the specified power and sample size. For larger sample sizes, the difference can be closer to your equivalence limits.
To more fully investigate the relationship between the sample size and the difference that the test can accommodate at a given power, use the power curve.
Method
| Power for difference: | Test mean - reference mean |
|---|---|
| Null hypothesis: | Difference ≤ -0.425 or Difference ≥ 0.425 |
| Alternative hypothesis: | -0.425 < Difference < 0.425 |
| α level: | 0.05 |
Results
| Sample Size | Power | Difference |
|---|---|---|
| 4 | 0.9 | -0.278474 |
| 4 | 0.9 | 0.278474 |
| 8 | 0.9 | -0.329164 |
| 8 | 0.9 | 0.329164 |
| 12 | 0.9 | -0.348248 |
| 12 | 0.9 | 0.348248 |
These results show how increasing the sample size increases the size of the difference that can be accommodated at a given power level:
- With 4 participants in each sequence, the power of the test is at least 0.9 when the difference is between approximately −0.278 and 0.278.
- With 8 participants in each sequence, the power of the test is at least 0.9 when the difference is between approximately −0.329 and 0.329.
- With 12 participants in each sequence, the power of the test is at least 0.9 when the difference is between approximately −0.348 and 0.348.
Ratio
This value represents the ratio of the mean of the test population to the mean of the reference population. To perform power calculations for a ratio you must select a hypothesis about Test mean / reference mean (Ratio, by log transformation).
Note
The definitions and interpretation in this topic apply to an equivalence test that uses the default alternative hypothesis for the ratio (Lower limit < test mean / reference mean < upper limit).
Interpretation
If you enter the sample size and the power of the test, then Minitab calculates the minimum and maximum ratios that the test can accommodate at the specified power and sample size. For larger sample sizes, the ratio can be closer to your equivalence limits.
To more fully investigate the relationship between the sample size and the ratios that the test can accommodate at a given power, use the power curve.
Method
| Power for ratio: | Test mean / reference mean |
|---|---|
| Null hypothesis: | Ratio ≤ 0.9 or Ratio ≥ 1.1 |
| Alternative hypothesis: | 0.9 < Ratio < 1.1 |
| α level: | 0.05 |
Results
| Sample Size | Power | Ratio |
|---|---|---|
| 10 | 0.9 | 0.91749 |
| 10 | 0.9 | 1.07903 |
| 20 | 0.9 | 0.91207 |
| 20 | 0.9 | 1.08544 |
These results show how increasing the sample size increases the range of ratios that the test can accommodate at a given power level:
- With 10 observations in each sequence, the power of the test is at least 0.9 when the ratio is between approximately 0.92 and 1.08.
- With 20 observations in each sequence, the power of the test is at least 0.9 when the ratio is between approximately 0.91 and 1.09.
Sample Size
The sample size is the total number of observations in the sample. For a 2x2 crossover study, the sample size refers to the number of participants in each sequence of the study.
Interpretation
Use the sample size to estimate how many observations you need to obtain a certain power value for the equivalence test at a specific difference.
If you enter a difference (or ratio) and a power value for the test, then Minitab calculates how large your sample must be. Because sample sizes are whole numbers, the actual power of the test might be slightly greater than the power value that you specify.
If you increase the sample size, the power of the test also increases. You want enough observations in your sample to achieve adequate power. But you don't want a sample size so large that you waste time and money on unnecessary sampling or detect unimportant differences to be statistically significant.
To more fully investigate the relationship between the sample size and the difference (or ratio) that the test can accommodate at a given power, use the power curve.
Method
| Power for difference: | Test mean - reference mean |
|---|---|
| Null hypothesis: | Difference ≤ -0.425 or Difference ≥ 0.425 |
| Alternative hypothesis: | -0.425 < Difference < 0.425 |
| α level: | 0.05 |
Results
| Difference | Sample Size | Target Power | Actual Power |
|---|---|---|---|
| 0.0 | 2 | 0.9 | 0.978589 |
| 0.1 | 2 | 0.9 | 0.931544 |
| 0.2 | 3 | 0.9 | 0.972795 |
| 0.3 | 6 | 0.9 | 0.943646 |
| 0.4 | 107 | 0.9 | 0.900500 |
These results show that, as the size of the difference increases and approaches the value of the equivalence limit, you need a larger sample size to achieve a given power. If the difference is 0.1, then you need only 2 participants in each sequence to achieve a power of 0.9. If the difference is 0.4, you need at least 107 participants in each sequence to achieve a power of 0.9.
Power
The power of an equivalence test is the probability that the test will demonstrate that the difference is within your equivalence limits, when it actually is. The power of an equivalence test is affected by the sample size, the difference, the equivalence limits, the variability of the data, and the significance level of the test.
For more information, go to Power for equivalence tests.
Interpretation
If you enter a sample size and a difference (or ratio), then Minitab calculates the power of the test. A power value of at least 0.9 is usually considered adequate. A power of 0.9 indicates that you have a 90% chance of demonstrating equivalence when the difference (or ratio) between the population means is actually within the equivalence limits. If an equivalence test has low power, you might fail to demonstrate equivalence even when the test mean and the reference mean are equivalent.
Usually, when the sample size is smaller or when the difference (or ratio) is closer to an equivalence limit, the test has less power to claim equivalence.
Method
| Power for difference: | Test mean - reference mean |
|---|---|
| Null hypothesis: | Difference ≤ -0.425 or Difference ≥ 0.425 |
| Alternative hypothesis: | -0.425 < Difference < 0.425 |
| α level: | 0.05 |
Results
| Difference | Sample Size | Power |
|---|---|---|
| 0.3 | 8 | 0.984946 |
| 0.3 | 12 | 0.999089 |
| 0.4 | 8 | 0.189487 |
| 0.4 | 12 | 0.244815 |
In these results, for a difference of 0.3, the power of the test is approximately 0.98 with a sample size of 8, and 0.99 with a sample size of 15. For a difference of 0.4, the power is approximately 0.19 with a sample size of 8, and 0.24 with a sample size of 12.
Power curve
The power curve plots the power of the test versus the difference between the test mean and the reference mean.
Interpretation
Use the power curve to assess the appropriate sample size or power for your test.
The power curve represents every combination of power and difference (or ratio) for each sample size when the significance level and the standard deviation (or coefficient of variation) are held constant. Each symbol on the power curve represents a calculated value based on the values that you enter. For example, if you enter a sample size and a power value, Minitab calculates the corresponding difference (or ratio) and displays the calculated value on the graph.
Examine the values on the curve to determine the difference (or ratio) between the test mean and the reference mean that can be accommodated at a certain power value and sample size. A power value of 0.9 is usually considered adequate. However, some practitioners consider a power value of 0.8 to be adequate. If an equivalence test has low power, you might fail to demonstrate equivalence even when the population means are equivalent. If you increase the sample size, the power of the test also increases. You want enough observations in your sample to achieve adequate power. But you don't want a sample size so large that you waste time and money on unnecessary sampling or detect unimportant differences to be statistically significant. Usually, differences (or ratios) that are closer to the equivalence limits require more power to demonstrate equivalence.
In this graph, the power curves for all sample sizes show that the test has adequate power (≥ 0.9) for a difference as large as approximately ±0.25. The sample sizes refer to the number of participants in each sequence. The power curve for a sample size of 4 shows that the test has a power of 0.9 for a difference of approximately ±0.28. The power curve for a sample size of 12 shows that the test has a power of 0.9 for a difference of approximately ±0.33. Notice that increasing the sample size to 12 only very slightly increases the difference that can be accommodated at a power of 0.9. For each curve, as the difference approaches the lower equivalence limit or the upper equivalence limit, the power of the test decreases and approaches α (alpha, which is the risk of claiming equivalence when it is not true).
