2-sample hypothesis test

Use 2-sample hypothesis tests to compare two samples with each other, for example, a 2-sample t-test.

In This Topic

2 proportions
2-sample t
Mann-Whitney
Paired t

2 proportions

Use a 2 proportions test to analyze observed differences in the process proportion (defective) at two settings of an input. Use this test when the data from the process are discrete and have exactly two levels, for example, pass or fail, and the factor being evaluated has exactly two levels, for example, fast or slow, before or after. You must collect a sample at each level of the input variable.

Answers the questions:

If I change an input from one level to another, does the process proportion defective stay the same or does it change?
Is the process proportion defective the same before and after a change has been made to the process?

When to Use	Purpose
Mid-project	Fixing an input at two different settings (levels) helps to determine which inputs have significant influence on the proportion defective of the output.
End of project	Verify a significant reduction in the process proportion defective results from the implemented improvements. Of course, this step assumes one of the goals of the project was to reduce the proportion defective.

Data

Your data must be discrete Y values at two levels (for example, good or bad) and a single X at two levels.

Guidelines

Develop a sound data collection strategy to ensure that your conclusions are based on truly representative data.
Use Minitab’s Power and Sample Size command to determine the sample size necessary to detect the smallest difference of interest with sufficient power.
The data must fit a binomial distribution:
- Each test result has exactly two possible outcomes.
- The probability of a particular outcome is constant.
- The trials are independent of each other.
Use this test to generate a confidence interval for the differences in the proportions.

How-to

Verify the measurement systems for the Y data and the input X are adequate.
Develop a data collection strategy (who should collect the data, as well as where and when; how many data values are needed; the preciseness of the data; how to record the data, and so on).
Use one of three methods to enter the data in Minitab:
- Enter results (for example, pass or fail) in two columns, one for each factor level.
- Enter results (for example, pass or fail) in one column with the factor level in a second column
- Use summarized data (the most common approach) by entering the number of trials and number of events (for example, pass) for both factor levels directly in the Minitab dialog box.
Determine the hypotheses. The alternative hypothesis (Ha) is what you are trying to prove with the data. The alternative hypothesis for a 2-proportions test is whether the two proportions are not equal or if one is greater or less than the other. The null hypothesis (Ho) is the opposite of the alternative hypothesis.

For more information, go to Insert an analysis capture tool.

2-sample t

Use a 2-sample t-test to analyze the difference between the observed process mean at two settings of an input. To use a 2-sample t-test, you must collect a sample of data at both levels of the input variable.

Answers the questions:

If I change an input from one level to another level, does the process mean stay the same or does it change?
Is the process mean the same before and after a change has been made to the process?

When to Use	Purpose
Mid-project	Fixing an input at two different settings (levels) helps determine which inputs have significant influence on the mean of the output.
End of project	Verify a significant difference exists between the means of the pre-project process and the post-project improved process. Of course, this assumes that one of the goals of the project was to shift the location of the process (change the process mean).

Data

Your data must be values for continuous Y (output) and a single X (input) at two levels.

Guidelines

Develop a sound data collection strategy to ensure that your conclusions are based on truly representative data.
Use Minitab’s Power and Sample Size command to determine the sample size necessary to detect the smallest difference of interest with sufficient power.
The data must be continuous and reasonably normal, and the samples must be independent.
A 2-sample t-test is very robust to violations of the normality assumption, especially if the sample sizes are large (n > 25).
You should always graph your data when you use a statistical test. For a 2-sample t-test, use histograms or normal probability plots to evaluate reasonable normality and to check for outliers.
By default, a 2-sample t-test does not assume equal variances for the two samples; however, you can assume equal variances for a slightly more powerful test. Use a 2 variances test to determine if the variances are equal. The test is also quite robust to unequal variances if the sample sizes are approximately equal.
If you have discrete numeric data from which you can obtain every equally spaced value and you have measured at least 10 possible values, you can evaluate these data as if they are continuous.

How-to

Verify the measurement systems for the Y data and the input X are adequate.
Develop a data collection strategy (who should collect the data, as well as where and when; how many data values are needed; the data’s precision; how to record the data, and so on).
You can use one of three methods to enter data in Minitab:
- Enter data in two columns, one for each factor level.
- Enter Y data in one column with the factor level (X) in a second column.
- Use summarized data by entering the sample size, mean, and standard deviation for both factor levels directly into the dialog box.
Determine the hypotheses. The alternative hypothesis (Ha) is what you are trying to prove with the data. For a 2-sample t-test, the alternative hypothesis states either the two means are not equal or one mean is greater than the other mean. The null hypothesis (Ho) is the opposite of the alternative hypothesis. For a 2-sample t-test, the null hypothesis states the two means are equal. First, develop a sound data collection strategy to ensure that your conclusions are based on truly representative data.

For more information, go to Insert an analysis capture tool.

Mann-Whitney

Use the Mann-Whitney test to analyze the observed differences in the process median between two input settings. This test is similar to a 2-sample t-test, and is an alternative for cases where the data from the two samples are not reasonably normal.

Answers the questions:

If I change an input from one level to another, does the median of the process stay the same or does it change?
Is the median of the process the same before and after a change has been made to the process?

When to Use	Purpose
Mid-project	Test which inputs have significant influence on the output by fixing an input at two different settings (levels).
End of project	Verify a significant difference exists between the medians of the pre-project process and the post-project, improved process. Of course, this step assumes one of the goals of the project was to shift the process median.

Data

Your data must be a continuous value for Y (output), and a single X (input) at two levels.

Guidelines

Develop a sound data collection strategy to ensure that your conclusions are based on truly representative data.
The Mann-Whitney test is from a category called nonparametric statistics. It is meant to be an alternative to the parametric test (2-sample t-test) for cases where the normality assumption fails badly. Such cases do not occur frequently because parametric tests are robust; therefore, the parametric tests are recommended for the majority of cases.
While the Mann-Whitney test does not assume normality, it does assume equal variances. In the majority of cases, the 2-sample t-test is more powerful.
For these reasons, it is usually recommended that you use the t-test whenever the data are reasonably normal (or the sample sizes are large), and use the Mann-Whitney test as a last resort.
You should always graph your data whenever you use a statistical test. For the Mann-Whitney test, histograms or dotplots of the two samples, or a side-by-side boxplot, show the relative locations of the samples. The histograms and boxplots also identify outliers.
If you have discrete numeric data from which you can obtain every equally spaced value and you have measured at least 10 possible values, you can evaluate these data as if they are continuous.

How-to

Verify the measurement system for the Y data is adequate
Develop a data collection strategy (who should collect the data, as well as where and when; how many data values are needed; the preciseness of the data; how to record the data, and so on).
Enter Y data in one column and the factor levels (X) in a second column.
Determine your hypothesis. The alternative hypothesis (H_a) is what you are trying to prove with the data. The alternative hypothesis for a Mann-Whitney test can be whether the first process median is greater than, less than, or not equal to the second process median. The null hypothesis (H₀) is the opposite of the alternative hypothesis.

For more information, go to Insert an analysis capture tool.

Paired t

Use a paired t-test to analyze observed differences in sample units that are subjected to two different inputs. To use a paired t-test, you must subject the exact same sample units to both levels of the input variable to remove potential effects due to differences in the sample units themselves, which might mask the effect due to the change in the input. For example, you want to test whether two types of tires result in differences in gas mileage. The variation in gas mileage due to different cars in the sample would be much greater than variation due to different tires. A paired t-test accounts for the differences in cars.

Answers the questions:

If I change an input from one level to another, does the process mean stay the same or does it change?
Is the change in the process mean due to changing the input independent of the items tested?

When to Use	Purpose
Mid-project	Fixing an input at two different settings (levels) helps to determine which inputs have significant influence on mean of the output.
Mid-project	Verify changes to inputs result in significant differences from the pre-project mean, provided you can test on the same units as those in the pre-project sample.

Data

Your data must be values for continuous Y (output) and one X (input) at two levels.

Guidelines

Develop a sound data collection strategy so that you ensure that your conclusions are based on truly representative data.
The differences (calculated for each sample run at the two levels of X) must be continuous and reasonably normal.
A paired t-test is very robust to the normality assumption, especially if the sample sizes are large (approximately larger than 25).
You should always graph your data whenever you use a statistical test. For a paired t-test, use a histogram or normal probability plot of the differences to evaluate reasonable normality and to check for outliers.
If you have discrete numeric data from which you can obtain every equally spaced value and you have measured at least 10 possible values, you can evaluate these data as if they are continuous.

How-to

Verify the measurement systems for the Y data and the input X are adequate.
Develop a data collection strategy (who should collect the data, as well as where and when; how many data values are needed; the preciseness of the data; how to record the data, and so on).
You can use one of two methods to enter the data in Minitab:
- Enter data in two columns, one for each factor level – the data from each unit must be in the same row of the two columns.
- Use summarized data by entering the sample size, mean, and standard deviation of the differences observed in each sample unit (Y at level 1 minus Y at level 2) directly into the dialog box.
Determine your hypotheses. The alternative hypothesis (Ha) is what you are trying to prove with the data; for a paired t-test, it is whether the mean of the paired differences (Y at level 1 minus Y at level 2) is not equal to, greater than, or less than zero. The null hypothesis (Ho) is the opposite of the alternative hypothesis.

For more information, go to Insert an analysis capture tool.