Resampling in Minitab

After a row is drawn, it goes back into the pool before the next draw. A row can appear zero, one, or several times. The resample stays the same size as the original sample.

No. It reveals what the observed data imply under a specific resampling rule. A biased, dependent, or unrepresentative sample remains problematic.

Not always. It depends on what confidence interval you are calculating, the sample size, and the method used.

The key distinction is that a standard confidence interval for the mean does not necessarily require the raw data to be normally distributed. What matters is whether the sampling distribution of the mean is approximately normal.

For example, individual Charpy values could be right-skewed:

80, 95, 110, 115, 125, 140, 160, 220, 310, …

If you repeatedly took samples of 50 specimens and calculated the mean of each sample, those means would tend to form a more nearly normal distribution because of the Central Limit Theorem.

Suppose two rolling practices have an observed mean Charpy difference of about 31 J.

Randomization or permutation test: start with the null hypothesis that the two groups are equivalent. Combine the observations, shuffle the A/B labels, split them back into groups of the original sizes, calculate the mean difference, and repeat thousands of times. This creates a null distribution.

Bootstrapping: preserve the groups and repeatedly sample with replacement from each group. Calculate the statistic each time. This creates a bootstrap distribution that describes the precision and uncertainty of the estimate.

Suppose the observed data are:

The observed difference in means is 50 J. The randomization test asks: if A and B really had no effect, could a 50 J difference arise just by randomly assigning these observations to two groups?

First remove the labels:

200, 210, 220, 150, 160, 170

Then randomly assign three observations to A and three to B. Every original observation appears exactly once in that randomization.

Valid: A = [200, 160, 170]    B = [210, 220, 150]

Not valid: A = [200, 200, 170]

The second assignment is invalid because the original dataset contained only one 200 J observation. Without replacement means that once an observation is assigned during one randomization, it cannot be selected again in that same randomization. Bootstrap sampling is different: repeated values are allowed.

Begin with the observed difference:

D_observed = 31 J

Assume the null hypothesis is true:

H₀: μ_A = μ_B

Shuffle the group labels thousands of times. After each shuffle, calculate:

D* = x̄_A* − x̄_B*

For a two-sided test, count randomized differences at least as extreme as the observed result in either direction:

|D*| ≥ |D_observed|

Suppose 10,000 randomizations produce:

Extreme results = 145 + 135 = 280

p = 280 / 10,000 = 0.028

If A and B truly had no difference, about 2.8% of random label assignments produced a difference at least as extreme as the 31 J result observed.

A one-sample randomization test follows the same broad idea, but there are no A/B group labels to shuffle. Suppose:

H₀: μ = 180 J

Sample: 185, 190, 175, 200, 195

The sample mean is 189 J, so the observed difference from the null value is 9 J. One possible one-sample method is a sign-randomization test. First calculate the deviations from 180:

+5, +10, −5, +20, +15

Keep the magnitudes but randomly flip their signs. For example:

+5, −10, +5, −20, +15

This randomized set has a mean deviation of −1 J, corresponding to a randomized sample mean of 179 J. Repeating the sign randomization creates a null distribution centered around the hypothesized value.

Yes. That objection is correct.

If we simply reorder:

185, 190, 175, 200, 195

into:

200, 175, 195, 185, 190

the mean does not change. Reordering one sample cannot create a useful null distribution. A one-sample randomization procedure must randomize something other than the observation order.

In a sign-randomization approach, the signs attached to the deviation magnitudes change. For example, the magnitudes 5, 10, 5, 20, and 15 could become:

+5, −10, +5, −20, +15

−5, +10, −5, +20, +15

Those different sign patterns produce different means.

No. Minitab uses a different approach.

Sample: 185, 190, 175, 200, 195    x̄ = 189    H₀: μ = 180

Minitab first shifts every observation by the same amount so the adjusted sample mean equals the hypothesized mean. The required shift is:

180 − 189 = −9

The null-adjusted data become:

176, 181, 166, 191, 186    adjusted mean = 180

Minitab then repeatedly resamples with replacement from this null-adjusted sample, keeping the original sample size. Each resample produces a new mean, building a null distribution centered around 180.

176, 176, 191, 181, 191 → mean = 183

166, 181, 166, 186, 176 → mean = 175

In short, Minitab uses bootstrap-style resampling after first forcing the data to represent the null hypothesis.

Minitab’s 2-sample randomization test follows the classic permutation logic. Suppose:

x̄_A = 209    x̄_B = 178.4    D_observed = 30.6 J

H₀: μ_A − μ_B = 0

Minitab pools all observations, randomly reallocates them into two groups while preserving the original group sizes, and calculates a new difference in means.

D* = x̄_A* − x̄_B*

Repeating this process thousands of times creates a distribution centered near zero, representing differences expected under the null hypothesis. For a two-sided test, Minitab counts the proportion satisfying:

|D*| ≥ 30.6

That proportion is the randomization p-value.

Yes. In ordinary nonparametric bootstrapping, if the original sample size is n = 6, every bootstrap resample also contains six observations. Those observations are selected with replacement.

Original sample: [100, 110, 120, 130, 140, 150]

Every resample contains exactly six selections, but some original observations may appear several times while others may not appear at all. Do not confuse these numbers:

n = 6 → observations inside each bootstrap resample

B = 1,000 → number of separate bootstrap resamples

You create 1,000 separate samples of six observations each and usually calculate one statistic from each sample. Keeping the resample size at six approximates how the statistic would vary across repeated studies having the same sample size as the original study.

The practice set was organized around five Minitab resampling methods:

Use 1,000 or 5,000 resamples to start. Higher numbers of resamples give smoother resampling distributions.

These methods in Minitab fall under resampling statistics. They are powerful because they do not rely as heavily on normal-distribution assumptions as many traditional t-tests and z-tests. They use repeated random sampling or simulation to estimate uncertainty, significance, or confidence intervals.

1. Bootstrapping for 1-Sample Function

Bootstrapping repeatedly:

1. Randomly samples your data.
2. Samples with replacement.
3. Calculates a statistic each time.
4. Builds a distribution of that statistic.

“Function” simply means the statistic you care about: mean, median, standard deviation, percentile, capability metric, and so on.

It is used when you want confidence intervals, robust estimates, small-sample analysis, or non-normal-data analysis.

For example, instead of assuming normality for 25 Charpy tests, bootstrap resampling takes repeated samples from those 25 observations, estimates the mean each time, and creates an empirical confidence interval.

2. Bootstrapping for 2-Sample Means

This uses the same concept but compares the mean of Group A with the mean of Group B. Both groups are repeatedly resampled.

It is used to estimate the difference between process means, the confidence interval for that difference, and the stability of the difference.

3. Randomization Test for 1-Sample Mean

This is a simulation-based hypothesis test. It asks whether a sample mean is significantly different from a target without relying on a theoretical t-distribution in the same way as the classical test.

4. Randomization Test for 1-Sample Proportion

This tests whether a proportion differs from a target proportion. A proportion can represent pass/fail, good/bad, defect/no defect, yield, scrap rate, or acceptance rate.

5. Randomization Test for 2-Sample Means

This tests whether two groups truly differ. It repeatedly shuffles group labels, recalculates the difference, and builds a null distribution.

Key Difference: Bootstrap vs. Randomization

Bootstrapping answers this question:

But instead of collecting new real-world samples, we repeatedly resample from the data we already have.

The Big Concept

Suppose you collected only five yield-strength observations:

Bootstrapping says:

What “With Replacement” Means

This is the most important concept. From the original data:

a bootstrap sample might be:

Another might be:

Some values repeat and some disappear because sampling is with replacement. After selecting a value, it goes back into the pool.

Why This Works

Your original sample is treated as the best estimate of the real population. Bootstrapping simulates thousands of alternate realities based on your observed process variation.

What Is a “Function”?

What Bootstrapping Produces

Usually, a confidence interval. For example:

Meaning:

Practical Steel Example

Suppose coiling-temperature measurements are 610, 618, 605, 622, and 611. The question is:

Bootstrap:

1. Resamples the data thousands of times.
2. Computes the mean each time.
3. Builds a distribution of means.
4. Calculates a confidence interval.

Step-by-Step in Minitab

Instead of estimating one mean, we estimate the difference between two means.

Bootstrap repeatedly:

1. Resamples Group A.
2. Resamples Group B.
3. Calculates both means.
4. Computes their difference.
5. Repeats thousands of times.

This builds a distribution of mean differences.

Example

But is that real, or random variation? Bootstrap helps estimate uncertainty around that difference.

What Bootstrap Does Internally

Standard group: original 51, 52, 50, 53, 49 → possible bootstrap sample 51, 51, 53, 49, 50 → mean = 51.0.

New group: original 54, 55, 53, 52, 56 → possible bootstrap sample 55, 55, 53, 52, 54 → mean = 53.8.

Repeat thousands of times. Values such as 2.8, 2.1, 3.4, 1.9, and 2.6 form the bootstrap distribution of mean differences.

Key Interpretation Rule

Why zero? Because:

means there is no difference between the group means.

Step-by-Step in Minitab

Now we move from estimating uncertainty with bootstrap to testing significance with randomization testing. This is a major conceptual shift.

This is fundamentally a hypothesis test, but performed using simulation instead of relying only on theoretical distributions.

Engineering Example

Suppose target yield strength = 51 MPa and the sample is 53, 52, 54, 51, 55. The observed mean is 53 MPa.

Now the question becomes:

Core Logic

The randomization distribution represents what random variation alone would produce if the null hypothesis were true.

The p-value

The p-value answers how unusual the observed result is under H₀. A small p-value means the observed result is rare in the simulated null world; a large p-value means random variation could readily explain it.

Step-by-Step in Minitab

The key is this:

Suppose the target mean is:

and the observed sample mean is:

Now the question becomes:

Why We Cannot Just Compare 53 vs. 51 Directly

Because samples naturally vary. Even if the true process mean is 51, one sample may average 49, another 52, another 50.5, and another 53 because of random process and sampling variation.

So we need a way to determine how much variation is “normal.”

The Dartboard Analogy

Imagine a dartboard whose target center is 51. Darts naturally scatter around the center. If one dart lands at 53, is that normal scatter or evidence that the aim has shifted?

You need to know how wide the natural scatter normally is. That is what resampling estimates.

What Randomization Builds

It builds the distribution of expected random variation under the null hypothesis — the null distribution.

Suppose Minitab simulates 5,000 samples under:

and generates means such as 50.8, 51.2, 50.5, 51.7, 49.9. Most are near 51 because under H₀, 51 is assumed true.

Then compare the real mean, 53, and ask:

Randomization testing is basically:

We are resampling the original dataset, but in a way that simulates what random samples would look like under the null hypothesis. That subtle distinction is the whole trick.

Bootstrap

So bootstrap directly resamples the original data with replacement.

Randomization Test

Before resampling, the data is shifted/recentered to align with the target mean. Then resampling/simulation occurs.

For example, suppose:

The data are centered at 53, but under H₀ they should be centered at 51. So Minitab first transforms the data.

Then it repeatedly samples from the recentered data, computes means, and builds the null distribution.

The recentering step exists because:

Without recentering, the simulation would still reflect your observed sample mean instead of the hypothesized mean. That would break the logic of the test.

The Core Goal

Suppose:

If we directly resample the original data, simulated samples will naturally center around 53, not 51. The simulation would represent a world where the true mean is 53, which contradicts the null hypothesis.

Then the test would effectively ask:

which is meaningless. Of course 53 is common if reality is centered at 53.

The Recentering Equation

where:

• x_i = original observation
• x̄ = observed sample mean
• μ₀ = hypothesized mean
• x_i^* = recentered observation

What Recentering Preserves

Recentering changes the center, but preserves the spread, variability, shape, skewness, outliers, and process behavior.

Visually, it simply slides the whole distribution from a center of 53 to a center of 51 while leaving the shape the same.

This is a significance test for the difference between two means.

Engineering Example

The question is:

The Null Hypothesis

Under H₀, group labels should not matter. That is the entire philosophy of the test.

Why This Differs From 1-Sample Randomization

A 1-sample randomization test needed recentering because there was a target mean. A 2-sample randomization test usually does not need that same recentering step. Under H₀, both groups are treated as coming from the same population, so instead we shuffle the group labels.

How the Randomization Works

Original data:

The null distribution represents:

Then compare the observed difference, +3 MPa, and ask:

Why Label Shuffling Makes Sense

If H₀ is true, group membership is meaningless. Therefore, randomly reassigning labels should produce equally plausible datasets.

If randomized differences resemble the observed difference, the observed difference is not unusual. If the observed difference is much larger than what randomization usually produces, that is evidence the groups differ.

Step-by-Step in Minitab

Now we move from means to proportions. This is extremely important in quality engineering because many industrial problems are naturally pass/fail, defect/no defect, cracked/not cracked, downgraded/not downgraded, or accepted/rejected.

Quality Example

Suppose historical pass rate = 90%, so the target proportion is:

Now suppose 100 pipes are tested and only 83 pass:

The question is:

Hypotheses

For a two-sided question:

For a deterioration question:

The Key Concept

Under H₀, the process still has a 90% pass probability. The randomization procedure simulates what pass/fail outcomes would naturally look like if that were true.

With proportions, the outcomes are binary. If:

then each simulated observation has a 90% probability of Pass and a 10% probability of Fail.

For 100 pipes, one simulation might produce 91 Pass / 9 Fail; another 88 / 12; another 93 / 7. These results build the null distribution of pass rates.

Then compare the actual 83% and ask:

Step-by-Step in Minitab