Hierarchical Data Analysis for Steel Charpy Testing

Start with your own data

Scenario Configurator — describe your data structure

Name the units and variables exactly as they appear in your project. The guide will translate your structure into an analysis path, warn you about invalid replication, and recommend a primary model plus sensitivity checks.

Quick-pick — which strategy fits your situation?

Orientation

Learning objectives

This lesson takes you from a raw specimen table to a defensible engineering conclusion. It is designed for analysts who must explain not only what the model says, but why the evidence deserves trust.

By the end, you will be able to:

• Identify the independent experimental unit and the measurement unit.
• Classify variables as heat-, pipe-, test-set-, or specimen-level.
• Explain pseudoreplication using ICC, design effect, and effective sample size.
• Select aggregation, cluster-robust, GEE, mixed-model, Bayesian, or bootstrap methods.
• Validate a three-level model and communicate limitations responsibly.
• Implement a defensible workflow in Excel, Minitab, Python, or R.

Case question

A steel mill wants to know whether heat-level calcium and sulfur are associated with transverse base-metal Charpy energy after accounting for product dimensions and rolling conditions. Each heat supplies several pipes, and each pipe supplies three specimens. The temptation is to run ordinary regression on every specimen row. This lesson shows why that shortcut can be badly misleading—and what to do instead.

Foundation 1

Why ordinary regression fails

If calcium and sulfur are measured once per heat, repeating those values on every pipe or specimen does not create more independent chemistry observations.

The fastest way to understand the problem

Key terminology

Core formulas

Foundation 2

The eight-gate analysis workflow

A mixed model is not the first step. The first step is proving that the data structure, estimand, and model are aligned.

Decision point

Select the method from the question and data

What each method estimates

Complete case study

From 486 specimen rows to one defensible conclusion

A fictional X65 dataset contains 36 independent heats, 2–7 pipes per heat, and three full-size transverse base-metal Charpy specimens per pipe tested at −5°C. The imbalance is intentional: real mill datasets are rarely perfectly balanced.

Step 1: define the question and estimand

This is an observational between-heat association. It is not a causal claim and it is not a probability-of-passing model. The response is average Charpy energy in joules—not the minimum specimen result.

Step 2: classify each variable

1. Heat: Ca, S, chemistry, caster, melting practice.
2. Pipe: OD, gauge, rolling practice, exit and coiling temperatures.
3. Test set: test temperature, orientation, specimen size, location.
4. Specimen: specimen number and energy.

Step 3: validate the data structure

Step 4: quantify clustering without overstating it

Assume the fitted three-level variance model estimates heat ICC = 0.38. The average cluster contains 13.5 specimens because 486 ÷ 36 = 13.5.

This approximation shows how correlation erodes the apparent information in 486 rows. It does not turn 84.5 into the sample size for Ca. Because Ca changes only between heats, the Ca association is supported by 36 independent Ca-bearing units.

Step 5: build the transparent heat-level sensitivity model

Aggregate to one row per heat and fit:

There are 36 rows and four estimated parameters, including the intercept, so the residual degrees of freedom are:

Step 6: compare complementary analyses

Step 7: make the engineering interpretation

Before using the statement operationally, confirm that grade, product dimensions, rolling practice, test temperature, slab thickness, specimen size, orientation, and production period either overlap sufficiently, are controlled appropriately, or define a deliberately restricted population.

Interactive laboratory

See why more rows are not always more evidence

Change the design. The lab calculates raw observations, design effect, and approximate effective sample size. It also explains why adding independent heats is usually more valuable for a heat-level chemistry effect.

Compare two ways to spend the next 36 tests

Variable-level challenge

Select the level where each variable normally changes. Your goal is to recognize when a column repeated down a spreadsheet is still a higher-level variable.

Evidence quality

Model validation and sensitivity checks

A coefficient table is the beginning of validation, not the end. Each diagnostic below tests a different way the conclusion could fail.

Diagnostic action matrix

Minimum sensitivity set

• Primary analysis fitted exactly as planned.
• One-row-per-heat model compared with the hierarchical result.
• Every heat removed once; effect contrast and interval retained.
• Exclusion and missing-data rules varied within defensible limits.
• Raw and normalized Charpy definitions compared only where scientifically appropriate.
• Product, grade, practice, and time overlap visualized.
• Prediction assessed with entire heats held out—not random specimen rows.

Execution

Statistics Implementation

Software does not choose the experimental unit for you. Complete the hierarchy map and estimand first; then use the playbook that matches your environment.

Minitab Navigation

Exam Tips

import pandas as pd
import statsmodels.formula.api as smf

df = pd.read_csv("hierarchical_charpy_practice.csv")
required = {"heat_id","pipe_id","charpy_j","ca_ppm","s_ppm","exit_temp_c"}
assert required.issubset(df.columns)

# Verify heat-level predictors are constant within heat.
assert df.groupby("heat_id")["ca_ppm"].nunique().max() == 1
assert df.groupby("heat_id")["s_ppm"].nunique().max() == 1

# Heat-level sensitivity model: one independent row per heat.
heat = df.groupby("heat_id", as_index=False).agg(
    charpy_mean_j=("charpy_j", "mean"),
    ca_ppm=("ca_ppm", "first"), s_ppm=("s_ppm", "first"),
    pipes=("pipe_id", "nunique"))
heat_fit = smf.ols("charpy_mean_j ~ ca_ppm + s_ppm", data=heat).fit()

# Marginal sensitivity: cluster-robust covariance by heat.
robust = smf.ols(
    "charpy_j ~ ca_ppm + s_ppm + exit_temp_c", data=df
).fit(cov_type="cluster", cov_kwds={
    "groups": df["heat_id"], "use_correction": True
})

# Random heat intercept plus pipe-within-heat variance component.
# pipe_id must be globally unique, as it is in the practice dataset.
mixed = smf.mixedlm(
    "charpy_j ~ ca_ppm + s_ppm + exit_temp_c", df,
    groups="heat_id", re_formula="1",
    vc_formula={"pipe": "0 + C(pipe_id)"}
).fit(reml=True)

print(f"rows={len(df)}, heats={df.heat_id.nunique()}, "
      f"pipes={df.pipe_id.nunique()}")
print(heat_fit.summary())
print(robust.summary())
print(mixed.summary())

library(readr)
library(dplyr)
library(lme4)
library(lmerTest)

d <- read_csv("hierarchical_charpy_practice.csv")
stopifnot(max(tapply(d$ca_ppm, d$heat_id, function(x) n_distinct(x))) == 1)

m <- lmer(charpy_j ~ ca_ppm + s_ppm + exit_temp_c +
          (1 | heat_id/pipe_id), data = d, REML = TRUE)
summary(m)
isSingular(m)
plot(m)
qqnorm(resid(m)); qqline(resid(m))

# Leave one entire heat out at a time.
loo <- lapply(unique(d$heat_id), function(h) {
  fit <- lm(charpy_j ~ ca_ppm + s_ppm + exit_temp_c,
            data = filter(d, heat_id != h))
  c(omitted_heat = h, ca = coef(fit)["ca_ppm"])
})
do.call(rbind, loo)

# Each sampled heat occurrence receives a NEW bootstrap cluster ID.
import numpy as np

rng = np.random.default_rng(42)
heat_ids = df["heat_id"].unique()
boot_coefs = []

for b in range(2000):
    sampled = rng.choice(heat_ids, len(heat_ids), replace=True)
    pieces = []
    for instance, heat_id in enumerate(sampled):
        part = df.loc[df.heat_id.eq(heat_id)].copy()
        part["boot_heat_id"] = f"b{instance}"  # preserves multiplicity
        pieces.append(part)
    boot = pd.concat(pieces, ignore_index=True)
    bh = boot.groupby("boot_heat_id", as_index=False).agg(
        charpy_mean_j=("charpy_j", "mean"),
        ca_ppm=("ca_ppm", "first"), s_ppm=("s_ppm", "first"))
    if bh[["ca_ppm", "s_ppm"]].drop_duplicates().shape[0] > 3:
        fit = smf.ols("charpy_mean_j ~ ca_ppm + s_ppm", bh).fit()
        boot_coefs.append(fit.params["ca_ppm"])

print(np.quantile(boot_coefs, [0.025, 0.5, 0.975]))

Decision language

When you see this, it means this

Failure prevention

Common mistakes, warning signs, and corrections

Apply the workflow

Engineering review challenge

Take it to work

Field checklist for hierarchical engineering data

• Match the analysis unit to the level at which the predictor varies.
• Heat-level chemistry needs independent heats.
• Pipe and specimen replication estimate lower-level variation but do not replace heats.
• Use descriptive, marginal, and hierarchical methods according to the goal.
• Check convergence, influence, overlap, missingness, and engineering definitions.
• Cluster bootstrap resamples whole heats and needs a reasonable number of them.
• Report effect size, uncertainty, practical meaning, range, and causal limits.

One-page decision rule

Pre-release analysis checklist

• The engineering response and decision threshold are defined.
• Heat, pipe, test-set, specimen, and crossed factors are mapped.
• Distinct independent units—not rows—are counted for every key predictor.
• Units and chemistry conversions are explicit.
• Grade, practice, dimensions, temperature, specimen size, and time overlap are reviewed.
• The primary estimand and model are named before interpretation.
• Convergence, singularity, residuals, influence, and grouped validation are checked.
• Effect size and uncertainty are translated into a meaningful engineering contrast.
• Extrapolation and causal limitations are stated.
• The analysis can be reproduced from the dataset, code, and decision log.

Suggested next lesson: Building and validating mixed-effects models in Minitab, Python, and R.

References and further study

• Pinheiro, J. C. & Bates, D. M. Mixed-Effects Models in S and S-PLUS. Springer.
• Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press.
• West, B. T., Welch, K. B. & Galecki, A. T. Linear Mixed Models: A Practical Guide Using Statistical Software. CRC Press.
• Official documentation for Minitab mixed-effects models, Python statsmodels MixedLM, and R lme4.