Data AnalyticsIntermediate45 minutesInteractive

Dealing with Outliers: A Steel & Mining Detective's Guide

Thirty years on shop floors and mine sites taught me one thing: every dataset has a troublemaker. Your job isn't to exorcise it — it's to interrogate it. We'll use blast-furnace temperatures, tensile tests, ore grades, and a runaway conveyor to learn four detection methods and, more importantly, what to do once you've caught your suspect.

Why this matters: delete a genuine high-grade ore pocket and you throw away the best day the pit ever had. Keep a mis-keyed tensile reading and you ship steel on a lie. Outlier handling is a judgment skill, not a delete key.

Learning objectives

By the end of this lesson, you will be able to:

Define an outlier and distinguish measurement error, data error, and genuine extreme values.
Calculate the Z-score, Modified Z-score (MAD-based), IQR/Tukey fences, and Grubbs' test statistic by hand and in Excel.
Select the right detection method based on sample size, distribution shape, and data sensitivity.
Investigate a flagged point using a disciplined root-cause procedure before deciding what to do with it.
Explain masking and swamping, and why deleting outliers without investigation is a data-integrity risk.
Execute the full workflow in both Excel and Minitab, and connect it to ASQ CSSBB/CQE exam expectations.

Prerequisites

You should be comfortable with the mean, median, and standard deviation, and have seen a boxplot before. If Chebyshev's inequality and the Empirical Rule are new to you, our Chebyshev vs. the Empirical Rule lesson is a good warm-up — outlier fences are built directly on those spread ideas.

What exactly is an outlier?

Here's the definition I've used since my first plant assignment: an outlier is an observation that is inconsistent with the bulk of the data, given some assumed model of how the data should behave. That's it. Statistics gives you tools to flag candidates — it does not tell you why a point is weird. That part is still your job, and it's the part most courses skip.

In thirty years of chasing weird numbers through blast furnaces, rolling mills, and open pits, every flagged point eventually sorted itself into one of four buckets:

🚫 The value itself is wrong

Fix the pipeline, not just the number.

Measurement / instrument error

The sensor lied — drift, no tare, a stuck gauge.

Data / transcription error

A human or a paste error — 950 typed for 590.

✅ The value is correct

Respect it — investigate the process behind it.

Genuine process shift

A real, assignable cause — material swap, tool change.

Genuine rare event

The process working exactly as it does — a rich ore pocket, a true tail value.

Every flagged point sorts into one of these two houses before you decide what to do with it.

🔧 Measurement / instrument error

A thermocouple drifts, a scale isn't tared, a gauge sticks. The process was fine; the sensor lied.

⌨️ Data / transcription error

Someone typed 950 instead of 590, pasted a column one row off, or a unit got mixed up (mm vs. cm).

⚙️ Genuine process shift (special cause)

A real, assignable cause changed the process — a raw-material swap, a tool change, an operator error. Real, and worth fixing at the source, not the spreadsheet.

💎 Genuine rare event (natural extreme)

The process is behaving exactly as it always does, and this is simply a rare tail value — or, on a good day, a genuinely rich ore pocket. Delete this and you're editing reality.

The trap: statistical tests can tell you a point is far from the rest. They cannot tell you which of the four buckets it belongs in. That takes investigation — walk the floor, pull the calibration log, check the sample log book.

Key terminology

Z-score

Number of standard deviations a point sits from the mean. Sensitive to the very outliers it's trying to detect.

MAD

Median Absolute Deviation — the median of the absolute deviations from the median. A robust stand-in for standard deviation.

Modified Z-score

A Z-score built from the median and MAD instead of the mean and SD, so it resists being fooled by the outlier itself.

IQR / Tukey fences

Q3 − Q1 defines the middle 50% of the data. Fences at 1.5×IQR (mild) and 3×IQR (extreme) beyond Q1/Q3 flag candidates — distribution-free.

Grubbs' test

A formal hypothesis test (assumes approximate normality) for whether the single most extreme value is a statistical outlier.

Masking & swamping

Masking: a second outlier hides the first by inflating the mean/SD. Swamping: a robust method over-flags normal points near a cluster of real outliers.

Winsorizing

Replacing extreme values with the nearest non-extreme value instead of deleting them — keeps the sample size, tempers the influence.

Trimming

Removing a fixed percentage of the most extreme values from each end before calculating statistics (e.g., a 5% trimmed mean).

Influential point

In regression, a point that changes the fitted line disproportionately — not always the same thing as an outlier in Y alone.

Watch masking happen, with real numbers

Same furnace data from the worked example. Adding a second extreme reading doesn't just fail to get caught — it drags the mean and SD along enough to un-flag the first one too.

This is exactly why the decision framework says: cross-check the plain Z-score against IQR fences or the Modified Z-score before concluding "no outliers here."

The four detection methods

Here are the four tools I keep in my back pocket, from quickest-and-dirtiest to most formal.

1. Z-score

Z = x − x̄s

x̄ = sample mean, s = sample standard deviation. Common flag threshold: |Z| > 3.

2. Modified Z-score (MAD-based)

M = 0.6745 (x − x̃)MAD MAD = median(|xᵢ − x̃|)

x̃ = median. Iglewicz & Hoaglin's recommended flag threshold: |M| > 3.5. The 0.6745 constant rescales MAD to be comparable to a standard deviation under normality.

3. IQR / Tukey fences

Lower = Q1 − 1.5(IQR) | Upper = Q3 + 1.5(IQR)

IQR = Q3 − Q1. Use 3×IQR for "extreme" outliers instead of "mild" ones. This is the rule your boxplot's whiskers are already using.

Schematic view (axis broken for clarity) using the rolling-mill coil-width practice numbers: Q1 = 1249, Q3 = 1252, upper fence = 1256.5 mm. The 1310 mm reading sits far beyond the fence.

4. Grubbs' test (ESD, one-sided or two-sided)

G = max|xᵢ − x̄|s

Reject (flag as outlier) if G exceeds the critical value G_crit, which depends on n and your significance level α (typically 0.05) via the t-distribution. Assumes the data are approximately normal.

Note: Grubbs' test formally checks only the single most extreme point. To check for more than one outlier, use an iterative approach (remove, retest) or the Generalized ESD test — but iterate carefully, since removing points changes your critical value each round.

Which method, when?

Small n, unsure of shape

Use IQR fences. Distribution-free, robust, and it's literally what your boxplot already shows.

Approximately normal, need a formal test

Use Grubbs' test. Gives you a defensible p-value for a report or a capability study.

Suspect more than one outlier

Use the Modified Z-score. Median and MAD aren't dragged around by the very points you're hunting.

Quick screen, large n, normal-ish

Use the plain Z-score. Fast, familiar, but weak when the outlier itself has inflated the mean and SD (masking).

Time-ordered process data

Use a control chart (I-MR or X-bar/R). A point beyond 3σ control limits is a special cause in context, not just a static outlier.

Regression / multiple variables

Check leverage, Cook's distance, and Mahalanobis distance — a point can be unremarkable on each variable alone and still be a multivariate outlier.

My rule of thumb after 30 years: run at least two methods. If the Z-score, Modified Z-score, IQR fences, and Grubbs' test all agree, you have real confidence. If they disagree, that disagreement is itself information — usually about masking.

Step-by-step investigation procedure

Plot it first. Boxplot, individual value plot, or run chart — before any formula. Your eyes catch things statistics miss.
Screen with a distribution-free method. IQR fences work regardless of shape and cost nothing to compute.
Confirm with a formal test if needed. Grubbs' test if approximately normal; otherwise document the IQR/Modified-Z flag.
Investigate the root cause before touching the data. Pull the calibration log, the sample log, the shift log, the material certs. Talk to the operator.
Classify it. Measurement error, data error, genuine process shift, or genuine rare event — pick one, with evidence.
Decide and document. Correct (if the true value is known), exclude with a documented reason, Winsorize/trim for a specific analysis, or keep it as-is. Never delete silently.
Re-run the analysis both ways. Report how the conclusion changes with and without the point — this transparency is what separates rigor from cherry-picking.

The cardinal sin: deleting a point because it "ruins the story" or fails a normality test, with no investigation and no documentation. An auditor — or an ASQ exam question — will find this every time.

Worked example: the furnace mystery

Problem: Sixteen consecutive hot-metal temperature readings (°C) from a blast furnace: 1494, 1502, 1497, 1508, 1499, 1505, 1496, 1501, 1503, 1498, 1500, 1506, 1495, 1504, 1499, 1612. Is 1612°C an outlier, and what should we do about it?

Plot it. Fifteen readings cluster tightly around 1500°C; 1612°C sits well clear of the pack on a boxplot.

Screen with IQR. Q1 = 1497.75, Q3 = 1504.25, IQR = 6.5. Upper fence = 1504.25 + 1.5(6.5) = 1514.0°C. 1612°C is far beyond it.

Confirm with Grubbs. Mean ≈ 1507.44, s ≈ 28.17, G ≈ 3.71. For n = 16 at α = 0.05, G_crit ≈ 2.59. Since G > G_crit, we reject the null — 1612°C is a statistical outlier.

Investigate. The maintenance log shows the thermocouple was replaced two readings later after failing a calibration check — it had drifted high after a furnace reline.

Classify and decide. This is a measurement error. We exclude the reading from the process-capability calculation, log the reason (thermocouple drift, confirmed by maintenance record), and flag the sensor for more frequent calibration.

Payoff: excluding this one documented point drops the calculated standard deviation from ≈28.2°C to ≈4.2°C — the furnace was never actually running that wild. Reporting 28.2°C as "typical" variation would have triggered an unnecessary and expensive process investigation.

Before: including the glitch

SD ≈ 28.2°C — looks like a wild process.

After: glitch excluded

SD ≈ 4.2°C — a tight, well-behaved process.

One documented, investigated exclusion changes the entire capability story — from "alarmingly variable" to "running exactly as designed."

Interactive investigation lab

Pick a case file from the plant or the pit, choose your detection lens, and see who gets flagged. Add a rogue reading of your own and watch masking happen in real time.

Case files

Data (comma-separated) — edit freely

Choose your detection lens

—

Mean / SD

—

Median / MAD

—

Q1 / Q3 / IQR

—

Boxplot & strip plot

Every dot is one reading. Red dots are flagged under the current lens; the amber band shows that lens's fence(s).

Every reading, every lens — see the four methods argue with each other.
#	Value	Z-score	Modified Z	Inside IQR fences?

Interpretation guide

When you see this, it means this.
When you see this	It means this	What to say / do
All four methods flag the same point	Strong, convergent evidence of a genuine outlier.	Proceed to root-cause investigation with confidence.
Z-score misses it, but Modified Z and IQR flag it	Classic masking — the outlier has inflated the mean and SD enough to hide itself from a plain Z-score.	Trust the robust methods (median/MAD, IQR) over the Z-score here.
Grubbs' test says "not significant" on skewed data	Grubbs assumes approximate normality; a skewed shape can violate that assumption.	Check a probability plot first; fall back to IQR fences if normality is doubtful.
A cluster of points near the fence all get flagged	Possible swamping — legitimate points crowded by a genuinely shifted subgroup.	Investigate whether a real process shift (not individual point errors) occurred.
The flagged point has a confirmed calibration failure	Measurement error.	Exclude from analysis, document the reason and evidence, fix or replace the instrument.
The flagged point matches a known process/material change	Genuine special cause.	Keep it, but segment the analysis; investigate and correct the process, not the data.
The flagged point has no assignable cause after real investigation	Possibly a genuine rare event from natural variation.	Keep it in the primary analysis; consider reporting a robust statistic (median/IQR) alongside the mean/SD.

Common mistakes and exam traps

Judgment error

Deleting without investigating

Why wrong: you may be discarding real, important information (or hiding a real defect). Avoid it: always document a root cause before removing a point.

Method error

Using Grubbs' test on clearly skewed data

Why wrong: Grubbs assumes approximate normality; skew inflates false positives/negatives. Avoid it: check shape first, or use IQR fences instead.

Calculation error

Confusing MAD with average absolute deviation

Why wrong: MAD uses the median of absolute deviations from the median — not the mean. Excel's AVEDEV computes a different, less robust quantity. Avoid it: build MAD explicitly with MEDIAN(ABS(...)).

Statistical trap

Ignoring masking

Why wrong: a second extreme point can inflate the mean and SD enough that the plain Z-score misses both. Avoid it: cross-check with Modified Z-score or IQR fences, which resist masking.

Exam trap

Treating "outlier" and "error" as synonyms

Why wrong: a statistical outlier can be a completely genuine, correctly recorded extreme value. Avoid it: the ASQ exams expect you to know that removal requires a documented, investigated reason.

Process error

Fixing the data instead of the process

Why wrong: if a genuine special cause produced the point, excluding it from a report without fixing the root cause lets the problem recur. Avoid it: route genuine special causes to corrective action, not just to the exclusions column.

Statistics Implementation: Excel & Minitab

Excel Functions

Functions used to detect and characterize outliers in Excel.
Function	Syntax	Purpose	When to use it
`AVERAGE`	`=AVERAGE(range)`	Sample mean, x̄	Baseline center for the Z-score method.
`MEDIAN`	`=MEDIAN(range)`	Robust center, x̃	Foundation for the Modified Z-score and MAD; unaffected by the outlier itself.
`STDEV.S`	`=STDEV.S(range)`	Sample standard deviation	Denominator of the Z-score and Grubbs' G, for a sample (n − 1 divisor).
`STDEV.P`	`=STDEV.P(range)`	Population standard deviation	Only when your data is the entire population, not a sample — rare in process work.
`STANDARDIZE`	`=STANDARDIZE(x, mean, sd)`	Computes a Z-score directly	Fast one-cell Z-score without building the subtraction/division yourself.
`QUARTILE.INC`	`=QUARTILE.INC(range, quart)`	Q1 (quart=1) and Q3 (quart=3), inclusive method	Standard, most-taught method for building IQR/Tukey fences.
`QUARTILE.EXC`	`=QUARTILE.EXC(range, quart)`	Q1/Q3, exclusive method	Use when your course or standard specifically calls for the exclusive definition; gives slightly different fences on small n.
`PERCENTILE.INC`	`=PERCENTILE.INC(range, k)`	Any percentile	Cross-checking quartiles or building custom fences (e.g., 5th/95th percentile).
`ABS`	`=ABS(number)`	Absolute value	Core building block for MAD, Modified Z-score, and Grubbs' numerator.
`AVEDEV`	`=AVEDEV(range)`	Average absolute deviation from the mean	Note: this is not MAD. Don't substitute it for the median-based MAD formula.
MAD (array formula)	`=MEDIAN(ABS(range-MEDIAN(range)))`	Median Absolute Deviation	Enter as an array formula (Ctrl+Shift+Enter in older Excel, or automatically in dynamic-array Excel/365). Required for the Modified Z-score.
`LARGE` / `SMALL`	`=LARGE(range,1)` / `=SMALL(range,1)`	Extreme values by rank	Pull the most extreme value to feed Grubbs' numerator, max\|xᵢ − x̄\|.
`T.INV.2T`	`=T.INV.2T(probability, deg_freedom)`	Two-tailed inverse t critical value	Computes the t value needed inside the Grubbs' critical-value formula.
`T.DIST.2T`	`=T.DIST.2T(x, deg_freedom)`	Two-tailed p-value from a t statistic	Convert a Grubbs-derived t statistic back into a p-value for reporting.
`COUNT`	`=COUNT(range)`	Sample size, n	Needed everywhere — Grubbs' critical value depends heavily on n.
`IF` / `AND`	`=IF(ABS(z)>3,"Flag","OK")`	Flagging logic	Turn any of the above statistics into a pass/fail column for review.

Excel use cases

Building a Z-score column (data in A2:A17, mean in D1, SD in D2): =(A2-$D$1)/$D$2, filled down, then flag with =IF(ABS(B2)>3,"Flag","").

Building the Modified Z-score: with median in D3 and MAD in D4 (array formula above), =0.6745*(A2-$D$3)/$D$4, flag at |M| > 3.5.

Building IQR fences: =QUARTILE.INC($A$2:$A$17,1) and =QUARTILE.INC($A$2:$A$17,3) in two cells, then =Q3-Q1 for IQR, and =Q1-1.5*IQR / =Q3+1.5*IQR for the fences.

Assembling Grubbs' critical value: with n in D5, α = 0.05: =T.INV.2T(0.05/D5, D5-2) gives the t value, then =((D5-1)/SQRT(D5))*SQRT(t^2/(D5-2+t^2)) gives G_crit. Compare against G = =MAX(ABS(A2:A17-$D$1))/$D$2 (entered as an array formula).

Caution: Excel has no single built-in "Grubbs' test" function — you assemble it from T.INV.2T, exactly as shown above. Don't confuse this with a simple Z-score screen.

Minitab Navigation

Descriptive statistics & quartiles

Stat → Basic Statistics → Display Descriptive Statistics. Returns mean, median, SD, Q1, Q3, and IQR directly — no manual formulas needed.
Alternative: Stat → Basic Statistics → Graphical Summary adds a normality test and confidence intervals alongside the same statistics.
Use Graphical Summary when you also need to judge whether Grubbs' normality assumption is reasonable.

Formal outlier test

Stat → Basic Statistics → Outlier Test. Runs Grubbs' test directly on a column and reports the flagged value and p-value.
Use this when you need a defensible, documented p-value rather than an eyeballed fence.
Check the normality assumption first (Graphical Summary's Anderson-Darling test) — Minitab will not stop you from misapplying it to skewed data.

Visual screening

Graph → Boxplot. Minitab marks points beyond 1.5×IQR with an asterisk automatically — your fastest first look.
Graph → Individual Value Plot or Graph → Scatterplot for a raw view without box-plot compression.
Use the boxplot for a quick screen, then follow up with Outlier Test or Descriptive Statistics for the formal number.

Time-ordered process data

Stat → Control Charts → Variables Charts for Individuals → I-MR. Flags points beyond 3σ control limits in the context of process order, not just static distribution.
Use this instead of a static outlier test whenever the data are collected over time — a control chart also tells you whether the shift persisted.
Stat → Quality Tools → Individual Distribution Identification helps confirm which distribution model is reasonable before trusting Grubbs or a capability study.

Exam tips

ASQ CSSBB / CQE exam connections: outlier detection sits under the Measure phase / basic statistics domain. Expect questions that (1) ask you to compute a Z-score, IQR fence, or interpret a Grubbs' test result by hand or from output; (2) test whether you know Grubbs' test requires an approximate normality assumption while IQR fences do not; (3) present a scenario and ask what you should do before removing a point (the answer is almost always "investigate and document," not "delete"); and (4) distinguish an outlier from a point that is merely influential in a regression context. In real-world capability studies, always report whether outliers were included or excluded and why — an auditor will ask.

Practice activity

A rolling mill logs 12 coil widths (mm): 1250, 1248, 1252, 1251, 1249, 1253, 1247, 1250, 1252, 1249, 1251, 1310. Using the IQR method, is 1310 mm an outlier? (Q1 = 1249.00, Q3 = 1252.00.)

Final summary

An outlier is a point inconsistent with the bulk of the data — not automatically an error.
Four core detection tools: Z-score, Modified Z-score (MAD-based), IQR/Tukey fences, and Grubbs' test — each with different assumptions and blind spots.
Masking (an outlier hiding itself or another) and swamping (flagging good points near a real cluster) are real failure modes — cross-check methods.
Detection is step one. Investigation — instrument logs, sample logs, operator interviews — decides what a flagged point actually is.
Never delete a point without a documented, investigated reason; report results with and without the point when in doubt.
Excel builds each method from AVERAGE/MEDIAN/STDEV.S/QUARTILE.INC/T.INV.2T; Minitab's Outlier Test, Boxplot, and I-MR chart give the same answers with far less manual formula work.

Decision rule: screen with IQR fences (always works) → confirm with Grubbs' test if approximately normal → investigate root cause before any exclusion → document the decision either way.

Remember this: statistics finds the suspect. Investigation finds the verdict.

Suggested next lesson: Chebyshev vs. the Empirical Rule.

References

NIST/SEMATECH, Grubbs' test for outliers, e-Handbook of Statistical Methods.
NIST/SEMATECH, Detection of outliers, e-Handbook of Statistical Methods.
Iglewicz, B. and Hoaglin, D., How to Detect and Handle Outliers, ASQC Quality Press.
Microsoft Support, T.INV.2T function.
Minitab Support, Outlier Test overview.

Dealing with Outliers: A Steel & Mining Detective's Guide

Learning objectives

Prerequisites

What exactly is an outlier?

🚫 The value itself is wrong

✅ The value is correct

🔧 Measurement / instrument error

⌨️ Data / transcription error

⚙️ Genuine process shift (special cause)

💎 Genuine rare event (natural extreme)

Key terminology

Z-score

MAD

Modified Z-score

IQR / Tukey fences

Grubbs' test

Masking & swamping

Winsorizing

Trimming

Influential point

Watch masking happen, with real numbers

The four detection methods

1. Z-score

2. Modified Z-score (MAD-based)

3. IQR / Tukey fences

4. Grubbs' test (ESD, one-sided or two-sided)

Which method, when?

Small n, unsure of shape

Approximately normal, need a formal test

Suspect more than one outlier

Quick screen, large n, normal-ish

Time-ordered process data

Regression / multiple variables

Step-by-step investigation procedure

Worked example: the furnace mystery

Before: including the glitch

After: glitch excluded

Interactive investigation lab

Case files

Choose your detection lens

Boxplot & strip plot

Interpretation guide

Common mistakes and exam traps

Deleting without investigating

Using Grubbs' test on clearly skewed data

Confusing MAD with average absolute deviation

Ignoring masking

Treating "outlier" and "error" as synonyms

Fixing the data instead of the process

Statistics Implementation: Excel & Minitab

Excel Functions

Excel use cases

Minitab Navigation

Descriptive statistics & quartiles

Formal outlier test

Visual screening

Time-ordered process data

Exam tips

Practice activity

Check your understanding

Comprehension quiz

Final summary

References