UpSkill Sprint Consulting
Bias never averages out.
Bias is the lean that doesn't cancel out. Take more samples, run more tests, collect more data, and it stays right where it is. Try it on the gauge target.
This lesson covers what bias is, the three families it comes in, 83 named biases with worked examples, a quiz, and a toolkit for pushing back.
Three one-page PDF posters to print for the office wall, share with your team, or keep beside you during your next analysis.
Bias is a systematic error: a consistent tendency for a result to be pushed away from the truth in one direction. It comes from how data is collected, how it's measured, how it's analyzed, or how the human in the loop thinks about it.
The key word is systematic. Random error scatters results around the truth, so averaging helps. Bias moves the whole scatter, so averaging just gives you a very precise wrong answer. In MSA terms, noise is a precision problem and bias is an accuracy problem.
The word shows up in two worlds that turn out to be the same world:
In statistics, an estimator is biased when its average across repeated samples doesn't equal the value it's trying to estimate.
In psychology, a cognitive bias is a predictable shortcut in human judgement that bends conclusions the same way every time.
Both are a thumb on the scale. One lives in the data pipeline, the other lives in our heads, and most real failures involve both.
3,000 simulated samples at each size, drawn from a population with true variance 1.
More data fixes noise. Only better design fixes bias.
World War II. Bombers come back from missions riddled with bullet holes, clustered on the wings, the tail and the middle of the fuselage. Armor is heavy, so you can only add it in a few places. The military's first instinct was to reinforce where the holes were.
Statistician Abraham Wald, working with the Statistical Research Group at Columbia, pointed out what everyone was missing. Take a guess before you reveal it.
Armor the places without holes: the engines and cockpit. The sample only contained planes that made it home. Planes hit in those spots didn't come back to be counted. The missing data was the most important data.
That's survivorship bias, and it's hiding in more plant data than you'd think. A model trained only on pipe that reached final inspection has never seen the pipe that was scrapped upstream.
Every pipe that gets scrapped upstream disappears from the final inspection database. Build a model on that database and it has never seen a failure. Illustrative numbers.
A useful way to sort the whole zoo is by where the lean enters: the data and the math, the people making judgements, or the system everyone operates inside.
Built into how data is sampled, measured, modelled and monitored. It skews numbers even when every person involved is being careful and fair.
"Is the data a fair picture of reality?"
Cognitive shortcuts in memory, social behaviour and belief. Fast and usually useful, but predictably wrong in specific situations.
"Am I seeing what's there, or what I expect?"
Embedded in history, social norms and institutional rules. Nobody has to intend it; the structure produces it and the data records it faithfully.
"What did the past bake into the data?"
Each stage of turning reality into a decision has its own biases. Select a stage to jump to its biases in the field guide.
Five small experiments with simulated data. Press the buttons and watch the lean appear.
Toughness varies along this simulated coil: the slower-cooled middle is weaker. Cutting test coupons from the head is convenient. See what it does to the estimate.
400 customers surveyed, 48 replied. Each square is one customer.
Chance of at least one false "significant" result when testing variables that truly have no effect, at α = 0.05.
30 defects placed completely at random along a 12 m pipe. The shaded band is the densest metre.
100 fair coin flips. The ringed run is the longest streak. Streaks are normal in random data.
Every bias from your six reference graphics, merged where they overlap, each with a plain definition and a worked example. Tap "Show example" on any card. Graphic numbers refer to the order the images were shared.
The group you study differs systematically from the group you want conclusions about.
The collection method makes some units more likely to be picked than others, so the sample stops being random.
People or units drop out partway through a study, and the ones who leave differ from the ones who stay.
Only the survivors are visible, so the failures that would change the story are missing.
People choose whether to be included, and the people who opt in differ from those who don't.
The people who don't answer differ in important ways from the people who do.
Part of the population has little or no chance of being included in the sample at all.
Data points or groups are removed, carelessly or deliberately, in a way that skews results.
The data under-represents conditions or groups the conclusion or model will later face.
Data from one period is treated as if it describes a different period.
The measuring process reads consistently high or low, or measures some groups differently from others.
People remember past events inaccurately, coloured by what happened afterwards or how they feel now.
The person recording results sees what they expect to see.
What gets written down reflects what people find notable, not how often things actually happen.
Results tend to tilt towards the interests of whoever paid for the study.
The interviewer's wording, tone, body language or reactions nudge how people answer.
Participants work out what the study is looking for and change their answers or behaviour to fit it.
Some respondents habitually pick the endpoints of a scale, regardless of how strongly they actually feel.
Respondents default to the middle option to avoid committing, which hides real opinions. Also called central tendency bias.
Earlier questions prime how people answer later ones.
People give the answer they think the group or the majority holds, rather than their own view.
A factor that affects both the supposed cause and the outcome is left out, so its effect gets credited to something else.
Treating correlation as causation, or getting the direction of cause wrong.
Trying many models or variables and reporting only the best fit, which overstates how well it will really perform.
The relationship a model learned changes over time, so its predictions quietly degrade.
A model's outputs shape the data it later learns from, reinforcing its own assumptions.
Small errors early in a pipeline compound through each later step.
A system is used in a way that differs from how it was designed and validated.
Following a model's recommendation only when it matches what we already believe.
Trusting an automated system over our own correct judgement or contradictory evidence.
Leaning too heavily on the first number or piece of information we encounter.
Judging how likely something is by how easily examples come to mind.
Reacting differently to the same information depending on how it's presented.
Continuing because of what has already been spent rather than what lies ahead. Also called escalation of commitment.
Preferring things to stay the same, so any change feels like a loss.
Preferring to eliminate a small risk entirely over substantially reducing a larger one.
Believing past independent random events change the odds of future ones.
Seeing meaningful patterns or streaks in random data.
Spending disproportionate time on trivial issues because they're easy to have opinions about. Also called bike-shedding.
People with limited skill in an area tend to overrate it, while experts see how much they don't know.
Once we know something, it's hard to imagine not knowing it.
Forgetting information we know we can easily look up. Also called digital amnesia.
Unfinished tasks stick in memory more than completed ones.
Placing extra value on things we built or partly created ourselves.
Our sense of time stretches or compresses under stress, fear or exertion.
Remembering events that didn't happen, or that happened differently.
Taking on ideas suggested by others as our own memories or conclusions.
A forgotten memory resurfaces and is mistaken for a brand-new idea.
Romanticising the past and expecting the future to be worse.
Overestimating the likelihood of bad outcomes.
Overestimating the likelihood of good outcomes.
Believing a treatment will work can produce real improvement. Flagged in the source graphic as not strictly a cognitive bias.
Blaming other people's behaviour on their character and our own on circumstances.
Crediting our successes to our own skill and our failures to outside factors.
Giving more trust and leniency to people in our own group.
Seeing members of other groups as all alike while seeing our own group as diverse.
Assuming an individual has traits we associate with their group, without real information about them.
Unconscious associations that shape our judgements without our awareness.
The desire for harmony and agreement overrides critical thinking.
Adopting an idea or behaviour because many others already have.
Giving extra weight to an authority figure's opinion regardless of the evidence.
One positive trait colours our impression of everything else about a person or thing. It works for negative traits too.
Overestimating how many people share our opinions.
Believing others notice our appearance and mistakes far more than they do.
Blaming a victim less when we can imagine being in their position, and more when we can't.
The more people present, the less likely any one of them is to act. Flagged in the source graphic as not strictly a cognitive bias.
We tend to like people more after doing them a favour.
A belief becomes more credible simply by being repeated in public.
Believing other people are more influenced by media and advertising than we are.
Seeking out, noticing and remembering information that supports what we already believe.
Judging an argument's logic by whether we like its conclusion.
Evidence against a belief can make some people hold it more firmly.
Doing the opposite of what we're told when we feel our freedom is being restricted.
Believing we see the world objectively and that people who disagree must be uninformed or biased.
Expecting others to be more self-interested than they really are.
Recognising bias in other people while believing we're less affected ourselves.
Assuming people generally get what they deserve.
Judging a decision by its outcome rather than by the quality of the decision itself.
Accepting vague, general statements as uniquely accurate descriptions of ourselves. Also called the Barnum effect.
Items placed higher in a list get chosen or trusted more, regardless of merit.
The data accurately records a past that was itself unfair or unrepresentative.
Shared norms and stereotypes in society seep into data and decisions.
An organisation's policies and practices produce skewed outcomes even when individuals mean well.
No biases match that search. Try a broader word like "data" or "memory".
Nine scenarios. Pick the bias that's doing the damage.
You can't switch bias off, in data or in people. You can design processes that catch it.
Let chance pick the sample, and make sure every subgroup gets represented. Beats: sampling, selection, undercoverage, self-selection.
Track dropouts, non-responders, scrapped units and excluded rows. Ask who never made it into the dataset. Beats: survivorship, attrition, nonresponse, exclusion.
Use neutral wording, randomize question order, keep surveys short, and collect sensitive answers anonymously. Beats: interview, question order, conformity, demand characteristics, neutral and extreme response.
Hide the group label, heat history or expected answer from whoever measures or judges. Beats: observer, confirmation, halo, implicit.
Check bias, linearity, stability and repeatability before trusting the numbers. Beats: measurement, error propagation.
Write down the hypothesis and analysis plan first, and correct for multiple comparisons. Beats: model selection, confirmation, clustering illusion.
Use designed experiments where you can, and stratify or adjust where you can't. Beats: omitted variable, cause-effect.
Control-chart model inputs and errors, and audit how the tool is actually used on the floor. Beats: concept drift, feedback loops, deployment, temporal.
Run a pre-mortem, appoint a devil's advocate, and let the most junior person speak first. Beats: groupthink, authority, bandwagon, sunk cost.
Ask whether the choice was sound given what was known at the time. Beats: moral luck, self-serving, hindsight-driven recall.
Use a reference standard to check measurement bias, and compare the sampling mix with the population you intend to describe. A spreadsheet can reveal a mismatch; it cannot repair an unrepresentative sampling plan by itself.
| Function | Syntax | Purpose | When to use it |
|---|---|---|---|
AVERAGE | AVERAGE(number1, [number2], …) | Mean of measurements | Compare a gauge's mean reading with a known reference. |
VAR.S | VAR.S(number1, [number2], …) | Sample variance with the n − 1 correction | Estimate variance from a sample of repeated readings. |
VAR.P | VAR.P(number1, [number2], …) | Variance of the entire supplied population, divided by n | Use only when those readings really constitute the entire population of interest. |
COUNTIF / COUNTIFS | COUNTIF(range, criteria)COUNTIFS(criteria_range1, criteria1, [criteria_range2, criteria2], …) | Count records in one or several groups | Compare tested and untested units across shifts, coil positions, or other strata. |
AVERAGEIF / AVERAGEIFS | AVERAGEIF(range, criteria, [average_range])AVERAGEIFS(average_range, criteria_range1, criteria1, [criteria_range2, criteria2], …) | Compare means within defined groups | Check whether day/night or head/middle/tail samples differ. |
Gauge offset: Enter four readings of a 10.00 mm reference in B2:B5: 10.2, 10.3, 10.1, 10.4. =AVERAGE(B2:B5)-10 gives +0.25 mm. =VAR.S(B2:B5) gives about 0.0167 mm², whereas =VAR.P(B2:B5) gives 0.0125 mm². The first is a sample variance estimate; the second treats these four readings as the full population.
Coverage audit: If C2:C101 contains shifts and D2:D101 contains “Yes” for sampled units, =COUNTIFS(C2:C101,"Night",D2:D101,"Yes") counts night-shift samples. Divide by COUNTIF(C2:C101,"Night") to compare the night sampling rate with other shifts. Check that the denominator is nonzero.
Stratified mean: If coil position is in E2:E101 and toughness is in F2:F101, =AVERAGEIF(E2:E101,"Middle",F2:F101) gives the middle-coil mean. For a second condition, =AVERAGEIFS(F2:F101,E2:E101,"Middle",C2:C101,"Night") gives the night-shift middle-coil mean. Treat differences as a cue to investigate the sampling design, not proof of causation.
For CSSBB and CQE practice, first locate where the bias entered: selection, measurement, analysis, or interpretation. In measurement questions, separate bias (average offset from a reference) from repeatability (scatter of repeated readings). In sampling questions, identify who or what is missing before calculating a more precise estimate. State the population and sampling frame explicitly in a real analysis.
Software references: Microsoft Excel statistical functions; Minitab Gage Linearity and Bias Study; Minitab Type 1 Gage Study; Minitab descriptive statistics.
Knowledge check
Pick the strongest answer, then submit. Explanations appear after you submit.
