Measurement Error

Measurement error is the difference between a recorded value and the thing it was meant to describe. Some of it is random and averages out over many records; some of it is systematic and does not. NIST’s handbook defines bias as the difference between the average of repeated measurements and the true value, and notes that bias can be present even when every instrument is properly calibrated.

Where the error enters

Instruments: a server clock drifts, a sensor rounds to the minute, a thermometer reads high. People: an agent presses “reply” before writing, a respondent gives the socially expected answer, a clerk types 9 for 0. Definitions: a reply field that includes automatic acknowledgements records a zero-minute wait for a ticket a person answered forty minutes later; the value is exactly what the system was told to record and is still wrong for the question. Processing: rounding, unit conversion, and coding of free text add their own errors. Statistics Canada groups the survey versions of these as measurement, response, and processing errors.

Finding it and living with it

Random error shows up as scatter, and averaging many values reduces its effect only when the errors are independent of one another, or nearly so, and centre on zero; correlated errors and genuine differences between the people measured do not average away. Systematic error shows up as a consistent offset that no averaging removes, and it is found by comparing against an independent reference, checking a sample against source evidence, or reading the data dictionary to see what was really recorded. An odd value is not automatically an error: 3,795 clock minutes for a ticket created on Friday evening and answered on Monday morning describes what happened; the mismatch is in the time basis used, clock time instead of the support hours the question asks about, not in the record.

State the likely error alongside a result: reply times to the nearest minute, survey answers subject to self-report, a category coded by one person. A measurement with its error stated is more useful than a precise-looking number whose error is unknown.

References: NIST/SEMATECH e-Handbook: Bias and accuracy, Statistics Canada: Non-sampling error. Examples here are illustrative.