Quality Threshold

A quality threshold is the point at which a data check stops passing: zero null customer identifiers, no more than 0.1 percent of rows failing a range rule, freshness no worse than two hours. Tools expose it directly — dbt lets a test’s severity be warn or error and provides error_if and warn_if configs “to set custom failure thresholds in your tests,” and source freshness takes warn_after and error_after as a count and a period. Descriptions follow dbt’s documentation, checked in September 2026.

Setting the number is the part that gets done badly, because a threshold looks like a technical parameter and is actually a statement about what the organization is willing to publish.

Derive it from what it protects

The reliable method is to work backwards from the decision that consumes the data, which usually produces a number nobody would have chosen by feel.

• A deadline gives a freshness threshold. A report due at 08:00 after a 40-minute build makes 07:00 the moment an input is already late — so the threshold is 07:00, not “a few hours.”
• A materiality rule gives a tolerance. If a figure is reported to the nearest thousand, an error that cannot move it by a thousand does not need to block publication. If the figure feeds a payment, it does.
• An identity rule gives a zero. Duplicate primary keys, nulls in a join key, amounts in the wrong currency: these have no acceptable rate, because a single occurrence corrupts a result rather than degrading it.

That last category is worth separating explicitly. Some checks are structural — the data is either usable or it is not — and giving them a tolerance is a category error. Others are statistical, where a small rate is expected and the question is how much. Mixing the two produces suites where a genuine corruption passes because it fell under a percentage someone set globally.

Two levels, not one

A single threshold forces one response to every degree of failure. Two — a warning level and a blocking level — let the check say something useful at both. A null rate that rises from 0.1 to 1 percent is worth investigating and not worth stopping publication over; the same rate at 20 percent means the load is broken. Tools support this directly, and the value of using it is that alerts stop being binary: the warning level is where trends are noticed, and the blocking level is where publication is withheld and consumers keep the previous good version.

Both levels need the same four elements any measurable target needs: the population the rule applies to, where it is measured, over what window, and the value that counts as failing. A threshold without a window — “no more than 1 percent nulls” — is not enforceable until someone says whether that is per load, per day, or per month.

The failure mode to watch for

Thresholds drift downward under delivery pressure. A check fails, the release is urgent, the threshold is relaxed “temporarily,” and the suite returns to green without anything improving. This is observable and worth observing: a record of threshold changes, with who changed each one and why, distinguishes a deliberate recalibration from an accumulation of exceptions.

Two habits keep it honest. Give every threshold an owner — the person whose decision it protects, not the engineer who wrote the test — and a review date, because a number derived from a deadline stops being right when the deadline moves. And when a threshold is loosened to let something through, record it as an accepted risk with an expiry rather than as a settled value. Stated as a target someone owns, the whole arrangement is the same discipline an objective needs; without an owner it is a preference that erodes.

How thresholds fit with checks, contracts, and incident response is worked through in Passing Tests, Wrong Numbers.

References: dbt Documentation, Data tests; dbt Documentation, Sources (source data freshness).