Entity Resolution

Entity resolution is deciding which records, across systems or within one dataset, refer to the same real-world thing — the same customer, supplier, product, or person — when no shared identifier says so. AWS describes its service for this as one that “helps you match, link, and enhance related records stored across multiple applications, channels, and data stores.” Product descriptions below follow that documentation, checked in September 2026.

The problem exists only because of a missing key. Where two systems already share an identifier — a company registration number, an employee ID issued once at hire — records join and little here is needed — provided the identifier covers the same entity scope, is issued by one authority, and is accurate, complete, and never reused. Entity resolution is what fills in for an identifier that was never agreed.

Two approaches with different failure characteristics

Tools typically offer “rule-based matching, machine learning-based matching (ML matching), and data service provider-led matching,” and normalize inputs before comparing — “removing special characters and extra spaces, and formatting text to lowercase,” which alone resolves a surprising share of apparent mismatches.

Explainability is not an abstract virtue here. In any domain where a match decides who sees whose data, someone will eventually ask why two records were joined, and the answer has to be better than a number.

The two errors are not equally expensive

Every matching system makes both kinds of mistake, and the design question is which one to prefer.

• Over-merging joins two different entities. One party’s data becomes visible under another’s record, invoices reach the wrong company, and a person’s history is polluted with someone else’s. Undoing it is hard, because the merged record has already been read and acted on.
• Under-merging leaves one entity split. Outreach duplicates, customer value is understated, a support agent sees half the history. Fixing the link is simple — merge them when someone notices — but decisions, outreach, and support already given on the split history are not undone by the merge.

That asymmetry should set the threshold rather than a target duplicate rate. A program measured on duplicates alone will push toward over-merging, because duplicates are easy to count from the output while wrong merges are not — they need a labeled sample or approved cases to be measured, and otherwise show up as a complaint months later, if at all. Where a wrong merge exposes data across parties, the uncertain band belongs to a person: auto-merge above a high bar, auto-reject below a low one, and route the middle to review.

Matching is not merging

A useful implementation keeps the two steps apart. Resolution produces groups and an identifier — the documented output is “a consolidated data output table with similar records that have a common match ID” — and that is a link, not a rewrite. What the combined record should say is a separate decision governed by survivorship rules, and the assembled result is a golden record.

Keeping them separate is what makes a wrong match reversible: the source records still exist, the grouping can be corrected, and the derived record is rebuilt. Systems that merge destructively turn every matching error into a data-recovery problem.

Two operational habits follow. Store the reason for each match — the rule number or score — alongside the group, so a disputed decision can be reviewed rather than re-argued. And treat manual merges and unmerges as recorded decisions with a scope and an effective period, which the next automated run applies while they are in force; a re-match that silently undoes human corrections will be discovered by the person who made them.

How resolution fits with survivorship, implementation styles, and correcting mistakes is worked through in One Customer, Five Systems.

References: AWS Entity Resolution User Guide, What is AWS Entity Resolution?; Splink documentation, Evaluation; Splink documentation, Waterfall chart.