Fine-Tuning Datasets and Evaluation Splits

Consider a hypothetical team that fine-tunes a language model to draft replies to customer-support tickets. Before tuning, the model earns 61% on the team’s held-out set of 400 tickets; after tuning it earns 90%. The team ships. Within a month, the reply quality that customers see is nowhere near 90%. A review finds that the 400 held-out tickets were sampled at random from a pool in which every ticket belongs to a conversation thread, so most held-out tickets had a sibling from the same thread in the training data. The same customer name, the same order number and often the same sentence appeared on both sides of the split. The 90% measured how well the model recalled its training set. It said little about tickets from new customers.

Nothing in the training loop was wrong. The loss went down, the evaluation script ran, and the number was computed correctly. The failure was in the data design: the held-out set did not match the question the team needed answered, and no one had written that question down. Designing the data around that question comes first, and it covers what the dataset must represent, how it is sampled and labeled, how leakage and contamination are found, whether the data may be used at all, and how splits and statistics decide what an evaluation number can support.

The training methods are those of Fine-Tuning and Alignment: SFT, LoRA, RLHF, and DPO, which explains what supervised fine-tuning and preference training consume (demonstrations, and comparisons between two responses to the same prompt). The view of language models comes from Language Models: Objectives, Capabilities, and Limits, in particular the gap between what a model can do on some prompts and how reliably it does it. The code below uses Python 3.12 and the standard library only, and every dataset in it is synthetic.

Start from the task contract

A fine-tuning dataset is a statement about behavior. Each training example says that, given this input in this setting, the model should produce something like this output. Collecting examples before deciding what behavior is wanted produces a pile of text whose coverage nobody can describe. A short task contract fixes the question first. It states four things:

• Inputs. What arrives at inference time: which users, channels, languages, lengths, and formats, and what context accompanies the request (retrieved passages, conversation history, a system prompt).
• Outputs. What a good response looks like: required structure, tone, allowed and disallowed content, and when the right behavior is to decline or ask a question.
• What changes. Which behaviors fine-tuning is supposed to change, and which capabilities must stay as they are.
• Success criteria. How a response will be judged: programmatic checks, rubric scores from reviewers, task outcomes. These decide what an evaluation example must contain.

The third item is where fine-tuning differs from classical supervised learning. Supervised fine-tuning applies next-token training to demonstrations, so it teaches whatever regularities the demonstrations share: a response format, a style, a policy for refusals, a way of using supplied context. Whether it should be used to add facts the model lacks is a separate question. Gekhman et al. studied closed-book question answering and found that fine-tuning examples containing knowledge unfamiliar to the model were learned more slowly than examples consistent with what the model already knew, and that once learned they increased the tendency to hallucinate. In that setting, the data worked better for teaching how to apply existing knowledge than for adding new knowledge. The limits on factuality that the language-model article describes carry over: a dataset of correct answers does not make the model’s recall of those answers reliable. When the requirement is up-to-date or auditable facts, the usual design is to supply them in context through retrieval and use fine-tuning data to teach the model how to use that context.

Where examples come from

Task data comes from a handful of source types, and each one has a characteristic failure. The table summarizes them; the later sections return to each failure.

Source	What it gives	Characteristic failure
Written by annotators or domain experts	Controlled coverage and style; responses that follow the guidelines	Cost limits volume; annotators converge on a house style that differs from real inputs
Logs from an existing system or from users	Realistic input distribution	Reflects whichever users and prior system produced the logs; contains personal data; responses in logs are often not the responses you want to imitate
Curated public datasets	Cheap breadth	Unclear licenses; may overlap with evaluation sets or with the base model’s pretraining data; labels of unknown quality
Synthetic data generated by another model	Scale, and coverage of cases that are rare in logs	Inherits the generator’s errors and style; may carry usage restrictions from the generator’s terms; can embed evaluation items if the generator was prompted with them
Existing documents transformed into tasks (for example, questions written about a manual)	Grounding in authoritative text	Question and answer share a source document, so a random split places the same document on both sides

Synthetic generation deserves a closer look because it is cheap and easy to misuse. Self-Instruct, by Wang et al., has a model generate instruction–response pairs, filters out low-quality and near-duplicate ones, and fine-tunes on the result. The filtering step is part of the method, not an optional extra: unfiltered model output is a sample from the generator’s distribution, so training on it moves the tuned model toward the generator’s habits, including its mistakes. Treat generated data like any other source: record where it came from, sample it, and have people check a slice of it before it is trusted. For the instruction-following style of data, the glossary entry on instruction tuning gives a short orientation.

Coverage before volume

Volume is the number people ask about first, and it is the least informative property of a dataset. Zhou et al. fine-tuned a model with standard supervised loss on 1,000 carefully curated prompts and responses and argued that most of what an assistant needs from instruction tuning is limited and can be taught with a small, high-quality set. That result is specific to their model, data, and evaluation, but it illustrates the right priorities: what matters is whether the examples span the behaviors you need, and whether each one is a response you would be willing to see the model produce.

A practical way to reason about coverage is a slice matrix. List the dimensions along which inputs differ in ways that change the right response, then count examples in each cell. For the support-reply task, the dimensions might be ticket topic, language, input length, whether the ticket contains an order number, and whether the correct action is to answer, to ask a clarifying question, or to escalate. The matrix does three jobs. It shows which cells have no examples, so the gap can be filled deliberately. It shows which cells dominate, which indicates where a model will drift toward the majority behavior. And it defines the slices that the evaluation set must report on, so training and evaluation coverage are planned together.

Sampling: matching the data to the deployment distribution

A dataset is a sample, and the first question about any sample is which population it represents. The fine-tuned model will face the deployment distribution: the tickets that arrive next quarter, from the customers who write then. The collected pool is some other distribution, shaped by who was logged, which system handled them, what was retained, and what survived filtering. Where the pool systematically differs from the deployment population, that is selection bias, and no amount of additional data from the same process removes it. For example, a pool built from tickets that an earlier rule-based system escalated to humans contains only the hard or unusual tickets, and a model tuned on it has not seen the easy majority.

Written down, the question has two parts. The pool should cover the deployment population (every kind of input that matters appears), and the training mixture should weight it deliberately. Uniform random sampling from the pool does the second part by default, which is usually wrong for fine-tuning: it gives the most examples to the cases that are already common and easy, and almost none to the rare, high-stakes ones. Stratified sampling takes a quota from each slice instead. A quota raises a rare slice’s share of the training signal, and it changes the effective data distribution, so it is a design decision with a cost: the model sees rare cases more often than they occur and can over-apply their behavior.

There are two standard ways to handle the difference. One is to accept the changed mixture and check on a test set whose proportions match deployment that the common cases did not regress. The other is to restore deployment proportions in the loss with per-slice weights, \(w_s = \pi_s / q_s\), where \(\pi_s\) is the slice’s share of the deployment pool and \(q_s\) its share of the sample. Weights have a variance cost: a slice whose weight is 0.05 contributes almost nothing to the gradient, so the sampling quota has done little there. The usual compromise is to tune the quotas and measure per-slice results, instead of treating either extreme as correct. The glossary entry on sampling explains population, frame, and selection method; the quota and weighting choices here are training-design decisions.

The first program below makes the arithmetic concrete. It builds a synthetic pool of 9,100 tickets with six slices (three topics by two lengths) in which “legal” tickets are rare, draws 500 examples uniformly and 500 with quotas, and prints the loss weights that would restore pool proportions. The same program also contains the label-quality calculations used in the next section.

import random
from collections import Counter

# --- 1. Quota sampling by slice -------------------------------------------
random.seed(7)
pool = ([("billing", "short")] * 6000 + [("billing", "long")] * 1500 +
        [("refund", "short")] * 1200 + [("refund", "long")] * 300 +
        [("legal", "short")] * 60 + [("legal", "long")] * 40)
print("pool", len(pool), dict(Counter(pool)))

def quota_sample(items, quotas, rng):
    by_slice = {}
    for it in items:
        by_slice.setdefault(it, []).append(it)
    out = []
    for s, q in quotas.items():
        out += rng.sample(by_slice[s], min(q, len(by_slice[s])))
    return out

rng = random.Random(1)
uniform = rng.sample(pool, 500)
quotas = {("billing", "short"): 120, ("billing", "long"): 80, ("refund", "short"): 100,
          ("refund", "long"): 80, ("legal", "short"): 60, ("legal", "long"): 40}
stratified = quota_sample(pool, quotas, rng)
print("uniform   ", dict(Counter(uniform)))
print("stratified", dict(Counter(stratified)))

# per-slice weight needed to restore pool proportions in the loss
pool_c, strat_c = Counter(pool), Counter(stratified)
for s in quotas:
    w = (pool_c[s] / len(pool)) / (strat_c[s] / len(stratified))
    print(s, "loss weight to match pool:", round(w, 3))

# --- 2. Cohen's kappa ------------------------------------------------------
def kappa(a, b):
    n = len(a)
    po = sum(x == y for x, y in zip(a, b)) / n
    ca, cb = Counter(a), Counter(b)
    pe = sum(ca[k] * cb[k] for k in set(a) | set(b)) / n ** 2
    return po, pe, (po - pe) / (1 - pe)

# 100 items; the two annotators disagree on 6 of them.
a = ["ok"] * 88 + ["bad"] * 2 + ["ok"] * 4 + ["bad"] * 6
b = ["ok"] * 88 + ["ok"] * 2 + ["bad"] * 4 + ["bad"] * 6
po, pe, k = kappa(a, b)
print(f"observed agreement {po:.2f}, chance agreement {pe:.3f}, kappa {k:.3f}")

# --- 3. Label noise caps measured accuracy ---------------------------------
def measured(true_acc, noise, n=200000, seed=3):
    r = random.Random(seed)
    hit = 0
    for _ in range(n):
        correct = r.random() < true_acc          # model matches the true label
        flipped = r.random() < noise              # test label is wrong (binary task)
        hit += (correct != flipped)               # agreement with the noisy label
    return hit / n

for noise in (0.0, 0.03, 0.08):
    print(f"test-label noise {noise:.2f}:",
          " ".join(f"true {t:.2f} -> {measured(t, noise):.3f}" for t in (0.80, 0.90, 1.00)))

pool 9100 {('billing', 'short'): 6000, ('billing', 'long'): 1500, ('refund', 'short'): 1200, ('refund', 'long'): 300, ('legal', 'short'): 60, ('legal', 'long'): 40}
uniform    {('billing', 'short'): 326, ('refund', 'short'): 62, ('billing', 'long'): 81, ('refund', 'long'): 22, ('legal', 'short'): 8, ('legal', 'long'): 1}
stratified {('billing', 'short'): 120, ('billing', 'long'): 80, ('refund', 'short'): 100, ('refund', 'long'): 80, ('legal', 'short'): 60, ('legal', 'long'): 40}
('billing', 'short') loss weight to match pool: 2.637
('billing', 'long') loss weight to match pool: 0.989
('refund', 'short') loss weight to match pool: 0.633
('refund', 'long') loss weight to match pool: 0.198
('legal', 'short') loss weight to match pool: 0.053
('legal', 'long') loss weight to match pool: 0.053
observed agreement 0.94, chance agreement 0.836, kappa 0.634
test-label noise 0.00: true 0.80 -> 0.800 true 0.90 -> 0.900 true 1.00 -> 1.000
test-label noise 0.03: true 0.80 -> 0.782 true 0.90 -> 0.876 true 1.00 -> 0.970
test-label noise 0.08: true 0.80 -> 0.753 true 0.90 -> 0.836 true 1.00 -> 0.920

The pool has 100 legal tickets out of 9,100 (about 1.1%), so uniform sampling returned 9 of them in 500 draws. The quota sample has 100 legal examples, a 20% share, which gives the model something to learn from but means legal tickets are over-represented about 18 times relative to the pool. The weight column states this exactly: the legal slices would need weight 0.053 to match pool proportions, while the common short billing slice would need 2.64. Neither weighting nor quotas is correct by itself. What the numbers establish is the size of the distortion, so it can be chosen and then checked against a test set with deployment proportions.

Two more sampling rules prevent common problems. Cap how many examples any single user, thread, or source document may contribute, so that one prolific contributor does not dominate a slice and so that the group-level split described later has many groups to divide. And record the sampling process (the query, the date, the quotas, the random seed) with the dataset, because a dataset whose selection cannot be reproduced cannot be extended or audited.

Labels: what the response is and how to trust it

For supervised fine-tuning of a generative model, the label is the response, and a prompt usually has many acceptable ones. That changes what labeling quality means. In classification, a label is right or wrong. In a demonstration, a response can be correct but verbose, correct but in the wrong tone, polite but unsupported by the provided context, or fluent and wrong. The guidelines have to specify the dimensions a good response is judged on, with examples of acceptable and unacceptable responses for the hard cases, or annotators will each apply a private standard and the model will learn the average of their disagreements.

Measuring whether people agree

The standard check is to have two or more annotators independently label or rate the same items and measure agreement. Raw agreement is the fraction of items on which they match, but it is inflated when one label is common. Cohen’s kappa corrects for that by comparing observed agreement \(p_o\) with the agreement \(p_e\) that two annotators would reach if each labeled at random with their own label frequencies:

\[\kappa = \frac{p_o – p_e}{1 – p_e}\]

The numerator is how much better than chance the annotators agree, and the denominator is the most they could have exceeded chance. Kappa is 1 for perfect agreement, 0 when agreement equals what independent labeling would produce, and negative when it is lower. It is undefined when \(p_e = 1\), which happens if both annotators use a single label throughout. The worked case in the program has 100 replies labeled “ok” or “bad”: the annotators match on 94 of them, so \(p_o = 0.94\). But one annotator marks 8 replies bad and the other marks 10, so chance agreement is \(0.92 \times 0.90 + 0.08 \times 0.10 = 0.836\), and \(\kappa = (0.94 – 0.836)/(1 – 0.836) \approx 0.634\). Agreement of 94% sounds high, but most of it was available for free from the “ok” majority, and the part that required actually seeing the same defects is closer to two-thirds. The glossary entry on inter-annotator agreement covers the interpretation and the weighted variants for ordinal ratings.

Low agreement is information about the task or the guidelines, not only about the annotators. Read the disagreements before averaging or discarding them. Sometimes the guideline is ambiguous and needs a rule. Sometimes both answers are acceptable, which suggests turning the item into a preference pair or accepting multiple references. Record adjudicated decisions and unresolved items, and measure agreement again on a new batch after the guideline changes, not on the examples used to revise it.

Noise in training labels and in test labels

Label errors affect the two ends of the pipeline differently. Training tolerates a moderate amount of random noise, because the model averages over many examples, although systematic errors (for example, one annotator who always writes short responses) are learned as behavior. A test set does not average noise away: it caps the score a perfect model can reach and distorts comparisons between models. If a binary test label is wrong with probability \(e\) independently of the model, a model with true accuracy \(a\) is measured at

\[a_{\text{measured}} = a(1-e) + (1-a)e\]

The first term counts items where the model is right and the label is right. The second counts items where both are wrong, so the model’s wrong answer coincides with the wrong label. The simulation in the first program checks this: with 3% test-label noise, a model that is perfect measures 0.970, and a model with true accuracy 0.90 measures 0.876. With 8% noise, a perfect model measures 0.920. Two consequences follow. Differences between models shrink by the factor \(1 – 2e\), so a noisy test set makes real improvements look smaller. And a score near the noise ceiling stops discriminating: once models are within a few points of \(1 – e\), what separates them may be which mistaken labels they happen to reproduce. Northcutt et al. found label errors in at least 6% of the ImageNet validation set, estimated an average of about 3.3% across ten widely used benchmarks, and showed that at higher noise rates the ranking of models could change. The glossary entry on label noise discusses detection and repair. For evaluation data, the practical rule is to spend annotation budget on the test set before the training set: review the items on which strong models disagree with the label, and fix or remove the ones that are wrong.

Synthetic labels and preference pairs

When a model writes the responses, the quality gates have to do the work that annotator agreement does for people. Filter on checkable properties (format validity, length bounds, presence of required fields, agreement with a reference where one exists), remove near-duplicates, and have a person review a random sample from each slice and each generator configuration. Judge models can screen at scale, but they have their own preferences, such as favoring longer answers or their own style, so they screen candidates and do not replace the human sample. Keep the generator identity and prompt version in the example’s metadata, since a later finding that one generator configuration produced bad data should let you remove exactly that data.

Preference data has two further requirements, following the pairs \((x, y_w, y_l)\) from the fine-tuning article. First, both responses should be plausible candidates drawn from a distribution close to the model being tuned; a pair where one response is obviously broken teaches little. Second, randomize which response appears first and track ties, because annotators and judge models show position effects, and a dataset in which the better answer is usually the longer one teaches the model that length wins.

Leakage and contamination

Leakage is information crossing the boundary that evaluation is supposed to respect. The general case for predictive models, including the preprocessing, temporal, and group routes, is treated in Train, Validation, and Test Sets without Data Leakage. Contamination is the form that matters most for language models: content from an evaluation set (or something equivalent to it) is present in training data. For a fine-tuned model it can enter by three doors, which have different remedies.

• The fine-tuning set. You control this door. Training and evaluation items can be exact duplicates, near-duplicates, or members of the same group (same thread, customer, source document, or template family).
• The generator of synthetic data. If a model was prompted with evaluation items, or with material the evaluation items were derived from, to produce training data, the evaluation content flows into training through the generated examples.
• The base model’s pretraining data. You do not control this door, and often cannot inspect it. A public benchmark may be in the pretraining corpus of the model you start from, in which case a high score might reflect exposure instead of ability. The pretraining side of this problem, including deduplication and decontamination of the corpus, is covered in LLM Pretraining: Objectives, Data, and Scaling Laws.

Fine-tuning adds a fourth route that is not contamination but looks like it: repeated selection against the same test set. If checkpoints, prompts, or hyperparameters are chosen by looking at test results, information from the test set shapes the model even though no test example was ever in a training file. A later section quantifies this.

Forms the overlap takes

Form	Example	What catches it
Exact duplicate	The same ticket text in train and test	Hash of normalized text
Near-duplicate	The same ticket with an added greeting, different whitespace, or a changed order number	Shingle overlap (Jaccard), MinHash
Same group	Different messages in one thread; different questions written from one manual page	Group identifiers; split by group
Template family	Records produced by one script that differ only in slot values	Template extraction; clustering
Paraphrase or translation	The same question reworded, or in another language	Embedding similarity; model-assisted comparison; limited by what they can detect
Derived content	A training answer that quotes the reference answer of a test item	Overlap of outputs, not only inputs
Time	Training on records dated after the evaluation period when the deployment question is about the future	Time-based split

Detecting duplicates and near-duplicates

Detection works from cheap and exact to expensive and fuzzy. Normalize text first (lowercase, collapse whitespace, strip boilerplate such as signatures) and hash it; equal hashes are exact duplicates. For near-duplicates, represent each text as a set of word shingles (overlapping runs of k words) and compare sets by Jaccard similarity, the size of the intersection divided by the size of the union. Jaccard is 1 for identical sets and 0 for disjoint ones, and a threshold chosen per task (0.8 is a common starting point for near-duplicate removal) says how similar is too similar.

Comparing every pair is quadratic in dataset size. MinHash makes it cheap: apply a random hash function to every shingle, and keep the smallest hash value in each set. For two sets \(A\) and \(B\), the minimum over the union is equally likely to be attained by any element of the union, and the two sets share the same minimum exactly when that element lies in \(A \cap B\). Therefore

\[\Pr[\min h(A) = \min h(B)] = \frac{|A \cap B|}{|A \cup B|} = J(A,B)\]

Repeating with many independent hash functions and averaging the matches gives an estimate of the Jaccard similarity from short signatures, and locality-sensitive hashing then groups signatures so only likely pairs are compared. Lee et al. built deduplication tools for pretraining corpora and reported that train–test overlap affected more than 4% of the validation sets of standard datasets they examined, which shows how common the problem is even in curated benchmarks.

The second program does three things on a synthetic corpus of 60 support threads with 5 messages each. It first shows detection on a pair of hand-written messages. It then builds a deliberately memorizing “model”, a one-nearest-neighbor classifier over word overlap, and evaluates it on 20 random splits and 20 group splits. The messages carry a weak category signal (a topic word in about a third of messages) and thread-specific tokens (a name and an order number) that repeat within a thread. The model has no ability beyond lookup, which makes it an instrument for measuring how much a split rewards lookup.

import hashlib, random, re
from collections import Counter

# ---- synthetic ticket threads: each thread has a weak category signal and
# thread-specific tokens (customer name, order id) that repeat across messages
CATS = {"billing": ["charge", "invoice", "card", "refund"],
        "login": ["password", "locked", "reset", "sign"],
        "shipping": ["parcel", "delivery", "late", "tracking"]}
COMMON = ["hello", "please", "help", "thanks", "issue", "again", "need", "with", "my", "the", "about"]
NAMES = ["alva", "borin", "cato", "dara", "elio", "fenn", "gaia", "hale", "iris", "jory"] * 6

def make_corpus(seed=11, threads=60, per_thread=5):
    r = random.Random(seed)
    rows = []
    for t in range(threads):
        cat = r.choice(list(CATS))
        name, order = f"{NAMES[t]}{t}", f"ord{r.randrange(10**5):05d}"
        for m in range(per_thread):
            words = [r.choice(COMMON) for _ in range(6)]
            if r.random() < 0.35:                      # weak, noisy category signal
                words.append(r.choice(CATS[cat]))
            words += [name, order, f"msg{r.randrange(50)}"]
            r.shuffle(words)
            rows.append({"thread": t, "label": cat, "text": " ".join(words)})
    return rows

rows = make_corpus()
print(len(rows), "messages,", len({x["thread"] for x in rows}), "threads")

# ---- 1. exact and near-duplicate detection --------------------------------
def norm(s): return re.sub(r"\s+", " ", s.lower()).strip()
def sha(s): return hashlib.sha256(norm(s).encode()).hexdigest()[:10]
def shingles(s, k=3):
    w = norm(s).split()
    return {tuple(w[i:i + k]) for i in range(max(1, len(w) - k + 1))}
def jaccard(a, b): return len(a & b) / len(a | b)

def minhash(sh, n=128):
    sigs = []
    for i in range(n):
        sigs.append(min(int(hashlib.md5(f"{i}|{x}".encode()).hexdigest()[:8], 16) for x in sh))
    return sigs
def mh_est(a, b): return sum(x == y for x, y in zip(a, b)) / len(a)

a = "please help with my invoice charge again thanks alva0 ord00012"
b = "please help with my invoice charge thanks alva0 ord00012 again"   # reordered
c = "please help with my invoice charge again thanks alva0 ord00012 now"  # one word added
print("same after normalization:", sha(a) == sha(a.upper() + "  "))
for name, x in (("reordered", b), ("one word added", c)):
    sa, sx = shingles(a), shingles(x)
    print(f"{name:15s} exact-hash equal: {sha(a) == sha(x)}  3-gram Jaccard {jaccard(sa, sx):.3f}  "
          f"MinHash estimate {mh_est(minhash(sa), minhash(sx)):.3f}")

# ---- 2. a memorizing "model": 1-nearest neighbour on shingle Jaccard ------
def predict(train, text):
    s = shingles(text, 1)
    return max(train, key=lambda t: jaccard(shingles(t["text"], 1), s))["label"]

def accuracy(train, test):
    return sum(predict(train, x["text"]) == x["label"] for x in test) / len(test)

def random_split(rows, seed, frac=0.3):
    r = random.Random(seed); idx = list(range(len(rows))); r.shuffle(idx)
    k = int(len(rows) * frac)
    return [rows[i] for i in idx[k:]], [rows[i] for i in idx[:k]]

def group_split(rows, seed, frac=0.3):
    r = random.Random(seed); ids = sorted({x["thread"] for x in rows}); r.shuffle(ids)
    held = set(ids[:int(len(ids) * frac)])
    return [x for x in rows if x["thread"] not in held], [x for x in rows if x["thread"] in held]

def overlap(train, test):
    return len({x["thread"] for x in train} & {x["thread"] for x in test})

for name, fn in (("random split", random_split), ("group split ", group_split)):
    accs, ov = [], []
    for seed in range(20):
        tr, te = fn(rows, seed)
        accs.append(accuracy(tr, te)); ov.append(overlap(tr, te))
    print(f"{name}: mean accuracy {sum(accs)/len(accs):.3f}  min {min(accs):.3f}  max {max(accs):.3f}  "
          f"threads shared by train and test (mean) {sum(ov)/len(ov):.1f}")
print("majority-class rate:", round(Counter(x["label"] for x in rows).most_common(1)[0][1] / len(rows), 3))

300 messages, 60 threads
same after normalization: True
reordered       exact-hash equal: False  3-gram Jaccard 0.455  MinHash estimate 0.406
one word added  exact-hash equal: False  3-gram Jaccard 0.889  MinHash estimate 0.844
random split: mean accuracy 0.771  min 0.644  max 0.856  threads shared by train and test (mean) 49.1
group split : mean accuracy 0.333  min 0.189  max 0.400  threads shared by train and test (mean) 0.0
majority-class rate: 0.35

Start with the detection lines. Hashing after normalization catches a trivial variation (case and padding). It does not catch a reordering or an added word. The three-word-shingle Jaccard similarity is 0.889 for the message with one extra word, which a threshold of 0.8 flags. The reordered message scores only 0.455, because reordering breaks most three-word sequences; shingles of single words would score it 1.0. The shingle size is therefore a choice with a trade-off: longer shingles are more specific and miss reorderings, shorter ones catch reorderings and flag more unrelated pairs that share vocabulary. The MinHash estimates (0.844 and 0.406) are close to the exact values but not equal to them, because 128 hash functions give a sampling error of a few points.

The split lines are the main result. On random splits the lookup model scores 0.771 on average, and on group splits 0.333, close to the 0.35 majority-class rate. The model did not change; only the split did. In the random splits, an average of 49.1 of the 60 threads appear on both sides, so most test messages have a sibling in training that shares a name and an order number. In the group splits no thread is shared. A real fine-tuned model will not behave exactly like this nearest-neighbor lookup, but it can use the same shortcut, and the size of the gap between the two splits tells you how much of a score depends on it. The same measurement works as a diagnostic on a real project: evaluate on both splits, and a large gap indicates the dataset contains group structure the model can exploit.

What detection does not establish

Passing a duplicate check does not certify a test set as clean. Yang et al. showed that rephrasing or translating benchmark samples evades n-gram matching and the string-based decontamination used by many projects, so a model could train on rewritten benchmark data and still pass the check; they proposed a stronger decontamination method that uses a language model to find rewritten samples. Embedding similarity (see Embeddings and Retrieval Representations) finds paraphrases that shingles miss, at the cost of false positives that need review, and it is still not complete. The practical posture is layered: exact and near-duplicate removal between train and every evaluation set; group-level splitting so related items are never separated; embedding-based search for the highest-similarity pairs, reviewed by a person; and a recorded statement of what was checked, so a later reader knows what the evaluation number can and cannot mean.

For the base model’s pretraining exposure, the checks are indirect. Compare the score on a public benchmark with the score on a freshly written set of the same kind; a large gap suggests exposure but is not proof, since the new items may be easier or harder. Prefer evaluation sets that you wrote from private material or that postdate the base model’s training data, and treat public benchmark scores as a reference point, not an unbiased estimate.

Dataset rights

Rights questions are decided by lawyers and organizational policy, and this article does not give legal advice. What an engineer can supply is the record that lets those decisions be made and enforced: for each source, what is known about where it came from and what it may be used for. The reason to do this before training and not after is that a trained model is hard to edit. Carlini et al. showed that an attacker could extract hundreds of verbatim training sequences from GPT-2, including names, phone numbers, and email addresses, and that larger models were more vulnerable; data that should not have been used cannot be reliably removed from the weights by deleting the file. The only dependable control is the gate before training, plus enough lineage to retrain without the data.

Several different kinds of permission apply to one example, and they can disagree.

• Copyright and license of the content. A license such as CC-BY allows use under conditions (attribution, for example); a share-alike license adds conditions on derivatives, and how those apply to a trained model is a question for counsel. Content with no license is not free to use because it is public. The glossary entry on open licenses introduces the common ones.
• Terms of the source. Websites, APIs, and data providers set terms that can restrict scraping, redistribution, or training use, separately from copyright. Terms of model providers can restrict using their outputs to train other models; these terms differ by provider and change over time, so record the version that applied when the data was generated.
• Personal data. Support tickets, chat logs, and forms contain identifiers and sensitive details. Whether you may use them for training depends on the notice and consent under which they were collected and on the applicable law. Scrubbing helps but is imperfect, and Carlini et al. show why leftover identifiers matter. See data privacy for the underlying concepts.
• Customer and contractual restrictions. A customer contract may allow using their data to deliver the service but not to train a model that other customers use.
• Annotation work. Contracts with annotators and vendors decide who owns the labels and whether they may be used for training and redistribution.
• Evaluation data. The same questions apply to test sets, and also to publishing them: a test set derived from restricted material cannot be shared with outside reviewers.

Documentation about licenses is itself unreliable. Longpre et al., in an audit of more than 1,800 text datasets, reported license omission rates above 70% and error rates above 50% in the license information on widely used dataset-hosting platforms. A license field copied from a hub is a lead for investigation, and you should follow it back to the original source and terms. Gebru et al. propose documenting each dataset with a datasheet covering its motivation, composition, collection process, and recommended uses, and the manifest below is a minimal operational version of that idea.

The third program implements a rights manifest and a gate. The scenario is hypothetical: four sources with different permissions, and a pool that also contains items from a source with no manifest entry. In the manifest, the support tickets are allowed for evaluation only (imagine a contract that permits quality review of the data but not training), outputs from a vendor model have no recorded usable terms, and the other two sources are cleared for both uses. The gate is a pure function of the manifest, so it fails closed: an example from an unknown source, or from a source whose record lacks any required field, is dropped.

import hashlib
from collections import Counter

# ---- rights manifest: one record per source ---------------------------------
MANIFEST = {
    "support_tickets_2023": {"license": "internal-customer-data", "terms_version": "msa-2023-04",
        "allowed": {"eval"}, "pii": "scrubbed", "owner": "support-ops"},
    "public_faq_pages": {"license": "CC-BY-4.0", "terms_version": "site-2024-01",
        "allowed": {"train", "eval"}, "pii": "none", "owner": "docs"},
    "vendor_model_outputs": {"license": "provider-terms", "terms_version": "unrecorded",
        "allowed": set(), "pii": "none", "owner": "ml"},
    "annotator_written": {"license": "work-for-hire", "terms_version": "contract-2024-02",
        "allowed": {"train", "eval"}, "pii": "none", "owner": "ml"},
}
REQUIRED = ("license", "terms_version", "allowed", "pii", "owner")

def gate(examples, use):
    kept, dropped = [], Counter()
    for ex in examples:
        rec = MANIFEST.get(ex["source"])
        if rec is None:
            dropped[(ex["source"], "no manifest entry")] += 1
        elif any(k not in rec for k in REQUIRED):
            dropped[(ex["source"], "incomplete manifest")] += 1
        elif use not in rec["allowed"]:
            dropped[(ex["source"], f"'{use}' not allowed")] += 1
        elif rec["pii"] not in ("none", "scrubbed"):
            dropped[(ex["source"], "pii unresolved")] += 1
        else:
            kept.append(ex)
    return kept, dropped

pool = ([{"source": "support_tickets_2023", "id": i} for i in range(40)] +
        [{"source": "public_faq_pages", "id": i} for i in range(30)] +
        [{"source": "vendor_model_outputs", "id": i} for i in range(20)] +
        [{"source": "annotator_written", "id": i} for i in range(10)] +
        [{"source": "scraped_forum", "id": i} for i in range(5)])
for use in ("train", "eval"):
    kept, dropped = gate(pool, use)
    print(use, "kept", len(kept), dict(Counter(e["source"] for e in kept)))
    for (src, why), n in sorted(dropped.items()):
        print("   dropped", n, src, "-", why)

train kept 40 {'public_faq_pages': 30, 'annotator_written': 10}
   dropped 5 scraped_forum - no manifest entry
   dropped 40 support_tickets_2023 - 'train' not allowed
   dropped 20 vendor_model_outputs - 'train' not allowed
eval kept 80 {'support_tickets_2023': 40, 'public_faq_pages': 30, 'annotator_written': 10}
   dropped 5 scraped_forum - no manifest entry
   dropped 20 vendor_model_outputs - 'eval' not allowed

Read the output by use. For training, only the 30 public FAQ examples and the 10 annotator-written examples survive. The 40 support tickets are dropped because training is not an allowed use, the 20 vendor-model outputs because their permissions are not established, and the 5 forum items because no one recorded where they came from. For evaluation, the tickets come back (80 examples kept), which is the intended asymmetry in this scenario. Two properties matter more than the specific rules: every dropped example has a stated reason that can be counted and reviewed, and an unknown source is excluded by default instead of included by default. Rights also have to travel with the data: the manifest record should be copied or referenced from each example’s metadata so that, if a source is withdrawn, the affected examples and the training runs that used them can be found.

Evaluation splits

With a task contract, sampled data, trusted labels, and cleared rights, the remaining design question is how to divide the data. Each portion has a purpose, and mixing purposes is the root of most evaluation errors. The glossary entry on evaluation sets gives the short version of the roles; for a fine-tuning project they are as follows.

Portion	Purpose	Rule of use
Training	Fit the adapter or weights	Never contains an evaluation item or a group member of one
Validation (development)	Choose learning rate, epochs, checkpoint, prompt template, data mixture	Used repeatedly; its scores become optimistic as choices accumulate
Test	Report the final estimate of performance on the deployment question	Used rarely and for decisions made after configuration is frozen
Retention set	Detect loss of capabilities the tuning was not meant to change	Drawn from the tasks that must still work; has no overlap with training
Regression canaries	A small set of known-important cases checked on every candidate	Fixed cases whose failure blocks release regardless of aggregate scores

Choose the unit of the split from the deployment question

The unit that gets divided between portions, whether single examples, groups, or time periods, should follow from what the model will face. Ask: when the model is used, what will be new relative to the training data? If the answer is “new customers”, split by customer. If it is “new documents”, split by document. If it is “next month’s tickets”, split by time, with training data before a cutoff and evaluation data after it. If it is “new phrasings of familiar requests”, an example-level split is defensible, provided near-duplicates are removed. The second program’s gap between random and group splits is the cost of ignoring this question.

Deployment question	Split unit	Also required
Will it work for customers it has never seen?	Customer or thread	Cap examples per customer so groups stay balanced
Will it answer new questions about documents it was never shown?	Source document	Questions written from one document stay together
Will it keep working as the product and language change?	Time (train before cutoff, evaluate after)	A gap between periods where outcomes depend on later information
Will it handle new phrasings of known tasks?	Example, after near-duplicate removal	Template-family checks, so surface variants do not slip across
Does it still do what it did before tuning?	Separate retention set	Evaluate the base model on it too, so the change is measured

Real deployments combine questions, so splits combine units: for instance, hold out whole customers and also require that all evaluation tickets postdate all training tickets. The more constraints, the less data remains for each portion, which is a real cost and an argument for collecting evaluation data on purpose instead of taking whatever is left over. Time-based splits matter beyond leakage: they make the evaluation answer the question the team will actually ask in production, and they reveal drift that a random split hides. The article on monitoring, drift, and retraining follows what happens to a model after deployment.

Make the assignment deterministic

Data accumulates, and a split that is re-randomized each time makes results incomparable and lets earlier test items drift into training. A stable alternative is to assign each group by hashing its identifier with a fixed salt: the hash value modulo 100 gives a bucket, and a bucket range maps to a portion. The same group always lands in the same portion, new groups are assigned without disturbing old ones, and nobody needs to store a list of assignments to reproduce them. The fourth program implements this and an audit that checks every group appears in exactly one portion.

import hashlib
from collections import Counter

# ---- deterministic group-hash split ---------------------------------------
def bucket(group, salt="split-v1"):
    h = int(hashlib.sha256(f"{salt}|{group}".encode()).hexdigest()[:8], 16)
    return h % 100

def assign(group):
    b = bucket(group)
    return "test" if b < 15 else "val" if b < 30 else "train"

groups_v1 = [f"thread-{i}" for i in range(2000)]
groups_v2 = groups_v1 + [f"thread-{i}" for i in range(2000, 3000)]   # data added later
a1 = {g: assign(g) for g in groups_v1}
a2 = {g: assign(g) for g in groups_v2}
print("split sizes v1:", dict(Counter(a1.values())))
print("split sizes v2:", dict(Counter(a2.values())))
print("old groups whose split changed after adding data:", sum(a1[g] != a2[g] for g in groups_v1))

# ---- split audit -----------------------------------------------------------
def audit(rows, split_of):
    seen = {}
    problems = []
    for r in rows:
        s = split_of(r)
        prev = seen.setdefault(r["group"], s)
        if prev != s:
            problems.append(r["group"])
    return sorted(set(problems))

rows = [{"group": f"thread-{i % 400}", "split": assign(f"thread-{i % 400}")} for i in range(1200)]
print("audit, clean assignment:", audit(rows, lambda r: r["split"]))
rows[5]["split"] = "test" if rows[5]["split"] != "test" else "train"   # a manual edit slips in
print("audit, one edited row:  ", audit(rows, lambda r: r["split"]))

split sizes v1: {'val': 264, 'train': 1409, 'test': 327}
split sizes v2: {'val': 412, 'train': 2099, 'test': 489}
old groups whose split changed after adding data: 0
audit, clean assignment: []
audit, one edited row:   ['thread-5']

The first split-size lines show the proportions (about 16% test and 13% validation in the first version; the hash is uniform, so realized sizes differ from the nominal ones by sampling variation). The line that matters is the zero: after 1,000 new groups were added, none of the 2,000 original groups changed portion. The audit demonstrates the check that should run before every training job. With the assignments intact it returns no violations; after one row is edited by hand, it reports the thread whose rows now straddle two portions. The same audit can be extended to compare near-duplicate clusters across portions, using the detection from the previous section. Keep the salt fixed for the life of a test set; a different salt creates a different split.

How many evaluation items are enough

An accuracy measured on \(n\) independent items has a standard error of \(\sqrt{p(1-p)/n}\), and an approximate 95% interval is \(p \pm 1.96\) times that. At \(p = 0.8\) the half-width is 0.111 for 50 items, 0.055 for 200, 0.025 for 1,000, and 0.011 for 5,000, as the first lines of the next program show. Half-width falls with the square root of \(n\), so halving the interval costs four times as many items. Card et al. report that underpowered experiments are common in NLP evaluations, so a test set can be too small to resolve the difference being claimed. The article on statistical inference for machine learning develops standard errors and intervals from first principles.

Comparing two models on the same items needs a different calculation, because the models’ results are correlated: easy items are easy for both. What matters is the items on which they disagree. Count the items the tuned model fixed (base wrong, tuned right) and the items it broke (base right, tuned wrong). The accuracy difference is the difference of those counts divided by \(n\). A paired bootstrap (resample the items with replacement, recompute the difference on each resample, and take the 2.5th and 97.5th percentiles; see the glossary entry on the bootstrap) gives an interval that reflects this pairing. In the program below, the base model scores 0.69 and the tuned model 0.76 on 300 synthetic items, with 27 items fixed and 6 broken. The paired interval for the difference is [0.033, 0.107], a half-width of 0.037, whereas treating the two accuracies as independent samples would give a half-width of 0.071, almost twice as wide. Pairing is free information: use it.

The same disagreement counts give a planning formula. With a discordant rate \(d_r\) (the fraction of items on which the two models differ in correctness) and a true difference \(\delta\), a normal-approximation sample size for 80% power at a two-sided 5% level is

\[n \approx \frac{(z_{0.975} + z_{0.80})^2\,(d_r – \delta^2)}{\delta^2} \approx \frac{7.84\,(d_r – \delta^2)}{\delta^2}\]

Here \(z_{0.975} = 1.96\) and \(z_{0.80} = 0.84\) are normal quantiles for the significance level and the power. With a discordant rate of 0.11, as in the example, detecting a 3-point improvement needs about \(7.84 \times (0.11 – 0.0009)/0.0009 \approx 950\) items, and detecting a 7-point improvement needs about 170. This is an approximation that assumes independent items, and these two sizes come from the formula, not from the program below. Apply it before collecting labels, because the answer decides the annotation budget.

Two complications arise in practice. First, items are often not independent. Messages in one thread, questions about one document, and variants of one template share difficulty, and the effective number of independent items is smaller than the count of rows. A rough guide is the design effect: with clusters of size \(m\) and within-cluster correlation \(\rho\), the effective sample size is about \(n / (1 + (m-1)\rho)\), so 300 items in clusters of 10 with \(\rho = 0.3\) behave like roughly 81 independent ones. The cluster bootstrap resamples whole groups instead of rows and accounts for this automatically. The program applies it to a synthetic set of 30 threads of 10 items each: for an accuracy of 0.570, the row-level bootstrap interval is [0.517, 0.627] and the thread-level interval is [0.493, 0.650], about 43% wider.

Second, aggregate precision says nothing about slices. A slice with 12 items at accuracy 0.8 has a half-width of 0.226, which cannot distinguish 0.6 from 0.95. Decide which slices must be reported, size them for the precision the decision requires, and say so when a slice is too small to support a conclusion. The slice matrix from the beginning of the article is the list.

import math, random
from statistics import mean

def wald(p, n): return 1.96 * math.sqrt(p * (1 - p) / n)
for n in (50, 200, 1000, 5000):
    print(f"n={n:5d}  p=0.80  95% half-width {wald(0.8, n):.3f}")

# ---- paired comparison on the same items ------------------------------------
rng = random.Random(5)
n = 300
base = [rng.random() < 0.70 for _ in range(n)]
# tuned model fixes some of the base model's failures and breaks a few passes
tuned = [(b or rng.random() < 0.20) and not (b and rng.random() < 0.04) for b in base]
print("base acc", mean(base), "tuned acc", round(mean(tuned), 3))
b01 = sum((not b) and t for b, t in zip(base, tuned))   # fixed
b10 = sum(b and (not t) for b, t in zip(base, tuned))   # broken
print("items fixed", b01, "items broken", b10)

def boot_diff(base, tuned, groups=None, reps=4000, seed=1):
    r = random.Random(seed); idx = list(range(len(base))); out = []
    if groups is None:
        for _ in range(reps):
            s = [r.choice(idx) for _ in idx]
            out.append(mean(tuned[i] for i in s) - mean(base[i] for i in s))
    else:
        gids = sorted(set(groups)); members = {g: [i for i in idx if groups[i] == g] for g in gids}
        for _ in range(reps):
            s = [i for g in (r.choice(gids) for _ in gids) for i in members[g]]
            out.append(mean(tuned[i] for i in s) - mean(base[i] for i in s))
    out.sort()
    return out[int(0.025 * reps)], out[int(0.975 * reps)]

lo, hi = boot_diff(base, tuned)
print(f"paired bootstrap 95% interval for the accuracy difference: [{lo:.3f}, {hi:.3f}]")

# unpaired normal approximation ignores that both models answer the same items
pa, pb = mean(base), mean(tuned)
se = math.sqrt(pa * (1 - pa) / n + pb * (1 - pb) / n)
print(f"unpaired half-width {1.96 * se:.3f}  (paired half-width {(hi - lo) / 2:.3f})")

# ---- clustered items --------------------------------------------------------
r = random.Random(9)
groups, cbase, ctuned = [], [], []
for g in range(30):                       # 30 threads x 10 items
    ease = r.random()                     # shared difficulty inside a thread
    for _ in range(10):
        groups.append(g)
        b = r.random() < 0.3 + 0.5 * ease
        cbase.append(b); ctuned.append(b or r.random() < 0.08)
zeros = [False] * len(cbase)               # boot_diff with an all-wrong "base" gives the tuned accuracy
lo1, hi1 = boot_diff(zeros, ctuned, None)
lo2, hi2 = boot_diff(zeros, ctuned, groups)
print(f"tuned accuracy {mean(ctuned):.3f} on 300 items from 30 threads")
print(f"item bootstrap [{lo1:.3f}, {hi1:.3f}]  thread bootstrap [{lo2:.3f}, {hi2:.3f}]")

# ---- slices -----------------------------------------------------------------
for k in (12, 40, 150):
    print(f"slice with {k:3d} items at p=0.80: half-width {wald(0.8, k):.3f}")

# ---- selecting the best of k candidates on one validation set ----------------
def selection_bias(k, n_val=100, trials=400, p=0.75, seed=2):
    r = random.Random(seed); gaps = []
    for _ in range(trials):
        val_scores = [sum(r.random() < p for _ in range(n_val)) / n_val for _ in range(k)]
        gaps.append(max(val_scores) - p)   # every candidate truly has accuracy p
    return mean(gaps)
for k in (1, 5, 20, 100):
    print(f"best of {k:3d} candidates, all truly 0.75: validation score exceeds truth by {selection_bias(k):.3f}")

n=   50  p=0.80  95% half-width 0.111
n=  200  p=0.80  95% half-width 0.055
n= 1000  p=0.80  95% half-width 0.025
n= 5000  p=0.80  95% half-width 0.011
base acc 0.69 tuned acc 0.76
items fixed 27 items broken 6
paired bootstrap 95% interval for the accuracy difference: [0.033, 0.107]
unpaired half-width 0.071  (paired half-width 0.037)
tuned accuracy 0.570 on 300 items from 30 threads
item bootstrap [0.517, 0.627]  thread bootstrap [0.493, 0.650]
slice with  12 items at p=0.80: half-width 0.226
slice with  40 items at p=0.80: half-width 0.124
slice with 150 items at p=0.80: half-width 0.064
best of   1 candidates, all truly 0.75: validation score exceeds truth by 0.000
best of   5 candidates, all truly 0.75: validation score exceeds truth by 0.050
best of  20 candidates, all truly 0.75: validation score exceeds truth by 0.079
best of 100 candidates, all truly 0.75: validation score exceeds truth by 0.103

The first block repeats the interval half-widths. The paired comparison and the cluster comparison follow, with the numbers discussed above. The slice lines give the half-widths for 12, 40, and 150 items. The last block quantifies selection on a validation set, which is the subject of the next subsection.

Selection on the validation set

Every choice made by looking at a score spends some of the data’s independence. The simplest case is picking the best of \(k\) candidates (checkpoints, hyperparameter settings, prompt templates) by validation score. The last block of the program simulates \(k\) candidates that all have true accuracy 0.75 on a validation set of 100 items. The average winning score exceeds the truth by 0.050 for 5 candidates, 0.079 for 20, and 0.103 for 100. Nothing is wrong with any candidate, and the winner is simply the one that was lucky on those items. This is why the test set is separate: the test score of the chosen configuration is not inflated by selection, and the validation score is. It is also why a test set loses its value when it is used for selection, even informally. Looking at test results to decide what to try next puts the test set in the validation role. If that happens, create a new test set from data that has not been used.

Two further practices keep a test set useful. Evaluate the final candidates on it once, and record that. And refresh it over time: an evaluation set that stays fixed while models are built against it becomes less informative, both because teams adapt to it and because later models or data may contain it. A second set built from newer material provides a check on whether gains on the old one were specific to it.

Measure what must not change

Fine-tuning can move capabilities that were not in the training data. The retention set and the base-model comparison exist to measure that: evaluate the base model and the tuned model on the same retention items and report the paired difference, as above. The design of that comparison, along with baselines, ablations, overfitting, and forgetting analysis, is the subject of Fine-Tuning Experiments: Baselines, Ablations, and Failure Analysis, which uses the splits defined here. For open-ended outputs, define the rubric or programmatic check at the same time as the test set, and keep the grader fixed across models; changing the judge between runs changes the measurement. The principles for application-level test sets that outlive a single model are developed in Evaluating AI Applications with Reproducible Test Sets.

Order of operations

The checks above depend on each other, and doing them in the wrong order reintroduces the problems. Deduplication has to precede splitting, because the splitter needs to know which examples are related. Rights filtering has to precede everything that copies data into new stores. Decontamination runs after the split, because it compares training against the evaluation sets that exist.

sources + task contract
        |
        v
rights gate (manifest: license, terms version, allowed uses, PII status)
        |
        v
normalize, exact hash, near-duplicate clustering   -> cluster / group ids
        |
        v
assign group -> train | validation | test | retention  (hash of group id + salt)
        |
        v
decontaminate: remove training items that match any evaluation item
        |
        v
label review: agreement on a sample, test-label review first
        |
        v
slice report: counts per slice and portion, interval widths, sampling record
        |
        v
audit: groups in one portion only; no near-duplicates across portions

Label review is placed late because it is expensive and should apply to the final membership of each portion. The audit runs at the end and again before every training job, since data changes and so does the split. The artifacts the pipeline leaves behind (manifest, group assignments, salt, thresholds, decontamination report, slice counts) make an evaluation number reproducible and tell a later reader what it was measuring.

References

• Zhou et al. (2023). LIMA: Less Is More for Alignment. NeurIPS.
• Wang et al. (2023). Self-Instruct: Aligning Language Models with Self-Generated Instructions. ACL.
• Gekhman et al. (2024). Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?. EMNLP.
• Northcutt, Athalye, and Mueller (2021). Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. NeurIPS Datasets and Benchmarks.
• Lee et al. (2022). Deduplicating Training Data Makes Language Models Better. ACL.
• Yang et al. (2023). Rethinking Benchmark and Contamination for Language Models with Rephrased Samples. arXiv preprint.
• Carlini et al. (2021). Extracting Training Data from Large Language Models. USENIX Security.
• Gebru et al. (2021). Datasheets for Datasets. Communications of the ACM.
• Longpre et al. (2023). The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing and Attribution in AI. arXiv preprint.
• Card et al. (2020). With Little Power Comes Great Responsibility. EMNLP.