Schema Variant Column

A schema variant column preserves a value that does not fit the inferred type of an existing column. In dlt, some type conflicts create a separate column under its schema evolution rules. Other tools or configured schema contracts may coerce, reject, or discard values instead; variants are not a universal connector feature.

Suppose amount_cents was an integer and a source sends the text “42.00”. A variant such as amount_cents__v_text can retain that text while the original numeric column is null for that row. A query summing only amount_cents then omits the affected value even though the load completed.

Inspect the source change before combining columns. The text might mean 42 cents or 42 dollars. Casting it to a number resolves neither the unit nor the rounding rule. Agree a canonical representation, preserve source evidence, and validate totals and affected row counts when repairing historical data.

Monitor new columns and null-rate changes as well as job status. Decide whether unexpected variants should be accepted, quarantined, or block publication. Variant preservation helps retain evidence, but it does not establish source completeness or business correctness.

See Managed Ingestion for the worked examples. Reference: dlt schema evolution.