Test Fixture

A test fixture supplies the controlled starting state for a test. It can include input rows, explicit schema, session settings, temporary files, and setup or cleanup of dependencies.

Choose small inputs that distinguish the rule being tested: zero from missing, a duplicate from a unique row, or an instant on a time boundary from one outside it. An explicit schema supports empty or all-null input without relying on inference.

A fixed random seed does not make every distributed execution deterministic. Record relevant versions, settings, clocks, and source snapshots. Avoid shared mutable state that lets one test change another test’s starting conditions.

A fixture is input, not the expected answer. Keep the expected result grounded in the rule, and verify that a nearby wrong implementation fails. Release temporary resources, while retaining sanitized failure evidence when necessary.

See Testing and Debugging Spark Data Pipelines for worked examples.

Reference: Official documentation.