Configuration Validation
Configuration validation checks whether supplied settings satisfy an explicit contract before code relies on them. It can reject missing keys, misspelled keys, invalid types, values outside a range, or inconsistent settings.
For a minimum amount, define the unit and allowed range. In Python, a dataclass annotation alone does not reject a string, and bool is a subclass of int. If true is not an allowed amount, test for that explicitly. Decide whether unknown keys are errors rather than silently ignoring them.
A syntactically valid destination can still be the wrong environment. Validate destination identity and permissions separately, and keep credentials out of logged settings. Capture the effective configuration version, including defaults, so the executed rule can be identified.
Changing a threshold changes business behavior. Validation establishes that the setting meets declared constraints, not that the chosen threshold is correct. Review rule changes with representative expected outputs.
Apply validation after resolving the precedence of command arguments, environment variables, files, and defaults. For a time interval, valid timestamps are insufficient if the end precedes the start. For a threshold, distinguish zero from a missing setting. Test these relationships before opening the source; record approved effective values so an operator can reproduce the choice without exposing secrets.
See Designing Maintainable Data Workflows for worked examples.
Further reading
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
