Schema Evolution and Compatibility

Schema evolution is changing a data structure while managing records and software that still use older versions. Compatibility asks which reader can interpret which writer’s data. Backward compatibility lets a new reader read older data; forward compatibility lets an older reader read newer data. State the versions and format being compared.

Adding an optional coupon field is not automatically safe. An old strict reader may reject unknown properties; a new reader needs a rule when older records lack the field. Alias and default behavior differ by format. Testing only the immediately preceding version does not establish compatibility with every retained historical version.

A field can remain an integer while its unit changes from dollars to cents. Structural validation may pass while metrics become wrong. Record semantic changes, preserve source versions, and normalize only when a valid mapping exists. Register and test a migration with affected consumers before retiring old producers.

Reference: Confluent compatibility directions.