Experiment Tracking

Experiment tracking records what was run and what resulted. A run is one execution, such as training a forecast model or evaluating a prompt configuration. Its record can connect code and data references, settings, measured results, and artifacts such as model files or reports.

Two scores need a common basis

Imagine run A reports an average error of 12 orders and run B reports 9. That comparison is incomplete if A used a holiday week and B used an ordinary week. Record evaluation periods, cases, metric definitions, and units so the team can compare candidates on an appropriate common basis.

An experiment groups related runs. Tracking software makes them searchable, but it does not decide whether the comparison is fair or the lower score is useful for the business task.

A record supports reproduction; it does not guarantee it

A random seed cannot restore a deleted dataset or recreate an external model that has changed. Preserve required versions and environment information where possible, and state replay limits where they remain.

Avoid logging secrets or unrestricted copies of private inputs. Useful tracking retains enough authorized evidence to explain a result. Promotion approval and the version actually deployed remain separate records.

Reference: MLflow: Experiment Tracking.