Feedback Loop (Machine Learning)

Google’s Machine Learning Glossary describes a feedback loop as predictions influencing the data used to train the same or another model. A recommender changes which films people see, and those viewing records train its successor. Whether this is harmful depends on the target: predicting behavior under the current recommendation policy differs from estimating what people would choose under another policy.

Three common forms

Only selected outcomes are observed. A lender learns repayment for approved loans, not for loans it did not make. A fraud system may obtain investigation labels mainly for flagged transactions, while chargebacks or other independent reports supply some additional outcomes. Without such coverage or deliberate exploration, evaluation on observed labels may not represent excluded cases. The severity of the problem depends on selection and the population the model is meant to serve.

Exposure changes behavior. An item’s position affects its chance of being clicked. Google’s Rules of Machine Learning, Rule 36, suggests learning a positional component separately from other features, ideally as an additive function without position-document cross-features. At serving, candidates receive a common default position value before ranking. That structure matters: simply removing a feature from an arbitrary model does not isolate relevance. This modeling choice also does not by itself resolve which items were exposed or guarantee identification of causal item effects.

The intervention changes the outcome. A retention offer can cause a customer to stay. If the training target is risk without an offer but the model ignores treatment history, learning from those outcomes can make it underestimate untreated risk. If the target is instead the outcome under a specified offer policy, policy-shaped labels may be appropriate. The decision policy, prediction target, and available comparison must be made explicit before calling the labels biased.

Why it is hard to see

Feedback can mislead both training and evaluation when the recorded outcomes do not answer the intended question. A successor may predict its predecessor’s policy outcomes well without making better decisions under a new policy. Input-only drift checks can miss this when the served population remains stable. Prediction distributions may still shift as the model changes; neither a stable input distribution nor a changed prediction distribution establishes the policy’s benefit. Evaluate the intended target and relevant coverage directly.

Keeping an honest sample

One way to obtain otherwise missing outcomes is an authorized random exploration policy. Approving or investigating a random share of borderline cases adds information about that region, not every excluded case. Record eligibility, actions, assignment probabilities, and outcome availability. Weighting requires positive observation or action probabilities in the target population and justified assignment and missingness assumptions, such as random assignment or adequate adjustment for confounding. It cannot recover outcomes in a region with zero probability of observation. A holdback group supplies a comparison without a specified intervention, subject to history and spillovers. Dropping affected cases is not a general correction either: it changes the population. The scope and cost of each design need explicit ownership.

A feedback loop differs from label delay: delay concerns when an outcome can be confirmed, while a policy can change the outcome itself or whether it is ever observed. Delay, censoring, and selective observation can coexist. How offline metrics can depart from what a deployed policy achieves, including position effects, is worked through in Offline Metrics and Online Outcomes.

References

• Google for Developers, Machine Learning Glossary (Creative Commons Attribution 4.0) — definition of feedback loop.
• Google for Developers, Rules of Machine Learning (Creative Commons Attribution 4.0) — Rule 36 on positional features.