LLMOps

LLMOps means the operational work around large-language-model applications. The term overlaps MLOps and software operations; its boundary is not a single universal standard. A team can practice LLMOps while using an external model and doing no model training itself.

A prompt change is an application change

A returns assistant may combine a model, instructions, a policy search index, and an order-service tool. Updating the policy index changes the evidence available to the assistant even when its model identifier stays the same. Record these components as a compatible application release so a failure can be investigated.

Evaluate the task: did the answer preserve policy conditions, cite relevant support, and describe the actual tool result? Also examine latency, failures of dependencies, and cost. A model-based judge can assist review, but its scores need checking against the intended criteria and human judgments.

Follow the request through operation

A useful trace connects retrieval, generation, tool requests, and observed outcomes. Collect only permitted data and apply access and retention controls.

If answers deteriorate, inspect the changed component before choosing a remedy. Reverting a model cannot repair a missing policy passage, and changing a prompt does not authorize an action that the application should forbid.

Reference: MLflow: LLM and Agent Evaluation.