Adoption and Benefits Realization

A fictional insurer’s new claims system went live on schedule. At the twelve-month review, roughly a third of branches used it for their routine claims workflow, a third combined new intake with old assessment screens, and a third relied mainly on spreadsheets before a clerk entered the results into a formal system. The old screens remained available in all branches. A separate case-level audit classified completed claims in the review period: 34 percent followed the new workflow end to end, 31 percent used a mixed workflow, and 35 percent were processed mainly in spreadsheets and then re-entered. Those transaction shares were measured separately from branch coverage; branches handled different volumes. Average active handling time was thirty minutes, compared with a pre-launch baseline of twenty-six. Observation found double entry among the sources of extra work. Finance had reduced the operations budget from month one using the forecast saving, before checking whether the work or expenditure had fallen. Staff had received a two-hour webinar six weeks before launch, with a telephony rollout and regional reorganization also demanding their time.

The case concerns a program that promised benefits from changed work. Migration, modernization, and transformation have different acceptance criteria. A migration can meet its agreed behavior, data, access, performance, and recovery requirements without improving a business process. Modernization can resolve a support or security need while preserving that process. A transformation claiming faster claims handling must establish what changed for staff and customers and whether the promised outcome followed. Cartwright, Horn, and Lewis emphasize organizational conditions in their legacy-displacement experience. Their description of technology as at most half the problem is an expression of that experience, not a measured allocation of effort or a rule for dividing a budget. Here the focus is adoption, work redesign, capabilities, competing demands, and the evidence used to accept or revise benefit claims.

Adoption is a measurement, not an assumption

The GOV.UK Service Manual provides a useful example of a defined adoption measure. Digital take-up divides completed digital transactions, including assisted digital transactions, by transactions across all channels in the same period. This transaction ratio is not a count of distinct people. It also does not say which internal system processed every subsequent step: a customer may complete an online submission while an assessor later uses an old application. For this insurer, end-to-end use of the new claims workflow is a separate operational measure. Define its eligible cases, completion event, observation period, and treatment of mixed paths, retries, and unknown records before comparing units. Partial use can still be measured at the relevant stage.

In the Service Manual’s setting, teams model channel use and support needs during discovery and alpha, establish an available baseline, and refine their model with beta data. Digital take-up is measured from public beta and reported monthly; the live channel-shift plan is reviewed every six months or annually. These are different schedules. An internal migration can adopt the same discipline of explicit definitions and continuing review, while setting its own measures, support arrangements, and decision cadence.

Layer	Question	Measure	The insurer at twelve months
Access and readiness	Can intended staff use the supported workflow?	Access coverage and demonstrated task readiness, reported separately	All had accounts; the webinar record did not establish readiness. Old screens remained available
Use	Which workflow completes the work?	Completed eligible claims classified by actual processing path, by unit and period	34 percent new end to end; 31 percent mixed; 35 percent mainly spreadsheets followed by formal entry
Process adherence	Does execution match the approved design?	Observed steps, controls, exceptions, and workarounds	Duplicate entry and spreadsheet processing; investigate whether the design or its execution needs correction
Effect	Are the intended outcomes observed?	Comparable active handling time, accuracy, rework, service time, and cost	Thirty active minutes versus a twenty-six-minute baseline; the target was fifteen

These are diagnostic views, not a causal ladder. Access and correct use can coexist with no benefit if the value hypothesis is wrong, the technology performs poorly, or the work has changed. Conversely, a useful partial deployment can improve a bounded task before full migration. Data product adoption likewise examines verified consumers and uses rather than product inventory. A Mandate Is a Measurement explains why required use needs task-success and support evidence as well as coverage: a mandate may establish a control, but it does not by itself establish a good experience. Define the capability and audience being measured instead of treating all use as one score.

The work has to be redesigned, and someone has to own that

In this case, double entry followed the inherited process rather than a refusal to learn. The legacy-displacement authors describe business processes becoming coupled to system constraints and to workarounds outside the software. A replacement can preserve those problems unless the work itself is examined. GOV.UK’s legacy guidance similarly asks teams to consider business processes, culture, responsibilities, and skills alongside technology. It recommends involving users and senior management, explaining the proposed benefits and disruption, and budgeting for research, testing, training, and communication. These activities provide evidence about the proposed change; they do not guarantee that every inconvenience will produce a later gain.

Give work redesign an accountable owner with authority over the affected process, resources, and controls, and responsibility that survives the program. That person may participate in the technology program; organizational location is less important than decision rights. For each role, document the steps and handoffs that disappear or change, the decisions and information involved, and the approvals and reports that must remain valid. Test the proposed workflow with its users. Set a target retirement date for an unnecessary old path, but approve closure only after required functionality, access, accessibility, exception handling, support, dependencies, record retention, and recovery arrangements are verified. If a unit is not ready, choose an authorized remedy, delay, or bounded exception. An intentional parallel path or assisted channel can remain legitimate.

People and skills are part of the target state

The target operating model needs people able to use, support, and change the system. The insurer had planned a webinar for branch staff but no adequate transfer from its two contractors to the operations team. GOV.UK’s legacy guidance stresses the ability to maintain services, work effectively with suppliers, and bridge old and new technologies; it also identifies unclear responsibilities and insufficient capacity as blockers. In this plan, learning and supervised practice therefore need funded time, tools, and cover for ordinary work. A document handover can support that learning, but the receiving team should demonstrate the tasks it is expected to perform.

Group	What changes for them	What the plan has to provide	The insurer
People doing the changed work	New steps, new decisions, new tools, for a period both systems at once	Training close to the date, on the redesigned process rather than the screens; time in the schedule; floor support in the first weeks; a channel for problems that is answered	A webinar six weeks early, on the screens; no support after week one
People who support and operate the new system	A system, a platform, and a vendor relationship they did not build	Knowledge transfer with a date and a test, not a handover document; on-call and incident practice before go-live; named ownership	Two contractors leaving at go-live; ownership unassigned for four months
People whose role shrinks or ends	Work may disappear, move, or be replaced by different work	Assess redeployment, retraining, or exit; follow applicable approval and consultation requirements and agree timely staff communication	The budget was reduced before verified workload or expenditure reductions; remaining staff still handled duplicate work
Managers of all three	New measures, new exceptions, a period of lower throughput	The measures they will be judged on during the transition, agreed in advance and different from steady state	Judged on handling time from month one

Several changes at once land on the same people

The insurer’s programs shared the same staff but planned their demands separately. That overlap needs investigation in its own right. DORA’s 2024 research concerns unstable organizational priorities and reports relationships with productivity and burnout among technical and adjacent roles. The researchers propose uncertainty, reduced control, and workload as possible explanations. Simultaneous programs are not the same variable: several well-coordinated changes can have stable priorities, while one program can repeatedly change direction. Applying the finding to branch work is a reason to examine the local conditions, not proof that three concurrent programs caused the observed outcome.

A portfolio view by affected group can expose those conditions. Record training and duplicate-work hours, dates, dependencies, available staff and support, and the new tools and decisions people must learn. Keep time demands separate from complexity and familiarity: cognitive load is not a sum of hours or a count of programs. Discuss the work with the receiving teams and compare demand with their capacity and service commitments. The decision may be to move a date, reduce scope, add support, or retain the sequence because it is feasible. Document the reason and review actual experience. Neither every monthly reprioritization nor every concurrent rollout has the same effect.

From forecast benefits to evidence of realization

Benefits realization connects an intended outcome to evidence and continuing ownership. A value hypothesis states what change is expected to help, for whom, and why. The Service Manual recommends deriving benefits from the service’s purpose, making assumptions explicit, and selecting a small set of headline measures with further diagnostic detail. Its four mandatory KPIs apply within that guidance’s service context; an insurer must choose measures suited to its own claims and obligations. Define the reference, target, prerequisites, accountable owner, evaluation method, and review decisions before reporting a benefit. Keep reviewing during delivery and after deployment, and transfer any continuing monitoring to an operating owner.

Element	What it records	The insurer’s claims benefit, rewritten
Benefit	The outcome, in the business’s terms	Claims handled with less handling time per claim, at the same or better accuracy
Hypothesis	The expected causal contribution and assumptions	The redesigned workflow is expected to reduce active handling from twenty-six to fifteen minutes by removing duplicate entry and separate lookups, while preserving accuracy
Baseline and measure	Comparable work, events, data sources, period, and analyst	Pre-launch mean of twenty-six active staff minutes per completed eligible claim; operations analytics combines work-session records and observation across formal and spreadsheet paths. Waiting is measured separately
Preconditions	Required capabilities and decisions, with owners	Tested workflow and controls; staff readiness; support capacity; validated retirement of redundant paths with approved exceptions; coordinated demands from other changes
Owner	Authority over the outcome and continuity after the program	The operations director owns the claims process and resources and participates in the program; operations retains responsibility afterward
Review dates	Plan-based reviews plus cohort-relative outcome checks	Monthly progress and risk reviews from program start; outcome reviews three, six, and twelve months after each cohort adopts, without suspending reviews if adoption is delayed
Disposition	Evidence-based status and next decision	At month twelve, the handling-time target was unmet. At month twenty-four, the observed gain remained short of target; its attribution and financial effect were reported separately

Separate the forecast from what was observed and what can be attributed to the intervention. Budget plans can use estimates with uncertainty and contingency; they must not label those estimates realized savings. Shorter active work may release capacity for other cases without reducing payroll. Cash savings require evidence such as lower paid overtime or contract expenditure; avoided future hiring is a separate counterfactual claim. Compare like-for-like periods and service levels, include transition and continuing operating costs, and avoid counting the same released capacity twice. GOV.UK’s benefits guidance distinguishes financial and non-financial benefits and calls for revisiting the economic model as evidence arrives. Twenty-Three of Twenty-Five Shipped explains the related distinction between platform use, observed outcomes, and attributed effects.

For handling time, specify active-work events rather than subtracting arbitrary workflow timestamps that may include waiting. Include rework and manual routes, disclose missing observations, and report unfinished cases alongside completed ones. Compare case complexity, staffing, volumes, and measurement versions. A before-and-after mean alone cannot isolate the effect of the new system from training, staffing, or concurrent change. Use a suitable comparison design and state its assumptions; where only a plausible contribution is established, say so. Verify accuracy and other guardrails before accepting a speed claim.

A missed review target triggers a decision, not automatic write-off. Investigate delay, partial delivery, measurement failure, and a disproved hypothesis. The owner may revise the forecast, fund a remedy, recognize a bounded benefit, or stop pursuing it, with reasons and the original target retained. Evidence can also justify leaving the plan unchanged. Once an outcome is accepted, identify who will monitor its persistence and what deterioration reopens the decision.

Why the organization pushes back, and what it takes

The legacy-displacement authors describe established structures rejecting new ways of working, using a telecom whose existing controls prevented the faster cycles its leaders wanted. This is an account of organizational experience, not a diagnosis that every objection is resistance. A concern may expose an unmet control, accessibility need, or operational dependency. Their discussion also relays Goldratt’s view that pressure, exhausted familiar approaches, and help with a first step can enable a paradigm shift. Treat that as a perspective rather than a requirement to try every old alternative before changing. In the insurer’s plan, useful help means supervised practice, workable processes, and a route for resolving exceptions; forcing use of an unready replacement is not a substitute for those conditions. DORA’s 2024 summary likewise emphasizes supportive leadership, learning, and the value of using cloud flexibility. Those organizational-performance findings do not redefine the acceptance criteria of every individual migration. Leaders still need to state what this program promises and provide the authority, information, and resources to test it.

Judging the organizational side of a transformation

Assess the organizational plan through its decisions and evidence. Adoption measures should identify their unit, denominator, completion rule, and gaps. Process owners need authority, tested changes, and a verified transition and retirement plan. Workforce preparation should include scheduled learning and demonstrated operational readiness. A cross-program view should support a reasoned decision about combined demands, whether or not a date moves.

Benefit records should distinguish forecasts, observations, attribution, and financial realization, with regular reviews and continuing ownership. Decide which projects to delay and which benefit claims to retain, revise, or stop from the evidence and available capacity, without setting a minimum quota for delay or abandonment. A green launch, training attendance, and a budget reduction describe activities or decisions; none alone establishes a successful transformation.

What a reviewer should ask

1. What share of eligible cases completes the defined workflow, how are mixed and unknown paths classified, and how does that differ from branch coverage or digital channel use?
2. What evidence authorizes closure of an old path, which exceptions or recovery routes remain, and what happens if a unit misses the target date?
3. For each role, what changes in the work, who owns the redesign, and who tested it with them?
4. Who will operate the new system after the program, when was the knowledge transferred, and how was that tested?
5. What else lands on the same people in the same quarter, and who holds the view across programs?
6. For each benefit, what is the hypothesis, the baseline, the owner, the preconditions, and the review dates?
7. Which outcomes are forecast, observed, attributed, or financially realized, and what evidence supports keeping, revising, or stopping each benefit claim?
8. What help does a branch get in its first two weeks, and who answers when it asks?

The insurer began remediation at the twelve-month review. The portfolio owner moved the next telephony rollout wave by a quarter after assessing shared staff and support demand. Twelve branch staff helped redesign the claims process over six weeks, followed by testing in two branches. Each region retired unnecessary old screens only after readiness and recovery checks, with two support people available locally for its first fortnight. The company rehired the two former contractors for a three-month transfer to two new operations engineers, including incident drills. Finance revised the budget and contingency using the remaining workload rather than waiting for a single metric to cross a threshold.

The first region showed a lower handling-time mean three months after remediation began; all regions had reported improvements by month twenty-one after the original launch. The board’s month-twenty-four pack used the same stated claim scope and observation rules: 94 percent of completed eligible claims followed the new workflow end to end. The other 6 percent and outstanding claims were reported separately, including approved exceptions and unresolved cases. Comparable observed active handling time was seventeen minutes, against twenty-six before launch, thirty at the first-year review, and a target of fifteen. Sampled accuracy checks met the agreed guardrail, with their coverage and uncertainty disclosed. The nine-minute observed gain was short of the eleven-minute target. Staffing and training had changed as well, so the pack described contribution evidence and remaining attribution uncertainty instead of assigning the whole gain to software. Payroll had not fallen; released capacity, any verified changes in overtime or contracts, and added support and operating costs were reported separately. The original cash-saving claim remained unverified, with an owner and another review date.

References:

• GOV.UK Service Manual: Measuring digital take-up (Open Government Licence v3.0).
• GOV.UK Service Manual: How to set performance metrics for your service (Open Government Licence v3.0).
• GOV.UK Service Manual: Using performance data to improve your service: an introduction (Open Government Licence v3.0).
• GOV.UK Service Manual: Measuring the success of your service (Open Government Licence v3.0).
• GOV.UK Service Manual: Measuring the benefits of your service (Open Government Licence v3.0).
• GOV.UK Service Manual: Data you must publish (Open Government Licence v3.0).
• Government Digital Service and Central Digital and Data Office: Managing legacy technology (Open Government Licence v3.0).
• DORA: Accelerate State of DevOps Report 2024 — official summary (CC BY 4.0).
• Ian Cartwright, Rob Horn, and James Lewis: Patterns of Legacy Displacement (2024).