Soft Deletion

Soft deletion separates logical removal from physical destruction. A record or object becomes unavailable to ordinary use while retained state supports recovery for a defined period. The authorized recovery path, preserved versions, and eventual purge depend on the implementation. Google’s SRE example uses administrative paths behind the product’s normal deletion behavior; that is one design rather than the definition of every soft-delete feature.

Google’s SRE book places it first among the layers protecting data — “the first layer is soft deletion (or ‘lazy deletion’ in the case of developer API offerings), which has proven to be an effective defense against inadvertent data deletion scenarios” — ahead of backups and validation. That ordering is the surprising part, and it has a reason.

Why it comes before backups

Layers are ordered by how common the failures they catch are, and the common failure is not a disaster. “When velocity is high and privacy matters, bugs in applications account for the vast majority of data loss and corruption events.” Strongly enough that the book puts it this way: “data deletion bugs may become so common that the ability to undelete data for a limited time becomes the primary line of defense against the majority of otherwise permanent, inadvertent data loss.”

Three sources of deletion are named, and the third is the one people do not plan for.

• Users. Accidental deletion, which carries a support cost.
• Code. And specifically, “the most devastating acute data deletion cases are caused by application developers unfamiliar with existing code but working on deletion-related code” — the person least equipped to be careful is working on precisely the dangerous path.
• Attackers. “In account hijacking scenarios, a hijacker commonly deletes the original user’s data before using the account for spamming and other unlawful purposes.” Deletion is not collateral damage there; it is step one.

Put those together and the conclusion follows: “when you combine the commonality of accidental user deletion with the risk of data deletion by hijackers, the case for a programmatic soft deletion and undeletion interface within and/or beneath your application becomes clear.”

User recovery and administrative retention

A user-visible trash folder can be the interface to retained deleted data. Google’s example adds a separate administrative recovery layer after the user empties the trash, including protection for deletion by third-party add-ons. The distinction in that example is between who can recover the data and at which stage; user recovery and soft deletion are not inherently unrelated mechanisms.

In that layered design, the trash handles a mistake the user can reverse directly, while authorized support can recover data after it leaves the trash and before the administrative retention window expires. State which operations advance the object through those stages and which permanently remove it.

The state is reachable only through named, privileged paths, which “may include legal discovery, hijacked account recovery, enterprise administration, user support, and problem troubleshooting.” Worth noting what that implies and the source does not spell out: those paths are a way to read data users believe is gone, so they belong under the same access control and audit expectations as any other privileged interface. A soft-delete facility without an audit trail has quietly created a category of invisible access.

The business case for building it is stated bluntly, and it is about support economics rather than heroism: Google implements this “for our most popular productivity applications; otherwise, the user support engineering burden would be untenable.”

How long, and where 60 days comes from

The retention window is the one parameter, and there is a documented range with an empirical bound. “Common choices of soft deletion delays are 15, 30, 45, or 60 days.” For the user-facing half of it, “the Gmail trash bin allows users to access messages that were deleted fewer than 30 days ago.”

The number most often cited as an upper bound comes from observation rather than convention: “in Google’s experience, the majority of account hijacking and data integrity issues are reported or detected within 60 days. Therefore, the case for soft deleting data for longer than 60 days may not be strong.“

That is a useful shape of argument to borrow even if your numbers differ, provided the word majority survives the borrowing. The window should be set from how long your own organization takes to notice this class of problem — which you can measure from your incident history, and which is a different question from how long you keep backups. But detection delay has a tail, and the events out in that tail are not less recoverable for being rare: past the window you chose, the cases that surface are ones the extra retention would have covered. So the decision weighs the shape of that tail and the loss those late cases carry against the cost and the exposure of holding deleted data longer — and against any retention or deletion obligation that applies to it, which is a question for whoever owns that obligation rather than one this kind of engineering source settles.

The inputs to the decision are named too: the delay “depends upon an organization’s policies and applicable laws, available storage resources and cost, and product pricing and market positioning, especially in cases involving much short-lived data.”

Living with a duty to actually delete

The obvious objection is that keeping deleted data contradicts a privacy obligation, and the source treats both as requirements rather than choosing between them. On one side: “any product that upholds the privacy of its users must allow the users to delete selected subsets and/or all of their data,” where the working definition of the privacy requirement is that “data must be destroyed within a reasonable time after users delete it.”

On the other: “soft deletion implies that once data is marked as such, it is destroyed after a reasonable delay.“

Read together, soft deletion is two promises rather than a weakening of one. The data becomes immediately unusable by the product, and it is genuinely destroyed after a stated, finite period. A system that keeps the first promise and not the second is not doing soft deletion; it is retaining data and calling it recovery. Which makes the destruction job as important as the marking: a soft-delete implementation whose purge never runs, or silently fails, has converted a privacy commitment into a liability, and nothing in the product surface will show it. That job deserves monitoring like any other.

What breaks when you implement it

The mechanism sounds like adding a flag. Four things commonly break, and they are the reason retrofitted soft deletion often behaves inconsistently.

• Uniqueness and identity. A retained row can still occupy a unique key. Reserving that key during retention is one policy; restricting uniqueness to live rows or releasing a reusable key permits a new record but creates a possible conflict on restoration. Keep stable record identities separate from reusable addresses. If the key now belongs to a new account, explicitly reject the restoration, choose another key, or use an authorized reconciliation procedure. A flag flip must not merge distinct owners or silently reassign their references.
• Counts and aggregates. Every query that forgets the filter reports numbers including deleted data. This shows up in dashboards, quotas and billing before anyone notices, which is why the filter belongs in one shared place rather than in each query.
• References. Restoring a record requires the things it pointed at to still exist. If children were hard-deleted while the parent was soft-deleted, undeletion produces a half-object — so deletion and restoration have to agree on the unit.
• Downstream copies. Search indexes, caches, exports and analytics stores learned about the record and may not learn about the deletion, or may learn about the undeletion late. The product can show a deleted item through a stale index long after the database stopped serving it.

The practical consequence is that soft deletion works best when it is implemented beneath the application rather than per-feature — which is what the source’s phrase “within and/or beneath your application” is pointing at. One place that knows what deleted means beats fifteen features each deciding.

Where it sits, and what it does not cover

It is the first layer, not the only one. Behind it come “backups and their related recovery methods” and then “regular data validation,” and replication sits across all of them as something that helps occasionally while “data recovery plans should not rely upon replication.”

A row-marker implementation can undo a covered deletion while retained data remains available; it does not by itself retain earlier values after an incorrect update or survive loss of its storage. Product features can cover more. For example, Azure Blob soft delete without a hierarchical namespace can preserve a snapshot before supported overwrite operations, with different behavior when versioning is enabled; metadata and other operations have their own limits. Check the documented operations, access paths, retention, and purge behavior of the actual implementation. Replication Is Not Recovery explains the wider recovery layers, and point-in-time recovery addresses recovery to a historical state.

Implement deletion and authorized recovery consistently across the relevant application paths. Choose retention from detection experience and applicable obligations, monitor the purge, and test downstream visibility, references, and key-reuse conflicts. When a deletion bug occurs, assess whether the retained object can be safely restored under its original identity and whether other systems need repair. Undeletion can be simple only when those conditions hold.

References

• “Data Integrity: What You Read Is What You Wrote,” Google SRE Book
• Microsoft Learn — Soft delete for blobs

The 60-day observation describes Google’s experience. Azure behavior depends on namespace, versioning, and the operation involved.