Five social listening failures to rehearse before one of them happens

What to take away

Rehearse five failuresa changed source route, a drifting classifier, a summary that loses its evidence, an agent acting out of scope, and mishandled personal data or protected content.
Write four things before each exercisea named owner, dated evidence, a stop condition, and a recovery test with a recorded result.
Scenarios are possible futures, not predictions. The Government Office for Science Futures Toolkit assigns no likelihood to them.
A rehearsal is proof only when the log entry is dated and signed by the owner.
Use the ICO breach reporting service for a reportable breach, and the CMA for competition and consumer concerns.

The five failures, with owners and stop conditions

Five failures, with owners and stop conditions

Failure	Owner role	Stop condition
Material source route changes	Source-access lead	Continuing permission for the intended retrieval cannot be shown
Classifier drifts after a change	Analytical assurer	New output has not passed the same decision-relevant validation
Generated summary loses its evidence	Report editor	A material sentence cannot be traced to the approved observations
Agent acts outside its authority	Service owner	An action lacks valid approval or exceeds its allow-list
Personal data or protected content mishandled	Privacy lead	Purpose, necessity, transparency, rights handling or reuse permission cannot be demonstrated

Run each row against the live England service, not a hypothetical one. The stop condition goes in writing before the exercise, so the halt is recorded rather than argued out.

The source route is the failure most often settled in advance. What a social listening contract must settle covers the source rights, audit and exit terms that decide when retrieval stops.

How to run a rehearsal in one sitting

Read the trigger aloud and check the owner role is reachable.
Pull the dated evidence the stop condition depends on from the log, not memory.
Apply the stop condition and record the outcome, including a decision to continue.
Run the recovery test on a low-risk live case, then log the result and the next re-run date.

Discussion proves nothing. A recovery test counts only if something was stopped, restarted or withdrawn, and the result was written down.

Example: a completed record for a changed source route

Completed record for a changed source route

Scenario: Material source route changes
Owner: Source-access lead, by role rather than name
Evidence: Signed contract version, supplier notice, connector log, affected reports
Stop condition: No evidence of continuing permission for the intended retrieval
Recovery test: Route disabled, new records quarantined, next report issued from authorised sources with a declared break
Result: Break declared in the report; downstream copies deleted and the deletion checked
Sign-off: Source-access lead, countersigned by the privacy lead

Keep role labels in the record and names in the staffing file. The record must survive a change of personnel, which is why it is dated.

The evidence log, re-run cadence and sign-off

One log per service, not one per tool. Each entry carries the scenario, owner role, trigger date, the evidence that fired the stop condition and where it sits.

The stop condition as written, and the decision taken.
The recovery test, its result and any exception allowed.
The next re-run date and the signature line.

Cadence follows change more than the calendar. Re-run the source-route rehearsal at every supplier notice and annually, and the classifier and summary rehearsals on each model, query or label change. Four jobs AI actually does in social listening sets the gate each must pass. Re-run the agent rehearsal whenever its allow-list changes.

The owner signs each entry; the privacy lead countersigns any entry touching personal data.

Where England reporting routes lead

When the data rehearsal becomes a live incident, assess the risk to people's rights and freedoms before deciding whether to report. The ICO's UK GDPR guidance and resources is the regulator's reading of lawful processing.

Reportable breaches go to the ICO through its breach reporting service, normally within 72 hours of becoming aware. Keep the containment record, the assessment and the decision together, and log the reasoning either way.

Competition or consumer protection concerns, such as an agent misdescribing a rival's offer, go to the Competition and Markets Authority, whose remit covers both.

The Data Protection Act 2018 sits behind that assessment, and the National Cyber Security Centre's incident response guidance covers detection, containment and remediation. Where data protection law reaches into social listening lists the records that shorten the assessment, including the lawful-basis note and the deletion trail.

Before you act

Book the slot and a deputy for each owner role before the exercise starts.
Export each stop condition's evidence from the system that holds it, not from a slide deck.
Name the low-risk live case each recovery test will use.
Confirm who can pull the kill switch on any agent, and test it.

Common questions

Which five failures should I rehearse?

A changed source route, a drifting classifier, a summary that loses its evidence, an agent acting out of scope, and mishandled personal data or protected content, each with an owner and a stop condition.

How is a stop condition written?

As a halt rule for when evidence fails, such as permission that cannot be shown, validation not repeated after a model change, or a sentence that cannot be traced to the approved observations.

How often are rehearsals re-run, and who signs?

The source-route rehearsal at each supplier notice and at least annually; the others on model, query or label changes. The owner signs, and the privacy lead countersigns where personal data is involved.