Operations
Part of Running social listening operations with access, rights and recovery kept apart
Service standards for social listening with thresholds someone can be held to
Define social listening service standards in England with explicit units, periods, denominators, evidence, owners, thresholds and breach responses.
Social listening service standards should reveal a broken control before an England-focused report reaches a decision-maker. There is no official English benchmark. Set local thresholds from harm, contract, capacity and baseline evidence, then preserve failures instead of averaging them away.
GOV.UK's service performance metrics guide applies to government services. Its emphasis on purpose, source and context is useful design discipline, not a required target for private teams.
What to take away
- Set local thresholds from harm, contract, capacity and baseline evidence rather than seeking an official English benchmark.
- Preserve failures instead of averaging them away, because one unresolved deletion or disclosure can matter more than routine passes.
- Use the GOV.UK service performance metrics guide for design discipline, not as a required target for private listening teams.
- Do not relabel adjudicated sample relevance as recall, since a feed cannot show conversations never available to it.
- Review definitions before changing targets and never rewrite history after a miss.
Source permission coverage
Unit and period: percentage points, checked at each weekly release review. Numerator: required sources with current entitlement, approved use, terms date and named owner evidenced. Denominator: every source required by the release record. Include blocked and unavailable feeds; exclude sources expressly removed from the approved scope before the period began.
Source permission coverage threshold
100% of required sources
current entitlement, approved use, terms date, named owner
Evidence: contract-linked source register
Owner: source custodian
Breach: suspend source, mark gap, new release decision
Evidence is the contract-linked source register. The source custodian owns it. Proposed threshold: 100 per cent, because partial permission cannot be made acceptable by another feed. Response: suspend the unsupported source, mark the reporting gap and obtain a new release decision.
Adjudicated sample relevance
Unit and period: percentage points for each query version and seven-day analysis window. Numerator: sampled items that two reviewers, after resolving differences against the codebook, classify as relevant. Denominator: all items drawn into the documented sample. Include ambiguous and duplicate returns; exclude only records the sampling protocol identified as technically unreadable before coding.
Adjudicated sample relevance process
- Draw documented sample per query version and 7-day window
- Two reviewers code items against codebook
- Resolve differences and classify relevant items
- Calculate percentage points
- If below signed project minimum, withhold conclusions
- Diagnose error classes and approve fresh validation
The sample file, query version and adjudication log are evidence, owned by the classification assurer. Threshold: the project-specific minimum signed before collection, with no industry figure implied. Response: withhold conclusions from that query, diagnose error classes and approve fresh validation before reuse.
Do not relabel this as recall. A feed cannot show all relevant conversations that were never available to it.
Rights-action propagation
Unit and period: percentage points for cases due in the monthly rights register. Numerator: due cases with a qualified decision and recorded outcome. This covers every applicable workspace, plus alert, export, report, integration and backup treatment. Denominator: all cases whose internal due date falls in that month.
Include refused or limited requests when reasons and communications are complete, but exclude only formally withdrawn cases. The case file and system map supply evidence, and the privacy lead owns the standard. Proposed threshold: 100%.
Response: escalate the case, pause affected reuse, investigate missing destinations, because ICO erasure guidance is under DUAA review; a named reviewer must set legal timing and outcome.
Material incident acknowledgement
Unit and period: percentage points for each calendar month. Numerator: material security alerts acknowledged by the duty owner inside the organisation's signed response window. Denominator: all alerts the incident criteria classify as material that month. Include supplier notifications and later false positives; exclude test events labelled before an exercise.
The timestamped incident log is evidence. The security lead owns it. Threshold: 100 per cent, while the response window itself comes from assessed harm and capability. Response: invoke the escalation route, preserve evidence, review cover and determine whether collection must stop. NCSC incident guidance supports preparing people, process and technical capability, but supplies no listening-industry time target.
Accessible report gate
Unit and period: percentage points for every proposed report release. Numerator: accessibility-blocking defects closed and retested. Denominator: all blocking defects recorded in the agreed test of that version. Include table, reading-order, keyboard, alternative-text and colour failures; exclude advisory improvements explicitly classified as non-blocking before testing.
The test plan, defect log and retest artefact are evidence. The report owner owns the gate. Threshold: 100 per cent. Response: withhold that format or issue an approved accessible alternative. The GOV.UK WCAG 2.2 guide states government-service requirements; private teams need advice on their own duties.
Review definitions before changing targets. Never rewrite history after a miss. Show the failed case beside the percentage, because one unresolved deletion or disclosure can matter more than a month of routine passes.
Before you act
- Set local thresholds from harm, contract, capacity and baseline evidence.
- Preserve failures instead of averaging them away.
- Review definitions before changing targets.
- Never rewrite history after a miss.
- Show the failed case beside the percentage.
- Do not relabel adjudicated sample relevance as recall.
Common questions
What should social listening service standards reveal?
They should reveal a broken control before an England-focused report reaches a decision-maker. There is no official English benchmark, so teams must set local thresholds from harm, contract, capacity and baseline evidence, then preserve failures instead of averaging them away.
Why should adjudicated sample relevance not be called recall?
A feed cannot show all relevant conversations that were never available to it. The metric measures sampled items classified as relevant after adjudication, not the feed's ability to capture every relevant conversation. Relabelling it as recall would misrepresent what the measurement actually covers. Honest measurement and reporting of such metrics is set out in a separate guide.
How should a material incident acknowledgement failure be handled?
Invoke the escalation route, preserve evidence, review cover and determine whether collection must stop. The threshold is 100 per cent, while the response window itself comes from assessed harm and capability. NCSC incident guidance supports preparing people, process and technical capability but supplies no listening-industry time target.
