A social listening strategy that begins with a decision and ends with a review gate

An England-focused customer-service team has until 30 June to decide whether to revise its complaint-escalation guide. The following hypothetical scenario tracks what the team checks, what the evidence can support and what passes to the decision owner. Its illustrative 70 per cent precision figure is not a finding about a real service or a measure of public opinion.

A guide revision with a deadline

The customer service director owns the choice: revise the complaint-escalation guide or leave it unchanged. The research question is whether public posts can reveal how people describe their first response to a complaint. The team records the deadline, affected population and possible actions before collection begins.

Separating a decision from its research question and query helps prevent strategy mistakes, such as monitoring without a choice to inform. The record names who may act, the group affected and what evidence could support changing course or stopping. If no plausible result can influence the guide decision, the team stops before gathering posts.

The team also considers whether listening is suitable. Service records can answer questions about logged contact volumes, while interviews can help when reasons and context matter. A designed survey is better suited to estimating a defined population; online observations may show emerging language or illuminate a known conversation, but not represent silent people.

What the team allowed itself to observe

Before searching, the team defines a qualifying observation, an England rule, the period and the limits on interpretation. A location mention, dashboard count or automated mood label is not treated as a population measure. The record describes the group affected and the inference the evidence can and cannot support.

An observation could be one public post, comment, review or video within the stated period. Duplicates, deleted records and inaccessible items are recorded separately. The chosen England rule is a verified English service location; an event held in England could also qualify if that rule is selected for the project.

A place name could refer to an event or employer rather than where a person lives. The team does not turn such a mention into an England audience count. If location cannot be verified through the approved method, the record stays marked unknown instead of being assigned to England.

A brand mention does not establish that its author belongs to the service population or describes the relevant experience. The observation stream can cover only part of the affected group. Keeping those distinctions in the target audience record prevents a source count from quietly becoming a claim about everyone using the service.

The source check starts with X recent search, public posts only, and the last 30 days; the team will not contact profiles. It checks authorisation and technical availability, then records available fields, historical window, deletion duties, quotas and notices of change. A source coverage check must be repeated because platform terms and technical access can change.

The register notes omissions and a next review date for each channel. X distinguishes recent search from full-archive search, with different access conditions. Reddit's Data API Terms allow access to be revoked, restrict use to approved purposes and require deletion of data no longer needed for that purpose.

The YouTube Data API search method has defined result types and parameters; its developer policy restricts certain derived metrics and uses of personal data. Those platform records describe their own services, not England relevance or complete coverage by a listening supplier. For England scope, the team names a source with that scope on its face, such as a council licensing register held under the Licensing Act 2003.

The England compliance record keeps the chosen legal context visible. A procurement demonstration uses the approved question, source list and test period, with returned and missing fields recorded. Claims such as all social media remain unproven until tested against the agreed scope and route.

A public page is not blanket permission to copy material, profile its author or contact that person. The team pauses if collection would bypass a platform control or exceed an approved purpose. Its team responsibilities record who owns the source register, the permitted route and the decisions that require escalation.

What the sample could and could not show

The query is a versioned set of inclusion and exclusion rules, not just words entered into a tool. Its record includes author, approval date, syntax, source and intended meaning. Before collection, the team tests actual language, supported spelling variants and untagged references against the question.

For this scenario, the query joins complaint or escalation with response or reply, excludes retweets and limits language to English: （"complaint" OR "escalation"） AND （"response" OR "reply"） -filter:retweets lang:en. It runs on X recent search across 30 days. The team checks ambiguity deliberately and retains the version used for the review sample.

Two reviewers independently label a sample from the actual stream for scope, relevance to the research question and any assigned category. They resolve disagreements and preserve decision notes. This is a quality-control check on the collected material, not an estimate of England-wide opinion.

The report defines precision as relevant sampled returns divided by all sampled returns reviewed. It states the sample period, source, query version, denominator and excluded duplicates. An illustrative report might show 70 per cent precision with those details beside it; the figure is not an England audience percentage or a real service finding.

The team does not call the measure recall unless it has a defensible set of all relevant items, which is usually unavailable. High precision can still miss unexpected language, private spaces or sources outside the authorised route. It stops if ambiguity overwhelms useful material or an access route falls outside approved terms.

Automated sentiment, topic, intent and risk categories are claims to test rather than established facts. The team defines each label, who will use it and which errors matter most. A false accusation of hostility may have different consequences from a missed neutral complaint, so the sentiment and intent record shows those distinctions.

The ICO's accuracy and statistical accuracy guidance distinguishes false positives from false negatives and says appropriate measures depend on context and risk. It warns that performance can shift as populations change. Because the guidance is under review following newer legislation, a named UK privacy and AI reviewer confirms its application before launch.

For each classifier, the team retains its model or rules version, label definitions, validation sample, human adjudication process and results by relevant subgroup or language where lawful and appropriate. Precision or recall is used only with defined denominators; no accuracy percentage is invented. If a material change invalidates the test, the team versions the instrument again before collection resumes.

The decision after scrutiny

Public availability does not authorise unrestricted use of personal data. Before collection, the controller records purpose, lawful basis, source permissions, fields, access roles, retention, sharing and rights handling, then checks whether a data protection assessment is required.

The ICO says people may still have a right to privacy information when their data comes from public sources, subject to detailed exceptions and exemptions. Combining public material with an existing profile can add risk. The team considers data protection principles and the right to be informed as part of its review.

The 2023 MRS Code of Conduct can inform participant welfare and reporting when research falls within its professional scope. It is not legislation and does not replace legal advice. The team separates observation from later contact: a public complaint does not provide permission to send promotional messages.

If the findings might support advertising comparisons, the team preserves the method and evidence behind them. CAP Executive guidance says identifiable comparisons must be verifiable; it concerns UK non-broadcast advertising and is not legal advice on a proposed claim. The team documents data sharing and keeps intended use within the approved purpose.

Only after the source route, question and query are approved does the team compare tools. Its requirements matrix covers access, query controls, export, audit history, deletion, roles, classifier validation, accessibility, incident handling, support and contract exit. Each supplier response is marked documented, demonstrated, tested or unresolved.

NCSC supplier assurance questions address incident responsibilities, subcontractors, access controls, security testing, return or deletion of data and supplier-risk changes. The relevant page was reviewed in 2020, so the team requests current evidence instead of relying on an old certificate or answer. Supplier comparison uses the same declared criteria.

A product returning more mentions may have broader sources, looser matching or more duplicates. The team compares tools using the same question, period, inclusion rules and reviewed sample. It does not call a supplier more accurate or comprehensive without a declared test, and includes supply-chain security and exit arrangements in the assessment.

The first ninety days test the operating method, not a promise of business improvement. The schedule gives each stage an owner, output and stop condition; a stage does not pass merely because the team has produced a volume chart.

At the review gate, the team keeps an observation, a finding, a recommendation and a decision distinct. The observation is what the authorised collection returned; the finding is an interpreted pattern with limits; the recommendation proposes a response and supporting evidence; the decision records what the owner chose and why.

High-risk items require human verification against their original context. Legal allegations, safety issues and personal data go to specialist escalation routes rather than a general content queue. The government social media playbook connects listening with strategy and evaluation, and recommends involving information assurance or data protection specialists early.

That playbook was last updated in January 2023 and describes UK government practice. It is a process example, not proof that a private organisation's plan is compliant or effective. Publication also needs named authorship and fact-checking, plus qualified privacy, research-ethics, AI, security, advertising and legal review.

The team defines operational measures before collection. Retrieval precision is relevant sampled returns divided by all sampled returns reviewed during the test period. Review completion is adjudicated items divided by items selected for human review, excluding documented withdrawals or inaccessible records.

Decision uptake is recommendations accepted for action divided by eligible recommendations considered by the named owner in the reporting period. Response timeliness is eligible escalations completed within the approved time divided by all eligible escalations due in that period. Each measure states its denominator and reporting period.

These operational measures do not measure population sentiment, causal effect or return on investment. The Government Communication Service Evaluation Cycle separates inputs, outputs, audience responses, outcomes and impact, and says evaluation should be planned at the start. It is government communications guidance, not a claim that listening itself proves impact.

If the strategy claims an intervention changed behaviour, its evaluation design must address alternative causes. A before-and-after listening comparison alone cannot separate a campaign from news, seasonality, platform changes or query changes. The team lists review triggers, including a source-policy change, new language, classifier update or abnormal missing-data rate.

A query revision, privacy complaint, security incident or new commercial purpose sends the plan back to the relevant approval gate. The team rechecks platform terms and regulatory guidance on the approval date, records conflicts and discloses what was not tested. Its final plan supports decisions about an observed stream, but cannot certify what England thinks or guarantee every relevant conversation was found.

Rules carried out of the scenario

The scenario's central rule is to connect the decision, observation boundaries, permissions, query, tool and action route. Keep the records current and their limits easy to find. If no plausible finding could affect the choice, stop before collection rather than creating a volume chart without a decision behind it.

Test queries and automated labels using reviewed samples; preserve denominators and disagreements; and avoid population claims the data cannot support. Recheck access, retention and supplier evidence at review. Use the quality review to confirm qualified scrutiny, then proceed, revise or close with a documented decision.