Measurement
Part of Measuring social listening honestly, from metric cards to causal evaluation
How a social listening dashboard should show its gaps
Show missing, delayed, unavailable and lineage-break states on a social listening reporting dashboard in England, with evidence labels and source blind spots.
A dashboard that shows only what arrived misleads by silence. Each tile should say whether its number is complete, late, not provided or cut off from its source, and a blank must never read as zero. Four states cover those cases.
A social listening reporting dashboard should help a named user make a stated decision. Platforms such as Brandwatch, Talkwalker, Meltwater, Sprout Social and Hootsuite may feed the view, but available sources and metrics vary. If a monthly report answers the question more clearly, do not build a permanent dashboard.
What to take away
- Give each affected tile an explicit statemissing, delayed, unavailable or lineage break.
- Zero means a defined population produced none, not that a feed failed or a field was absent.
- Every tile must link to a metric card covering population, numerator, denominator, unit, period, source, uncertainty and owner.
- Separate evidence lanes and label them as supplier-reported, human-coded, observational or causal estimate.
- Withhold an affected tile when lineage breaks rather than estimate a convenient replacement.
Give every gap one of four states
A gap needs a name, not a smooth line. Missing, delayed, unavailable and lineage break describe four different failures, and each calls for a different reader response. A tile that hides the difference invites a reader to guess.
| State | Meaning | What the tile does |
|---|---|---|
| Missing | An expected field or source is absent | Says so and names the absent field or source |
| Delayed | The expected data has not arrived | Says the data is late rather than showing a partial total as final |
| Unavailable | The source does not provide it, or access is unavailable | Says the measure cannot be had from this route |
| Lineage break | The data or definition cannot be traced through its source path | Is withheld, with no estimated replacement |
Zero is a result, and it is a different thing from any of these states. It means a defined population was observed and produced none. It does not mean a feed failed or the field was absent.
Tie each tile to its card
Every tile links to a metric card, and the card registry is built before the interface. A tile displays its definition version and latest extraction, and counts say plainly when no denominator applies. Full card specifications sit in measuring social listening honestly.
Metric card required fields
- Population and numerator
- Denominator and unit
- Period and timezone
- Inclusions and exclusions
- Source and uncertainty
- Owner and update trigger
Do not calculate a new ratio in the visual layer. Version-controlled analysis should produce the result and an evidence file, and the dashboard reads that output. This keeps a filter change from quietly altering the population while leaving the title untouched. What each figure can claim is set out for the five key metrics.
Label the evidence lane
Use distinct sections for source continuity, query relevance, classifier testing, provider-reported activity, operational outcomes and causal evaluation. Apply visible labels such as "supplier-reported", "human-coded", "observational" or "causal estimate". They describe evidence type, not quality grades.
Evidence lanes and labels
Lane
- Source continuity
- Supplier-reported
- Query relevance
- Human-coded
- Classifier testing
- Human-coded
- Provider activity
- Supplier-reported
- Operational outcomes
- Observational
- Causal evaluation
- Causal estimate
Visible label
- Source continuity
- Query relevance
- Classifier testing
- Provider activity
- Operational outcomes
- Causal evaluation
A causal estimate must rest on a credible design, and the attribution method decides which claim a number can carry. An evaluator estimating an intervention effect needs a different view from a communications lead investigating a sudden topic, so the lanes should be easy to tell apart at a glance.
Name each source's blind spots
Inputs may include X, Reddit, TikTok, Facebook and Instagram, review sites, forums and news publishers, through platform APIs, licensed suppliers or other permitted collection methods. Coverage depends on source access, product, contract and settings. It should not be presented as a complete record of online conversation.
Record relevant gaps on the cards: API access may be capped or sampled; deleted or edited posts may disappear or change; private or closed groups may be outside the collection; and language or dialect coverage may vary. Document bot and spam filtering, influencer weighting and sentiment-classification errors too.
These choices affect what a count or comparison means, so make them visible rather than treating them as generic missing data. X's metric documentation separates public, non-public, organic and promoted fields and gives an access window for some types. A source change should close or annotate the series, not vanish behind a smooth line.
Make the charts inspectable
Put the measure, geography and period in the title. Keep units on axes and notes near the display. Let users reach the source, downloadable result and revision history. Avoid colour-only meaning and preserve reading order when exporting.
The Government Analysis Function's accessible-chart checklist recommends body-text titles, source links and text alternatives. It is public analytical guidance, not evidence that a particular dashboard is accessible. Test the actual interface and exported format with relevant users and technology.
The Office for Statistics Regulation dashboard guidance is written for official statistics, not social listening. It says that after its questions you may find you do not need a dashboard, because a less complex product may meet user needs.
Its questions on user need, maintenance, archiving, revisions, methods and limitations can still help a brand team structure governance. Using them does not imply Code compliance.
Keep gaps from exposing people
Most viewers need aggregates and reviewed examples, not searchable personal profiles. Apply least access, suppress unsafe detail and define who may open underlying records. Removing a username may not anonymise unusual content or a small group.
ICO guidance says identifiability depends on context and reasonably likely means. Where the dashboard processes personal data, assess a lawful basis under UK GDPR Article 6 and apply the principles in Article 5. A named UK privacy reviewer should approve audience, linkage, export and retention.
Release and maintain it
Before release, reconcile displayed values to the signed analysis output, test filters and exports, check text alternatives and record known gaps. The owner approves the same version that users see.
Archive snapshots used for decisions, and keep any reader-facing wording consistent with the disclosure policy. Log each correction, its reason, the affected periods and the notification. Trigger review when a source, query, metric definition, classifier, geography rule, privacy decision or user need changes.
Before you act
- Name the decision, audience, refresh need and available action.
- Give every affected tile one of the four gap states.
- Connect every tile to a versioned metric card.
- Label the evidence type in each section.
- Withhold any tile whose lineage breaks.
- Test the interface and exports with users, and log corrections.
Common questions
What is the difference between a missing and an unavailable tile?
Missing means an expected field or source is absent. Unavailable means the source does not provide the measure, or access is unavailable. Both are labelled on the tile, because each tells the reader something different about what to do next.
Does a zero on a tile mean nothing was said?
Only when a defined population was observed and produced none. A zero does not mean a feed failed or a field was absent; those cases get the missing, delayed or unavailable state instead of a number.
When should a tile be withheld?
Withhold a tile when its lineage breaks, rather than estimating a convenient replacement. The data or definition cannot then be traced through its source path, so any substitute figure would have no card behind it.
