Predicting Norovirus with Twitter

transcript

Social Media Analytics Review and

Innovation Group

30/09/2015 Callum Staff

Agenda

1. Welcome, Introductions and Purpose

2. Governance Arrangements

3. Predicting Norovirus from Twitter

4. Social Media Research Project Guidance

5. Government Social Research Social Media Research Ethics Guidance

6. Relationships between Consumers and Food Business Operators

7. Value of Social Media Data in Policy

8. Future Meetings: Frequency and Content

Predicting Norovirus Rises with Twitter

30/09/2015 Callum Staff

Contents

• Background

• Method

• Model Applications

• Take Home Points

• Using Social Media Data

• Next Steps

BACKGROUND

BACKGROUND: Social Media As An Analytical Tool

• Does this new data source add value to our current knowledge?

• Public Health England – syndromic surveillance

• FSA Social Media Team – human observed monitoring

• Added value = knowing early there is a rise in cases

• Earlier we know = earlier we can intervene

METHODS

METHOD: Crowd-Sourcing Keywords

METHOD: CorrelatingJa

Lab Reports Sickness Tweets

METHOD: Correlating – Raw Values or Changes?

• Correlations between raw values – not indicative of whether a rise is going to

• Raw values stronger correlations than changes week to week

• Changes are calculated between fortnights not weeks because week to

week changes are too small

Correlations for #sicknessbug Raw Values 1 Week Changes 2 Week Changes

0.50 0.29 0.43

METHOD: Lagging the Data

Tweets

Lab Reports

METHOD: Lagging the Data

Tweets

Lab Reports

METHOD: What’s a Significant Change?

• Practically – any rise which is outside the normal noise

• On the model – any change in the top quartile

• Arbitrary

• Could do machine learning to look at what significant change classification

lead to the model being most accurate

METHOD: What’s a Significant Change?

Jan-12Fe

Mar-12

May-12

Jun-12Jul-1

p-12Oct-

Dec-12Jan

b-13Apr-1

May-13

Jul-13

Aug-13Se

Nov-13

Dec-13Jan

Mar-14Apr-1

4Jun-14

Jul-14

Aug-14Oct-

600Lab Reports Actual Sig. Change

METHOD: Logistic Regression Model

Given changes in Tweet volumes between weeks 1 and 3, is the change

in lab reports between weeks 4 and 6 significant?

• Significant Change = 1, Non-Significant Change = 0

• Uses exponential formula with Tweet volumes as parameters to give

probability

• Probability can be assigned to either of the binary categories based on a

predefined threshold (typically 0.5)

METHOD: Adjusting for Project Requirements

• Receiver Operating Characteristic Curve

• Adjusting the threshold = Adjusting # of true/false positives and true/false

negatives

• Want to increase the number of true positives in order to achieve early

detection

• Willing to sacrifice the model picking up false positives in other places

• Early warning system, not a call to arms

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1-Spec

Specificity:

Sensitivity:

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1-Spec

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1-Spec

Specificity:

Sensitivity:

MODEL APPLICATIONS

MODEL APPLICATIONS: Training and Testing

• Difficulty in that only had 2 and a half years of data

• Periods within this dataset where there were Twitter drop

outs/discontinuous lab reports

• Test set is current real time Tweeting

• Will review at the end of the Norovirus season (April)

MODEL APPLICATIONS: Final Predictive Model

MODEL APPLICATIONS: The Intervention

• Higher risk of project means low resource intensity required

• Needs to be easily deployable – match volatile nature of social media

• Using delivery partners:– NHS Choices – Elderly in hospitals/Care homes– Department for Education – Schools– FSA Comms Team – Food handlers

• Social/online media and contact with advocates in above sectors

USING SOCIAL MEDIA DATA

USING SOCIAL MEDIA DATA: Representativeness

• Tweeting Population versus Affected Population

TAKE HOME POINTS &

NEXT STEPS

TAKE HOME POINTS: Analytical/Comms Trade Off

• Variable correlations versus giving comms time to act

• Model accuracy versus early warning

• Choice of datasets

NEXT STEPS: Geotagging

QUESTIONS?callum.staff@foodstandards.gsi.gov.uk