Date post: | 22-Jan-2018 |
Category: |
Data & Analytics |
Upload: | crowdsourcing-week |
View: | 231 times |
Download: | 2 times |
Daniela Braga, PhDCEO
DefinedCrowd: Crowdsourcing,
Speech Data Science, AI
Crowdsourcing Week, June 15th 2017
definedcrowd confidential 15
The challenges of crowdsourcing NLP data
Crowd quality Data quality
• Language tests• Job specific tests• Real Time Audits• Built-in language/spam
validators
• Referral system• System of tokens• Legal/privacy compliance
(under NDA)
Quality gateways
Controlled crowd
• Checking for suspicious crowd behavior (multiple accounts creation, peaks of activity, specific job spam, IP check against country of living)
Machine Learning
Data quality control
• Validation steps• Inter-annotator
agreements• Precision and Recall
metrics
definedcrowd confidential 16
DefinedCrowd combines the best of professional services with SaaS companies