Post on 22-Jan-2018
transcript
NATURAL LANGUAGE UNDERSTANDING WITH MACHINE LEARNED ANNOTATORS & DEEP LEARNED ONTOLOGIES AT SCALE
David TalbyPh.D., MBA, CTO @ Atigeo
The problem
Whoneedstobevaccinated?
Whofitsthisclinicaltrial?
Whoisatriskforsepsis?
Whoisgettingmedsthey’reallergicto?
Whoonthisprotocoldidnothavethisside
effect?
At the beginning, there was search
Scalable&robustIndexingpipelineTokenizers&analyzersSynonyms,spellers&Auto-suggestFileformats&headerboostingRankers,link&reputationboosting
Then there was semantic search“cheap red prom dresses”“laptops under $500”“italian restaurants near me that deliver”“captain america civil war tonight”“nba scores”
DictionaryBasedAttributeExtraction
Dell - XPS 15.6 4K Ultra HD Touch-Screen Laptop - Intel Core i5 - 8GBMemory - 256GB Solid State Drive -Silver
MachineLearnedAttributeExtraction
If you go for the ambience, you'll bedisappointed. If you go for good,inexpensive and authentic Mexicanfood, then you're in the right place.
Then, you need to understand languagePrescribing sick days due to diagnosis of influenza. Positive
Jane complains about flu-like symptoms. Speculative
Jane may be experiencing some sort of flu episode. Possible
Jane’s RIDT came back negative for influenza. Negative
Jane is at high risk for flu if she’s not vaccinated. Conditional
Jane’s older brother had the flu last month. Familyhistory
Jane had a severe case of flu last year. Patienthistory
Human language is wonderfully nuancedJoe expressed concerns about the risks of bird flu. Nothing
Joe shows no signs of stroke, except for numbness. DoubleNegative
Nausea, vomiting and ankle swelling negative. Compound
(itgetsworse– inrealityalotoftextisn’tvalidEnglish)
Patient denies alcohol abuse. Speculative
Allergies: Penicillin, Dust, Sneezing. Compound
SENTENCEDETECTION
SECTIONDETECTION
TOKENIZER LEMMATIZER
STOPWORDREMOVAL
NEGATIONDETECTION
CONDITIONALSCOPE
SPECULATIVESCOPE
DATE NUMBER UNIT QUANITITY
CONCEPTEXTRACTION
Machine learned annotators
GrammaticalPatterns
If…then…
DirectInferences
Age<18==>Child
Lookups
RIDT(labtest)
Under-diagnosedconditions
FluDepression
ImpliedbyContext
relevantlabsnormal
Sometimes,it’seasiertojustcodeanannotation’sbusinesslogic
Butsometimesit’seasiertolearnitfromexamples:
Summary: How Summary: Why
1. Languagegetscomplexanddomainspecific
2. You’llneedmachinelearningearly
3. Bootstrap&thenexpandyourvocabulary
Whoneedstobevaccinated?
Whofitsthisclinicaltrial?
Whoisatriskforsepsis?