Future of Artificial Intelligence- Memory, Knowledge, and Language
Hang Li
Noah’s Ark Lab
Huawei Technologies
Global Artificial Intelligence Technology ConferenceBeijing, May 21, 2017
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Life without Memory - Tragic Story of Clive Wearing
• A musician in UK
• Suffering from amnesia (失忆症), after
contracting encephalitis
• Symptom: lacks ability to form new memories, and also cannot recall most of his past memories
• Can speak, eat, dress himself, walk, ride bike, even play the piano
• Can remember his wife’s name, but cannot remember his daughter’s name
• Only has moment-to-moment consciousness
Clive Wearing
Model of Human Brain
Modified from Frank Longo 2010
Sensory Register• vision 1 sec• auditory 5 sec
Short-term Memory• 18-30 sec• 7±2
• displacement
Long-term Memory
• synapse mod• limitless cap• declarative and non-declarative
Attention Consolidation
RetrievalCentral Executive Unit• planning• conscious thought
visualauditorykinaestheticolfactorygustatory
Characteristics of Long-term Memory
• Storing information and knowledge (not data)
• New content can be continuously added
• New content is associated with existing content
Current AI Systems Are All ‘Memoryless’
• Do not have long-term memory
• Repeatedly utilize learned functions
• Cf., Clive Wearing
)(
)(
xg
xf
AlphaGo
Self Driving Car)(xf
)(xg
)(xh
Loop of functions Finite state of functions
AI Systems Will Evolve to Next Level, When Having Long Term Memory
• Do AI Systems Have Consciousness?
• There is no consensus on consciousness
• Kaku: Consciousness means that the system changes internal states in interaction with environment
• In that definition, thermostat, sunflowers, etc have ‘consciousness’; AI systems also have ‘consciousness’
• AI systems will have `continuous consciousness’ with long term memory
Michio Kaku
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Information and Knowledge
Wisdom
Knowledge
Information
Data
• Information: facts about people, objects, and events
• Knowledge: theoretical or practical understanding of a subject
• Plato: Knowledge = Justified True Belief (JTB)
• No clear boundary between information and knowledge
• Most of our information and knowledge is acquired and learned from others
Intelligent Information and Knowledge Management System with Long Term Memory
Language Processing Unit
Short-term Memory
Long-term Memory
Consolidation
Central Executive Unit
Analysis
Q2
A1
Learning Phase
Q1
A2
… …
Intelligent Information and Knowledge Management System with Long Term Memory
Language Processing Unit
Short-term Memory
Long-term Memory
Retrieval
Central Executive Unit
Analysis
Q
A
Use Phase
Characteristics and Current Status
• Continuously accumulates information and knowledge
• Properly performs question answering in natural language,
– Answers when it knows
– Says “I don’t know”, when it does not know
• Ideally, system is fully automatically constructed without human involvement
• Becomes intelligent assistant of human
– Note that computer has two powerful capabilities, computing and storage
• Currently only partially realized, cf., search engine
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Knowledge Is Not Categorical- Example: Bachelor
• Bachelor: unmarried adult male
• How to judge the following?
– Unmarried father of child
– Man having fake marriage
– 17 year old high school student
– 17 year old playboy
– Homosexual lovers
– Arabic man with two wives to meet another fiancee
– Bishop
• From Terry Winograd
• Arthur has been living happily with Alice for the last five years. They have a two-year-old daughter and have never officially married.
• Bruce was going to be drafted, so he arranged with his friend Barbara to have a justice of the peace marry them so he would be exempt. They have never lived together. He dates a number of women, and plans to have the marriage annulled as soon as he finds someone he wants to marry.
• Charlie is 17 years old. He lives at home with his parents and is in high school.
• David is 17 years old. He left home at 13, started a small business, and is now a successful young entrepreneur leading a playboy's lifestyle in his penthouse apartment.
• Eli and Edgar are homosexual lovers who have been living together for many years.
• Faisal is allowed by the law of his native Abu Dhabi to have three wives. He currently has two and is interested in meeting another potential fiancee.
• Father Gregory is the bishop of the Catholic cathedral at Groton upon Thames.
Language Is PolysemousExample: Climb
• The boy climbed the tree.• The locomotive climbed the
mountainside.• The plane climbed to 30,000 feet.• * Smoke climbed from a chimney.• * An elevator climbed from one
floor to another.• The temperature climbed into the
90s.• Prices are climbing day by day.• The boy climbed down the tree
and over the wall.• We climbed along the cliff edge.• * The locomotive climbed over the
mountain.• He climbed out of a sleeping-bag.
18
• Climb: motion from lower level to higher level, along a path, by laborious manipulation of limbs
• Features: [ascend] [clamber]
• Climb is polysemous category consisting of several senses
• The senses are related through meaning chain A-B-C-D
• From Charles Fillmore
Language Is SynonymousExample: Distance between Sun and Earth
• distance from earth to the sun
• distance from sun to earth
• distance from sun to the earth
• distance from the earth to the sun
• distance from the sun to earth
• distance from the sun to the earth
• distance of earth from sun
• distance between earth sun
• "how far" earth sun
• "how far" sun
• "how far" sun earth
• average distance earth sun
• average distance from earth to sun
• average distance from the earth to the sun
• distance between earth & sun
• distance between earth and sun
• distance between earth and the sun
• how far away is the sun
from earth
• how far away is the sun
from the earth
• how far earth from sun
• how far earth is from
the sun
• how far from earth is
the sun
• how far from earth to
sun
• how far from the earth
to the sun
• distance between sun
and earth19
Combination of Neural Processing and Symbolic Processing
Easy to Interpret
Easy to Manipulate
Able to Deal with
Uncertainty
Robust to Noise
Symbolic Representation Neural Representation
Neural Symbolic Processing
Neural Symbolic Processing for Information and Knowledge Management
Central Executive Unit
Q
A
Short-Term Memory Long-Term Memory
Q A
Knowledgein symbolic representation & neural representation
Sym.Neu.
Language ProcessingModel
Encoder
Decoder
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Never Ending Language Learning (NELL)
• Task
– Initial ontology, few examples of each category, the web, occasional interaction from humans
– Extract more facts from web
– Learn to read better than before
• System
– Knowledge base with 15 million candidate beliefs
• Technologies
– Coupled semi-supervised learning, automatic discovery of new coupling constraints, automatic extending of ontology, staged curriculum
Mitchell et al. 2015
Memory Networks
• Long term memory + inference
• Model is learned
• Can answer factoid questions
• Acc = 40%+
• Example– John is in the playground.
– Bob is in the office.
– John picked up the football.
– Bob went to the kitchen.
– Q: where is the football?
– A: playground
memory
pre-process
generate
matchq
a
i
o
im
Weston et al. 2014
Differentiable Neural Computers
• DNC = neural network + external memory (matrix)
• Memory represents complex data structures
• Neural network, learned from data and supervised learning, controls access to memory
• Memory heads use three forms of differentiable attention
• Resembling mammalian hippocampus functions
Graves et al. 2016
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Question Answering from Knowledge Graph
(Yao-Ming, spouse, Ye-Li)
(Yao-Ming, born, Shanghai)
(Yao-Ming, height, 2.29m)
… …
(Ludwig van Beethoven, place of
birth, Germany)
… …
Knowledge Graph
Q: How tall is Yao Ming?
A: He is 2.29m tall and is visible from space.(Yao Ming, height, 2.29m)
Q: Which country was Beethoven from?
A: He was born in what is now Germany.(Ludwig van Beethoven, place of birth, Germany)
Question Answering
System
Q: How tall is Liu Xiang? A: He is 1.89m tall
Learning System
Answer is generated
GenQA
Encoder
Decoder
Short-Term Memory Long-Term MemoryLanguage Processing Module
Q
A
Q’
A’
Triples in symbolic representations (indexed) & neural representations
IndexTriples
Encoder creates question representation, decoder generates answer
Matches and retrieves most relevant answer representation
End-to-End Training
Question Answering from Relational Database
Relational Database
Q: How many people participated in the game in Beijing?
A: 4,200
SQL: select #_participants, where
city=beijing
Q: When was the latest game hosted?
A: 2012
SQL: argmax(city, year)
Question Answering
System
Q: Which city hosted the longest Olympic game before the game in Beijing? A: Athens
Learning System
year city #_days #_medals
2000 Sydney 20 2,000
2004 Athens 35 1,500
2008 Beijing 30 2,500
2012 London 40 2,300
Neural Enquirer End-to-End Training
Encoder
Short-Term Memory
Long-Term Memory
Language Processing Module
Q
A
Q’
A’
Features and values are in symbolic representations and neural representations
Encoder creates question representation, decoder simply returns answer
Matches question representation to table representations to find answer
Decoder
Matching
Neural Enquirer
• Five layers, except last layer, each layer has reader, annotator, memory • Reader fetches important representation for each row• Annotator encodes result representation for each row
Outline
• Memory and Intelligence
• Intelligent Information and Knowledge Management System
• Neural Symbolic Processing
• Related Work
• Our Work
• Summary
Summary
• Long term memory is indispensable for human intelligence
• AI systems will evolve to next level with long term memory
• Intelligent Information and Management System
– Can automatically acquire information and knowledge
– Can correctly answer questions from humans
• This should be most important topic for research in AI
• Neural Symbolic Processing should be most promising approach
• Recent research is making progress
• Many open questions and challenges