Date post: | 15-Dec-2015 |
Category: |
Documents |
Upload: | chetan-surpur |
View: | 28 times |
Download: | 4 times |
Building Intelligent MachinesChetan Surpur
MVTSA Technology Symposium – April 24, 2015
Agenda
• Machine Intelligence• The human neocortex• Research and applications• Future research
Agenda
• Machine Intelligence• The human neocortex• Research and applications• Future research
Agenda
• Machine Intelligence• The human neocortex• Research and applications• Future research
What Does the Neocortex Do?
Data streamretina
cochlea
somatic
The neocortex learns a model of the world, primarily through behavior.
Sensory arrays
Motor stream
The model is time-based and predictive.
• Uniform, heterogeneous• Plasticity in learning• Different regions are merely connected to different inputs and outputs• Hierarchical, like the world it models• Single learning algorithm
Cortical Theory
Hierarchy
Cellular layers
Mini-columns
Neurons: 5-10K synapses
Active dendritesLearning = new synapses
Remarkably uniform - anatomically - functionally
Sheet of cellsHTMHierarchical Temporal Memory
1) Hierarchy of identical regions2) Each region learns sequences3) Stability increases going up
hierarchy if input is predictable4) Sequences unfold going down
Questions
- What does a region do?- What do the cellular layers do?- How do neurons implement this?- How does this work in hierarchy?
2/3
4
65
Agenda
• Machine Intelligence• The human neocortex• Research and applications• Future research
2/3
4
5
6
Numenta Research Roadmap
Sensory-motor Sequences
High-order Sensory Sequences
Motor Sequences
Attention/Feedback
Streaming Data Applications
Capabilities: PredictionAnomaly detectionClassification
Extensively tested 2011-2013Commercial
Tested 2014
In research 2015
HTM
SDRs PredictionsAnomaliesClassification
Streaming Data Applications
Data stream
NumbersCategoriesDateTimeGPSWords
ApplicationsServersBiometricsMedicalVehiclesIndustrial equipmentSocial mediaComm. networks
Encoder
Turns data into Sparse Distributed
Representations(SDRs)
Grok: Anomaly Detection For Amazon Web Services
Unique value of HTM algorithms Automated model creation: configure hundreds of models in
minutes Continuously learning: automatically adapts to changes Detects sophisticated temporal anomalies
Anomaly in Geospatial Tracking Data (demo)
CLA
Encoder
SDRsPredictionAnomaly DetectionClassification
GPS+ Velocity
Encoder turns GPS data into SDRsWorks anywhere on Earth or in space
Anomaly in Geospatial Tracking Data (demo)
Direction anomaly
Learning a route
Speed anomalyPosition anomaly
Stock priceStock volumeTwitter volume
Companies sorted by unusual activity
Tweets reveal cause
Company monitorFull application mid 2015, free, open source
Document corpus(e.g. Wikipedia)
128 x 128
“Word SDRs”
- =
Apple Fruit Computer
MacintoshMicrosoftMacUnixOperating system….
Natural Language +
Training set
eats“fox”
rodent
1) Unsupervised Learning
2) Semantic Generalization
3) Many Applications
frog eats fliescow eats grainelephant eats leavesgoat eats grasswolf eats rabbitcat likes ballelephant likes watersheep eats grasscat eats salmonwolf eats micelion eats cowdog likes sleepelephant likes watercat likes ballcoyote eats rodentcoyote eats rabbitwolf eats squirreldog likes sleepcat likes ball---- ---- -----
Sequences of Word SDRs
HTM
Server metric anomalies
Geospatial tracking
Natural languagesearch/prediction
Company monitor
These HTM Applications Use Exact Same Code Base
Agenda
• Machine Intelligence• The human neocortex• Research and applications• Future research
2/3
4
5
6
Numenta Research Roadmap
Sensory-motor Sequences
High-order Sensory Sequences
Motor Sequences
Attention/Feedback
Extensively tested 2011-2013Commercial
Tested 2014
In research 2015
NuPIC open source project www.Numenta.org
Research Transparency
- Algorithms are documented
- Multiple independent implementations
- Numenta’s software is open source (GPLv3)
- Numenta’s daily research code is online
- Active discussion groups for theory and implementation
Thank you!Questions?
Backup Slides
Sensory-motor sequences
Sensory-motor sequences
Sensory-motor sequences
2/3
4
5
6
Goal Oriented Behavior via Feedback
Copy of motor commands
Sensor/afferent data
simple cells
complex cells
Stable feedback invokes union of sparse states in multiple sequences. (“Goal”)
Feedforward input selects one state and plays back motor sequence from there.
Sub-corticalmotor
simple cells
complex cells
Stable/Invariant patternfrom higher region
Higherregion
Cop
y o
f moto
r co
mm
an
d