Date post: | 15-Apr-2017 |
Category: |
Business |
Upload: | jeffrey-funk |
View: | 1,081 times |
Download: | 0 times |
Anti-Hebbian and Hebbian
(AHaH) Computing
MT5009
Analyzing Hi-Technology
Opportunities
1. Chow Ka Yau Daniel A0145207M
2. Muhammad Dzahir Bin Mohamed Zain Affandi A0129428Y
3. Gregory Chee Ken Khyun A0132405W
4. Jayapathma Herath Madhushanka Meranjan A0132398Y
5. Lim Yee Hao Marcus A0132390N
http://www.riken.jp/en/research/rikenresearch/highlights/7918/
• Current worldwide buzz - Big Data analytics
• Future: Not just analytics but also
• Solutions: Predictive and Prescriptive analytics
• We need
- Machine learning
- Intelligent computing
- Large scale simulations
Problem: Current Von-Neumann computing bottleneck
We believe, AHaH Computing can break the Von-Neumann
bottleneck and open up a new era of big data analytics
Problem Statement
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
http://hartenstein.de/cited/Damian_Millers-Award.pdf
Von-Neumann (VN) Architecture
and Limitationshttp://sybaseblog.com/2013/05/0
6/need-of-in-memory-technology-
sap-hana/
Memory access speed
Processor speed
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
http://www.gridgain.com/wp-content/uploads/2014/09/insideBIGDATA-Guide-to-In-
Memory-Computing.pdf
- Keeping data in a server’s RAM instead
of hard disk or flash devices
- Massive parallelization for faster
processing speeds
- Inexpensive way to speed up enterprise
software applications, including but not
limited to analytics
Challenges:
- Requires lots of RAM!
- Unsustainable brute force method
since data volumes continue to
explode in a big data world
- High power consumption
- Processor and memory are still
separated
http://www.toddmace.io/
1. Algorithmic Approach: In-Memory Computing
Solutions to break VN bottleneck
Increase
bandwidth
Cache
Pre
fetching
Multi
threading
Parallel
processing
Pipelining
In-
Memory-
Computing
(IMC)
1. Algorithmic Approach: Software
http://tonycosentino.ventanar
esearch.com/
Solutions to break VN bottleneck
2. Neuromorphic Approach
http://www.slideshare.net/Funk98/neurosynaptic-chips
IBM’s True North
supercomputer incorporates
the largest neuromorphic
chip in the world, but the
chip is not capable of
learning on its own
Solutions to break VN bottleneck
Limitations No active machine
learning on chip
No unsupervised
learning
Required
supercomputer
http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-49d0-ac6c-4b4395525901&v=default&b=&from_search=1
Aim: to achieve the level
whereby it is able to do
what brain does –
processing and memory
are performed by the
same component
https://upload.wikimedia.org/wikipedia/commons/d/df/PPTExponentialGrowthof_Computin
g.jpg
The Future of Computing
http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-
49d0-ac6c-4b4395525901&v=default&b=&from_search=1
3. Self - Organizational Approach
AHaH computing is more than just the integration of
memory and processing!
Anti-Hebbian and Hebbian
(AHaH) Computing
Solutions to break VN bottleneck
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
• Creation of conductive path through a common
medium
• It just happens naturally (self organization)
• No need for external control to produce path of
conduction
• Occurs in rivers, air, blood system etc.
Bifurcation video
AHaH PhenomenonExamples of Natural Adaptive System
http://knowm.org/blog/
It is this
energy
dissipating
pathways
competing and
conducting for
resources
It is a Learning Process
AHaH PhenomenonBiology – Brain Synapse
- Strong Spikes Strengthen the connection
- Open up new pathways
- Hebbian Learning
- Weak Spikes (misfiring) weaken the
connection
- Delete up pathways
- Anti - Hebbian Learning
Model
• Firing strengthen pathways by increasing
synaptic weight change. This Adaptation
is called Hebbian learning
• Misfiring weaken the pathway connection
by decreasing synaptic weight change.
This Adaptation is called Anti - Hebbian
learning
Which component can help to mimic this synaptic adaptation ?
?
?
http://www.smashinglists.com/top-10-amazing-facts-
about-the-human-brain/http://www.intechopen.com/books/reinforcement_learning/interaction_between_the
_spatio-temporal_learning_rule__non_hebbian__and_hebbian_in_single_cells__a_c
MEMRISTORS
AHaH PhenomenonMemristors - Mimic Synaptic Adaptation
Analogous to the adaptive water pipeline
1) High pressure difference -> more water flow -> diameter gets bigger
2) Cut water supply -> diameter stays same – Remembers how much water has flown
3) Less pressure difference -> less water flow -> diameter gets smaller
4) Water flows in the opposite direction if diameter gets two small
– Erasing the path and flow is bi-directional
Replace the water flow with Current flow
Replace the pressure with Resistance
Synaptic Adaptation
Memristors adapts its resistance as current flows through the memristor
Non - Volatile memory
Remembers how much current flowed through the memristor
InputsOutput
Memristors
http://cacm.acm.org/news/33675-memristor-minds-the-future-of-artificial-intelligence/fulltext
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
http://www.hpcwire.com/2015/09/09/knowm-snaps-in-final-piece-of-memristor-puzzle/
AHaH Computing ArchitectureOverall Architecture
Connected
MemristorsConnected KT-
Synapse in Parallel
– AHaH node
Map Many KT-Cores
into RAM
architecture
KT-Core
• Connected AHaH Nodes in
Parallel
• Column decoder and Row
decoder to select AHaH Nodes
• Controller to control Instruction
flow into AHaH Nodes
Basic RAM ArchitectureAHaH-NodeINPUTS
Output
http://i.cmpnet.com/pldesignline/20
05/07/zeidmanfigure1.gif
AHaH Unites Memory and processingExample- A . B = C
CPU
MEMORY
Row
Decorder
Column Decoder
InputOutput
B
A C
Row
Select
Column
SelectRead Write
CPU
1) Inputs are
connected with
multiple coresKT-RAM
Activation of
KT-Cores
2)Activation
Instruction
Act of accessing the
memory Becomes the act
of configuring the
Memory
- Weight Change
occurs inside memristors
KT-Core
During
Activation,
AHaH
Controller
connects with
Multiple AHaH
nodes
InputA
AHaH
Von-Neumann
3) Adaptation
Instruction
- AND Logic
4) Data Out
Output
C• Creates logics within the memory using sequential Instructions flow
• Adaption happens for free, because memristors adopt as we use them
• No back and forth data transfer between memory and CPU
ALU
C
BC
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
Comparison of AHaH, Von-Neumann and
Neuromorphic ComputingArchitecture:
Conventional
Computing
Neuromorphic
Computing
AHaH
Computing
Architecture Von Neumann Neural Network AHaH Architecture
Computing Unit CPU Synaptic Chip Synaptic Chip
Storing Unit Memory Synaptic Chip Synaptic Chip
Storing Element DRAM DRAM/SRAM Memristors
Suitability Logical and Analytical Machine Learning
(pattern recognition)
Logical & analytical and
Machine learning
(pattern recognition)
Processing Serial Processing
(multi cores)
Parallel Processing Parallel Processing
Backward
Compatibility
Only in Von-Neumann
Architecture
Unable to use in von-
Neumann architecture
directly
(Require Supercomputer)
Able to use in both Von-
Neumann & AHaH
Architecture
Power Consumption High Low Ultra Low
Speed Slow Fast Fast
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
• Barrier potential inherently exists between
CPU and RAM especially when the it is
physically separated.
• Sufficient energy must be applied to
overcome this barrier potential
• Power between the CPU and RAM heavily
depends on the distance between the CPU
and RAM
• Each read operation lowers the switch
barriers. Thus, the act of accessing the
memory becomes the act of configuring
the memory over time
In brains, d = 0 which
would mean high
amounts of power
saving
AHaH Advantages: Lower power
https://www.youtube.com/watch?v=CFSrC7kjbJo
Less processing
requests and reply
between CPU and
memory due to a
internal machine
learning
AHaH Advantages:Power and Time Reduction
https://www.youtube.com/watch?v=CFSrC7kjbJo
More back-and-forth operations! Single package reply!
http://www.colocationamerica.com/blog/energy-wasting-data-centers
http://perspectives.mvdirona.com/2009/05/the-datacenter-
as-a-computer/
AHaH Advantages:Data Center Power Consumption
Servers and Storage
• Increasing energy consumption
per year due to increasing data
storage demands
• Higher energy consumption
equates to higher electricity cost
expenditure
http://www.itbusinessedge.com/info/PP-BuildDataCenter-pg9.aspx
AHaH Advantages: Machine Learning (Example)
http://image.slidesharecdn.com/bmvass2014breckonml-140709055858-phpapp01/95/machine-learning-fro-computer-vision-a-whirlwind-of-key-concepts-for-the-uninitiated-7-638.jpg?cb=1405575596
http://images.slideplayer.com/11/3288366/slides/slide_2.jpg
AHaH Advantages: Machine Learning (Example)
Example:
Rich person has a high confidence level of having
high education
Rich person has a high confidence level of being old
Sequence 3: Unsupervised learning
Sequence 1: Supervised Learning
Sequence 2: Assign Label for KT- Cores (rich and poor)
Model
Input
Spikes
https://www.youtube.com/watch?v=CFSrC7kjbJo
https://www.youtube.com/watch?v=CFSrC7kjbJo
http://knowm.org/thermodynamic-ram-
technology-stack-published/
AHaH Advantages: Flexibility of AHaH
• Integration AHaH circuits with existing
integrated circuits technology by
apply it at the end of the process line
at the very end
• Front-end-of-line refers to the current
integrated circuits.
• Back-end-of-line refers to AHaH
circuit would be fabricated on top of
it.
• Able to take advantage of an
existing process that is working very
well and bringing it to the next level
of performance
• Create new computers that adapts as
it is usedhttps://www.youtube.com/watch?v=w7q07eKPM9U
AHaH
circuits
Current
integrated
circuits
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
“Most Memristors that I have seen do not behave like fast, binary, non-volatile, deterministic switches. This is a problem because this is how HP wants them to behave”
• Alex Nuegent – Lead Inventor and CEO of Knowm
HP’s Memristor Problem
The incumbents are limiting the applications of memristors
within the existing memory technology framework
Limit the use of memristor
http://knowm.org/the-problem-is-not-memristors-its-how-hp-is-trying-to-use-them/
https://indico.cern.ch/event/345619/session/1/contribution/10/attachments/681170/935777
/HW_trends_market_costs_BPS_Apr2015_v14.pdf
Memristors – Memory Trends: Cost
NVRAM become cheaper
NVRAM vs SRAM
• Closing gap
• IBM’s synapse chips uses SRAM
• Potential opportunity to expand AHaH computing market (memristor –NVRAM)
Memristors - Memory Trends: Storage Capacity
ReRAM ReRAM vs other emerging NVRAM
• Highest capacity over others
http://www.maltiel-consulting.com/ISSCC-2013-Memory-trends-FLash-NAND-DRAM.html
http://www.tomsitpro.com/articles/flash-data-center-advantages,2-744-3.html
Memristors - Memory Trends: Manufacturability
ReRAM vs other emerging NVRAM
• The only memory compatible for mass production
http://www.storagenewsletter.com/rubriques/market-reportsresearch/non-volatile-memories-yole/
NVRAM Trend AnalysisCurrent NVRAM Suppliers
ReRAM
Potential
Market
Non-Memristor based Memristor based
BIG DATA
Analytics
2015 onwards, prediction of the mass manufacturing of Memristor to be availableThis will bring forth the further improvement of AHaH computing architecture where AHaH synaptic chip can be produced based on memristors
http://www.reram-forum.com/2013/03/21/predicting-the-reram-
roadmap/
Future opportunity for Memristor production
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data
Definition of Big Data
Big Data 3V Model
Definition of Big Data
http://www.csc.com/insights/flxwd/78931-
big_data_universe_beginning_to_explode
The global production of data
is expanding and reach
~40ZB by 2020
https://www.capgemini.com/blog/capping-it-
off/2014/07/are-you-effectively-using-big-data
> 85% of an
organization’s data
is unstructured
Time and energy
consuming to
process unstructured
data
https://web-assets.domo.com/blog/wp-
content/uploads/2014/04/DataNeverSleeps_2.0_v2.jpg
Data velocity is
measured against
time
Enable real time
streaming
processing
Volume, Variety & Velocity
http://wikibon.com/wp-content/uploads/kalins-pdf/singles/big-data-vendor-revenue-and-
market-forecast-2011-2026.pdf
Wikibon forecasted Big Data
market to have 17% Compound
annual growth rate over 15 years
(2011-2026)
McKinsey Global Institute, Game changers: Five
opportunities for US growth and renewal, July 2013
Big Data identified as one of the
game changer that can boost US
annual GDP by 2020
Big Market for Big data
Business Intelligence
comprises of tools and
methodology for data
analyzing
Data / Big Data Analytics
can be grouped under
Business Intelligence
Past in Nature:
Descriptive and Diagnostic
Analytics
Future in Nature:
Predictive and
Prescriptive Analyticshttp://www.fyisolutions.com/blog/advanced-analytics-seminar/
Big Data Analytics
The Bottleneck is in technology
Not only need new algorithms and techniques but
breakthrough computing architecture
The Big Hurdle
http://image.slidesharecdn.com/finalpresentation-150305004602-conversion-gate01/95/presentation-on-big-data-analytics-15-638.jpg?cb=1425516438
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
http://ayata.com/stage/wp-content/uploads/ayata-infographic-2012-09-04.jpg
Big Data Analytics
ApplicationsHealthcare Industry
Given agility to government
to combat flu epidemic
dealing with vaccine
production/delivery rate vs
outbreak numbers in various
states
Autonomous Vehicle
Google AV to recognize &
anticipate what might be
coming in real time at a
junction
Oil & Gas Industry
Chevron need to analyze 50
terabytes of seismic data
Drilling miss cost USD$100M
Retail Industry
Starbucks marketing strategy
aligned to real time data
and responses
CONTENTS1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion
CONCLUSION
• Big market and growth of Big Data applications
• Von-Neumann architecture bottleneck is hitting the limits
• Cutting edge of AHaH computing architecture
• Real time processing (Integrated memory & processing)
• Ultra less power consumption and less heat generate
• Self-Organized approach
http://ekvv.uni-bielefeld.de/bilddb/bild?id=87240
ANY QUESTIONS?
For further readings on AHaH computing, please
visit www.knowm.org
(Startup for AHaH – started July 2015)