Date post: | 15-Apr-2017 |
Category: |
Data & Analytics |
Upload: | omid-mogharian |
View: | 37 times |
Download: | 0 times |
Hadoop Essential SetupA big data proposal
Who is thisweirdo?
Hiking, Table Tennis, Kicker, Traveling, Foods, Cultures, The BigbangTheory, Family Guy, Sherlok, …
What's Big Data Really?
There are only two hard things in Computer Science: cache invalidation and naming things.
Phil Karlton
Hadoop?
From Wikipedia
The genesis of Hadoop came from the Google File System paper ..... This paper spawned another research paper from Google – MapReduce: …. in January 2006. Doug Cutting, who was working at Yahoo! at the time, named it after his son's toy elephan
Map/Reduce
Shuffle(Transfer & Merge)
HDFS
By the wayHadoop 2.0
HadoopEcosystyem
ETL or ELT? ?
Source
TargetTarget
Source
Source
Essential Setup
+
Flume Agent
Flume Agent
Flume Agent
HDFSHDFS
HDFSHDFS
Pig
Hive
SparkHbase
Essential Setup
Essential Setup
Fast Data & Big Data
Lambda Architecture
Fast Data & Big Data
Now your
Turn, What's
your idea?