Big Data Theory and Case Study

7/24/2019 Big Data Theory and Case Study

1/59

1. Big Data, whats so Big Thing?


2/59


First, Big data is really Big


3/59


First, Big data is really Big - this mean VOLUME


4/59


First, Big data is really Big - this mean VOLUME

IDC in Digital Univers Study estimated in December 2012 a volume of 10.000Exabytes for 2015 and 40.000 Exabytes in 2020. A 50 fold increase in data volum

Health - Modern health information systems can generate several Exabytes ofpatient data, the so called "Health Big Data", per year.Source: InternationalJournal of Big Data Intelligence -

Health Big Data Analytics: Current PerspectiveChallenges and Potential Solutions - by Alex Mu-Hsing Kuo, Tony Sahama, Andre

Kushniruk, Elizabeth Borycki, Daniel Grunwell
http://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdi


5/59



6/59


Second, those data are coming from an extreme VARIETY of sources

Internal sources: ERP,CRM,WMS, Spreadsheets,

Social media, Facebook, Twitter

Video content: YouTube

Personal wearable personal health monitors

Healthcare data

Geospatial data

Sensors

etc.


7/59


Massive data streaming analysisthe VELOCITY

NYSE - NYSE Technologies receives four to five terabytes of a data a day anduses it to do complex analytics, market surveillance, capacity planning andmonitoringSource: Forbes

Every Car has more than 100 sensor and in Formula 1 for a car this number ismultiplied by a factor of 100.
http://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the-data-deluge-overwhelms-traditional-databases/?sf9689013=1http://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the-data-deluge-overwhelms-traditional-databases/?sf9689013=1


8/59


Those data are uncertainVERACITY

refers to the biases, noise and abnormality in data

Is about the trustworthy of data

1 of 3 business leaders dont trust the information they use to make decision


9/59



10/59



11/59

2. What can Big Data do for us?


12/59


Big data itself can do NOTHINGor very few but.


13/59


Big data itself can do NOTHINGor very few but.

Big Data ANALYTICS can do a lot!


14/59

2. What can Big Data do for us?Data vs. Information


15/59



16/59



17/59


This is what Data Analytics does:

Transform DATA in VALUABLE INFORMATIO


18/59


Dataraw facts usually

Some differences between data and information:

Data is used as input for the computer system. Information is the output of data.

Data is unprocessed facts figures. Information is processed data.

Data doesnt depend on Information. Information depends on data.

Data is not specific. Information is specific.

Data is a single unit. A group of data which carries news are meaning is calledInformation.

Data doesnt carry a meaning. Information must carry a logical meaning.

Data is the raw material. Information is the product


19/59


In a single phrase:

Information is interpreted, processed,

organized, presented data (with meaning)in a specific context.


20/59

3. Big Datas Pros and Cons

Pros:

Can improve every aspect of our li fe Can help us to find faster, treatments for uncured disease like: Cancer, Cardio vascular diseas

Alzheimer, etc.

Improvement in healthcare

Many health managers and experts believe that with the data, it is possible to easilydiscover useful knowledge to improve health policies, increase patient safety andeliminate redundancies and unnecessary costs. The objective of this paper is to discuss tcharacteristics of Health Big Data as well as the challenges and solutions for health BigData Analytics (BDA) the process of extracting knowledge from sets of Health Big Dataand to design and evaluate a pipelined framework for use as a guideline/reference inhealth BDA.Source: International Journal of Big Data Intelligence - Health Big DataAnalytics: Current Perspectives, Challenges and Potential Solutions - by Alex Mu-Hsing Ku

Tony Sahama, Andre Kushniruk, Elizabeth Borycki, Daniel Grunwell Can boost economy.

Will create new jobs

Can help us combat terrorism

Can help us reduce fiscal fraud

Etc.
http://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/info/ingeneral/forthcoming.php?jcode=ijbdihttp://www.inderscience.com/jhome.php?jcode=ijbdi


21/59


Cons:

Can make our life a chaos/havoc.

The Privacy will become a big challenge and issue.

Will change and eliminate old jobs increasing unemployment for poor skilledpopulation, generating necessity to requalify them more frequently.

Can help terrorist organizations also ( as can do with anti-terrorism fight)they alshave access to this technologies


22/59


Conclusion:

Big Data, will impact all of the human (and not only) life aspects Political,Economic/Financial, Social and Technological. Every aspect of our life will bechanged by this avalanche of data.


23/59

4. Big Data, Data Analytics andArtificial Intelligencethe (not so)

missing link

The three era of Analytics: (Institute of International AnalyticsIIA, SAS, etc)


24/59


missing link Analytics 1.0 (1954-2009) Data sources were relatively small and structured, and came from internal source

Data had to be stored in enterprise warehouses or marts before analysis;

The great majority of analytical activity was descriptive analytics, or reporting;

Creating analytical models was a batch process often requiring several month

Quantitative analysts were segregated from business people and decisions in

back rooms;

Very few organizations competed on analyticsfor most, analytics weremarginal to their strategy.


25/59


missing link Analytics 2.0from 2005 to 2012:

Data was often externally-sourced, and as the big data term suggests, was eithevery large or unstructured. The fast flow of data meant that it had to be stored anprocessed rapidly, often with massively parallel servers running Hadoop. The overspeed of analysis was much faster.

The new generation of quantitative analysts was called data scientists,

appeared


26/59


missing link Analytics 3.0starting nowadays

Analytics is driving all operational and strategic decisions and also products andservices for the companies.

Mix and synthetize the Analytics 1.0 and 2.0

In-memory analytics

Near just in time analytics

May require creation of Chief Analytics Officer roles or equivalent


27/59


missing link Analyze types: Descriptive Analytics: which report on the past;

Predictive Analytics: which use models based on past data to predict thefuture

Prescriptive Analytics: which use models to specify optimal behaviors andactions

Data mining

Machine Learning yes AI is coming into the game slowly but sure.


28/59

5. Big Data and Retail Revolution -The business value of Big Data

General Business Value of Big Data for all companies that wish to survivand compete in a data-driven economy.

Could increase turnover and profitability

Could reduce costs

Could optimize all business process

Could create a better focused customer approach


29/59

5. Big Data and Retail Revolution -The business value of Big Data Retail Business Value

Creating a personalized shopping experience Increase the precision of customer segmentation by analyzing customer

transactions and shopping behavior patterns across all retail channels.

Enrich your understanding of customers by integrating multichannel datafromonline transactions to social media and third-party datato develop a 360-degreview of each individual and identify emerging trends.

Optimize customer interactions by knowing where a customer is and deliveringrelevant real-time offers based on that location.

Predict consumer shopping behavior and offer relevant, products to influencecustomers to expand their shopping list. Even before they know that they need thproduct.

Using eyes sensors to offer special prices for the products that his look is pointing tand the pattern of the face/eyes expressions/look that could be interested but sthesitating.


30/59

5. Big Data and Retail Revolution -The business value of Big Data Retail Business Value

Enabling operational excellence

Predict optimal pricing and maintain a price leadership position by analyzing pricand demand elasticity.

Select the right merchandise for each channel and fine-tune local assortmentplanning by drawing on insights from social media, market reports, internal salesdata and customer buying patterns.

Optimize inventory across multiple channels by using leading indicators such ascustomer sentiment and promotional buzz to anticipate future demand.

Fine-tune store planograms by analyzing customer buying patterns and purchasintrends.

Improve logistics by using real-time traffic, weather data and more to re-routeshipments and avoid costly delays.


31/59


Retail Business Value

Optimizing merchandising and supply chains

Optimize staffing levels by predicting changes in customer demand.

Better match employee skills with retail store needs and create the right incentiveto drive strong sales performance.

Facilitate better-informed financial decision making by drawing on complete,

trustworthy and timely data from a wide array of sources. Improve fraud detection by analyzing large volumes of transactions.


32/59


Retail Business Valueconclusion:

Discovering the value of implementing big data solutions

Leading retailers are already discovering the tremendous value of implementingsolutions designed to analyze, organize and apply big data.

Creating a data-driven retail enterprise

Keeping retail focused on the customer with a 360 degree view and approach.


33/59

5. Big Data and Retail Revolution -The business value of Big Data Retail Business Valueconclusion:

Keeping retail focused on the customer with a 360 degree view and approach.


34/59

6. Big Data, Cloud and Mobility thehottest triad

Big Data Volumes needs Big Data processing capabilities

Cloud Computingthe solution by at least two of his features:

Rapid Elasticitymore power only when we need, as long as we need it

Measured servicespay for what you really use

Challenges of cloud computing:

SLAs

Portability

Audit

Mobility will allow users to access big data analytics applications whenand where they need them in such fast moving world.


35/59

6. Big Data Technology, Approachand Scenarios:

Technologies:

Open Source: hadoop, NoSQL, GRAPH DATABASES, mongoDB, Cassandra,etc

Proprietary: IBM SPSS/InfoSphere, SAS, SAP HANA, TERADATA, MICROSOFT,ORACLE, Revolution Analytics, etc


36/59


Big data ecosystem:


37/59


Big data ecosystem:


38/59


Enterprise Scenarios

Enterprise Scenario :

6 i


39/59



Enterprise Scenario 1: stage structured data

6 Bi D t T h l A h


40/59



Enterprise Scenario 2: process structured data

6 Bi D t T h l A h


41/59



Enterprise Scenario 3:process non-integrated & unstructured data

6 Bi D t T h l A h


42/59



Enterprise Scenario 4: archive all data in HADOOP

6 Bi D t T h l A h


43/59



Enterprise Scenario 5: access all data via the EDW

6 Bi D t T h l A h


44/59



Enterprise Scenario 6: access all data via Hadoop

8 Big Data Sec rit


45/59

8. Big Data Security

Data access: who have access and on which data?

Amplified technical impactIf an unauthorized user were to gain access tocentralized repositories, it puts the entirety of those data in jeopardy rather than asubset of the data.

Eg: Hive that provides data warehouse software that enables a SQL-like queryingexperience for the end user for Hadoop does not offer transactional support, a full typesystem, security, high concurrency, or predictable response times.

Approaches taht brings data into the Enterprise Data Warehouse and accessed throuold BI/Analytics tools could eliminate this shortcoming by using actual user rightmanagement solutions but not allow to use the full capacity of Big Data.

Data Privacy:

Privacy (data collection)Analytics techniques can impact privacy; for example,individuals whose data are being analyzed may feel that revealed information abouthem is overly intrusive.

Privacy (re-identification)Likewise, when data are aggregated, semi-anonymousinformation or information that is not individually identifiable information mightbecome non-anonymous or identifiable in the process using Behavioral analytics andpatterns.

9 Big Data Case Study


46/59

9. Big DataCase StudyMr. BricolageRomania

First challenge: the 4V of Big Data to support The 4 P of Marketing The 4 V

VolumeBig amount of data

Velocity - Analysis of Streaming Data

VarietyDifferent forms and sources of data

VeracityUncertainty of Data

The 4 P

Product

Place/Chanel

Price

Promotion



47/59


Case study: Case 1:

Objective: Assess and reduce/eliminate underperforming stock.

What we have used:

Stock turnover ratio = Cost of goods sold average stock holding

Stock aging

Days Sales Of Inventory

Contractual data

Historical data

How?

We created our own algorithms to determine goods movements between stores and proposolutions for reducing underperforming stock based on contractual data.



48/59


Case study: Case 1

Results:

We discovered that:

Over 40% of our goods was over 360 days old

The DSI (Days Sales Of Inventory) KPI was 190 days but hidden the real aged underperforming stock.

A very big amount of our goods was rotated between stores for multiple times, generatingsupplementary costs and hiding real stock aging.

Application proposed of the following solutions:

To return the goods to the suppliers and recover capital where the contracts allow that.

Return the underperforming goods to the supplier and replace them with better selling products.

Sales for the rests of underperforming products to unlock the capital.

In figures:

Over 3.000.000 Euro capital regained into the next 6 months.

DSI reduced to 120



49/59


Case study:

Case 2:

Objective: Increase prediction accuracy for turnover and gross margin over athree months period. (Starting accuracy 75%)

How: We used built in algorithms from MS SQL to generate mathematical modeusing 60% of historical data and tested over the rest of 40% data.

Results:

Over 86% accuracy after we used historical sales data only.

After we introduced some macroeconomics indicators: GDP, employment indicators(unemployment rate), Retail sales index, Consumer Price Index, average wage,Inflation, etc. and some demographic indicators, the accuracy increased to over 95%



50/59


Case study:

Case 3:

Objectives: Predictability of customer behavior and the possibility toundertake preemptive measures by sales agents.

Conditions: The test was made together with Fair Value Companyand used SAP HANA.

Results: Were astonishing (please see bellow):

Anomaly detected


51/59

Anomaly detectedCustomer 122

Anomaly detected


52/59

Anomaly detectedCustomer 12

Anomaly detected


53/59

Anomaly detected

Customer 12

Anomaly detected


54/59

Anomaly detected

Customer 16



55/59

9. Big Data Case StudyMr. BricolageRomania

Case study:

Case 4in progress:

Objectives: creating promotional campaigns that will allow us, usingcross-sell and upsell, to increase the margin with 10% compared withthe same period last year.

Conditions:

Partner PREDICTA with IBM SPSS

Will be used historical information about clients, products and promotions

Will be used macroeconomic indicators

Results: Deadline for solution 30th of May 2014.



56/59

9. Big Data Case StudyMr. BricolageRomania

Actual Challenges: Business and Analytics:

What we need to predict?

What we could optimize and how?

How to create a real customer personalized shopping experience?

What mathematical models we need to implement?

What data we need in order to make better analysis? Technological:

What architecture will be used in the future?

What providers and technology will better fit our needs?

9 Big Data Conclusions:


57/59

9. Big Data Conclusions:

Big data is not solely a technology issue.

Big data is a journey.

Big data is an incremental process.

Big data competencies need to be built within an analytics strategy


58/59

References:

Harvard Business Review - Analytics 3.0 http://hbr.org/2013/12/analytics-30/ar/1

ISACA White Paper January 2014 Generating-Value-from-Big-Data-Analytics

SAS - How will big data revolutionize retail? http://www.sas.com/en_us/news/sascom/2012q3/big-data-in-retail.h

SAS A Non-Geeks Big Data Playbook: Hadoop and the Enterprise Data War

International Institute for Analytics - Big Data in Big Companies - May 2013 By Thomas H. Davenport, Jill Dyc

Hadoop Iluminated

IBM-Oxford-IDG-Big Data Analytics Report

IBM Analytics_real-world_use_of_big_data_in_retail_Executive_Report

TWDI - Big Data Analytics By Philip Russom

Revolution Analytics - how-big-data-is-changing-retail-marketing-analytics

ORACLE - Information Management and Big DataA Reference Architecture

BIG DATA, BIG ANALYTICS John Wiley & Sons, Inc. By Michael Minelli, Michele Chambers, Ambiga

CSA_Big_Data_Top_Ten_Security Cloud Security Alliance

Microsoft_Big_Data_Booklet Microsoft Corporation

Exploring SAP NetWeaver BW on SAP HANA in combination with SAP BusinessObjects BI 4.x SAP AG

At NYSE, The Data Deluge Overwhelms Traditional Databases http://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the

overwhelms-traditional-databases/?sf9689013=1
http://hbr.org/2013/12/analytics-30/ar/1http://www.sas.com/en_us/news/sascom/2012q3/big-data-in-retail.htmlhttp://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the-data-deluge-overwhelms-traditional-databases/?sf9689013=1http://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the-data-deluge-overwhelms-traditional-databases/?sf9689013=1http://www.forbes.com/sites/tomgroenfeldt/2013/02/14/at-nyse-the-data-deluge-overwhelms-traditional-databases/?sf9689013=1http://www.sas.com/en_us/news/sascom/2012q3/big-data-in-retail.htmlhttp://hbr.org/2013/12/analytics-30/ar/1


59/59

Mihai Cristea CISA, CISM, CCSK

CIOMr. Bricolage RomaniaBrico Expert SA

Email: [email protected] LinkedIn: ro.linkedin.com/pub/mihai-cristea/5/a53/a34/
mailto:[email protected]://ro.linkedin.com/pub/mihai-cristea/5/a53/a34/http://ro.linkedin.com/pub/mihai-cristea/5/a53/a34/http://ro.linkedin.com/pub/mihai-cristea/5/a53/a34/mailto:[email protected]

Date post:	20-Feb-2018
Category:	Documents
Upload:	mihaicristea
View:	222 times
Download:	0 times

Big Data Theory and Case Study

Documents