Date post: | 22-Jan-2018 |
Category: |
Data & Analytics |
Upload: | gopi-suvanam |
View: | 248 times |
Download: | 2 times |
Data-driven Insights
Sept 21st 2017
Presentation by:
Gurpreet Singh & Gopi Suvanam
G-Square Solutions
Data Driven Insights – An Introduction
Need of Insights for Business: Business leaders and analytics teams are looking to derive meaningful actionables from the business data, on the go.
Conventional BI tools wont work: MIS/BI reports or only visualizations will not help the cause – what is needed is an ability to get data-driven insights which can help them get the pulse of the business and suggest meaningful action points.
Key is AI and Machine Learning: Data driven insights are the fuel for every business decision being taken. Rather than relying on an analyst to get insights, business users can get insights directly from Robo-data-scientists using AI and machine learning.
A point of view can be a dangerous luxury when substituted for insight and understandingMarshall McLuhan, Canadian Communications Professor
Data-to-Insight-to-Action journey
Source: Genpact report on Data-to-insight
The critical aspect in the Analytics function
which is often neglected is the Data-to-
insight and Insight-to-action journey.
1. Provide Visibility – Descriptive
analytics
2. Manage effectiveness – Insighting on
the data
3. Execute actions – Prescriptive
analytics
‘He who searches for pearls must dive below’ – John Dryden
How will we prefer to infer data?
‘Once we know something, we find it hard to imagine what it was like not to know it’ Chip & Dan Heath, Authors of Made to Stick, Switch
Some insighting illustrations
Opportunities multiply as they are seized – Sun Tzu
Insig
hts
Solving the problem of automated insighting
Break insighting into
several sub problems Analyse data for each sub-
problem Do you care: Are the
insights interesting
enough?
Breaking down
the problem
Is there a seasonality of trend?
Are KPIs related to any factors strongly?
Are factors related to each other?
Is there any interaction effect?
Are there sub-trends in Factors?
What are the causal relationships?
Analysing each part: Deep data-mining
ARIMA
Model
Multivariate
analysis
Mutual
Information
Deeper
relationships
through
Recursive Trees
Causality
test
Outliers
A note on Mutual Information
› Variance gives dispersion in normal distribution
› Entropy gives a measure understand dispersion for any distribution
› 𝐻 𝑋 = 𝑋 𝑝 𝑋 𝑙𝑜𝑔1
𝑝(𝑋)
› Mutual information is a measure of the mutual dependence between the two variables.
› 𝐼(𝑋; 𝑌) = 𝑥,𝑦 𝑝 𝑥, 𝑦 𝑙𝑜𝑔𝑝(𝑥,𝑦)
𝑝(𝑥)𝑝(𝑦)
› Point-wise mutual information is similar to mutual information, but it refers to a single
event whereas mutual information is the average of all possible events.
Consolidating
Insights
Is the insight statistically
significant?
Are the underlying variables
important for the user?
This is where use of machine
learning becomes important to
identify the right set of insights,
curate them and present them in
an automated fashion
Narrating Insights: Natural Language Generation
Generate several templates
Keep the language simple and direct
Take care of grammar
Make it interesting
Use right ajectives
Insights to automated decision making
Automated
insighting tool
should also give
out insights in
structured format
{
"insight_type": "Trend_Analysis_Variable",
"sub_type": "month"
"priority": ""high",
"variable": "Region",
"level":"Vidarbha & Chattisgarh“
"insight": "Total PARValue has increased over one month
for WB & OR by 27.0% not changed muchVidarbha &
Chattisgarh by 0.0%", "Total PARValue has overall
decreased however comparatively lowest fall over one
quarter for Vidarbha & Chattisgarh by -27.5% and
significantly decreased for East - UP by 120.4%",
"campaign_text": "Focus on Vidarbha & Chattisgarh"
}
Tech Stack
Columnar DB for
structured data:
MonetDB
NoSQL DB for
structured data:
MongoDB
PythonPandas, Numpy,
Scipy, NLTK
D3JS Angular
15
Q & A
Let data Insights lead the way forward.
+91 22 43470408
www.g-square.in
Mumbai, India
@company/g-square-
solutions-pvt-ltd
@GSquareSolution
@GSquareSolutions1