ANALYTICS & TECHNOLOGY
D3M- NYU STERN
Vishal Singh
Trends in
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel to Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
Randomized Controlled ExperimentsKey research tool
o Nobel prize in Economics 2019 (Banerjee, Duflo, Kremer)o for bringing experimental approach to alleviating global poverty
o Example: Uber Price Experimentso Going from Regular to a 1.2x surge 27% drop in demand (What is the price
elasticity?)
o Going from 1.9x to 2.0x surge, one would observe a six times larger drop in demand than in going from 1.8x to 1.9x surge. WHY?
o When the surge multiplier moved from 2.0x to 2.1x, people actually took more rides. WHY?
o Why is Uber so consistent in expected wait time for a ride regardless of demand conditions? Surge algorithm filters demand and encourages supply.
Moral Machine Experiment
o Classical Moral Puzzleo The Trolley Problem
o Harvard UG Philosophy class
o Moral Machine Experiment (Driverless car)o MIT, published in Nature
o Global Subjects, large number of conditions
o Video from Nature Journal
o Data Download
Do it if you can
All digital platforms provide automatic design for conducting/evaluating experimental outcomes (and optimize on best cells)
Analytics is super simple (only intuition needed is randomization). Use Insights from Behavioral economics/ Psychology
New Ideas: (1) Counterfactuals (what-if analysis), (2) Heterogeneous Treatment effects (Uplift Models)
Experiments
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel to Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
Current status of Deep Learning / Ai algorithmsInput huge amounts of data from a specific domain (e.g., images of Breast cancer, data on loan repayment histories) Output make decision in a specific case (Cancerous or Not?, Give a loan or Not?)
Algorithms Optimized on Specified goals (Minimize risk of mis-classification of Cancer, Maximize time spent on FB / Youtube).
Example 1: Speech Recognition/Translation
Microsoft Inspire 2019
Example 2: Image Recognition, AR, Levering Databases
Google I/O 2019
How do these deep learning algorithms work?Get intuition from Ludwig, tool recently released by Uber
Applications in Health DomainRegina Barzilay, MIT
(visit her page and read popular press articles if interested in topic)
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel for Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
The era of Open Source
Note: Most algorithms in this domain are Open Source (e.g. ONNX). What matters then is implementation—key ingredient there is DATA.
The Rise of Open Source Software, CNBC (Dec 11, 2019)
Pattern to observe: 1) We need large amounts of “labeled” data 2) Algorithms are open source
Who has the most data now? Who will control data in the future? Strategic consequences
How do we label data? Pictures, reviews, etc. are human coded. Does it generate Bias?
“In the age of Ai where data is the new fuel, China is the new Saudi Arabia” Kai-Fu Lee (Author, AI Superpowers)
Note: Most algorithms/research in this domain is Open Source (e.g. ONNX). What matters then is implementation—key ingredient there is DATA.
What jobs would be lost? US, China & Ai (Bloomberg 5 min)
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel to Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
We don’t need just data, we need “labelled” dataWhere do we get labelled data?
Source 1:
We are generating it every day. Every time we click, like, browse, watch, buy…
What can simple FB likes reveal about us?
Source 2:
Human Coding: Recruit people (for example on Amazon Mturk) to annotate images, classify tweets or reviews as positive of negative, etc.
Both can lead to significant BIASES in Ai algorithms
Next few slides are drawn from here
Classification Task
Human Coder
Many sources of BiasesWhere are human annotators coming from?
Biggest breakthrough in Ai algorithm over the past several decade. Requires large amount of labeled data, domain specific
The 4 V’s of Big DataVolume & Velocity: More or less handled through AWS/GCP/Azure
Variety (numeric, categorical, unstructured (text, image, videos)
Veracity: how accurate/truthful/representative your input data is
Deep Learning: Summary
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel to Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
Privacy & Accountability
2 Questions: 1) Who do you trust with your data?
o Between firms (e.g. FB vs. Google)o (Chinese or Indian or US) Government vs Private
enterprise? 2) Would you install a small chip for early detection of health problems? Next revolution is integration of bio-tech & Ai
Highly recommended books..
Yuval Harari: Will Technology Help Us Become Immortal?
Privacy & Accountability:
Interpretable Machine Learning
Growing emphasis on “Interpretable” machine learning
Products by tech giants & startups (Google, Microsoft, H2OAi)
Current examples of interpretability: 1) Variable importance2) Predicted values 3) Continued & Increasing use of Visual Analytics
Need for a product like JMP Profiler with better UI
Trends in Analytics1. Experimentation (& the use of behavioral economics/Psychology)
2. The Deep Learning Revolution1. Examples in Practice
2. Intuition for how it works (Ludwig by Uber)
3. Examples of specialized use cases
4. Data as fuel to Ai: strategic consequences
5. Labelled Data & Biases in Ai
3. Privacy & Accountability 1. Privacy
2. Explainable Machine Learning
4. Learning Resources
Final Exam
Part 1: Data ExplorationBasic Data Literacy: Everyone needs this regardless of your field of interest or industry/sector
Part 2: Regression/PredictionMore important if you plan on more analytical type career. Knowing every detail is not important, keep higher level intuitions.
Learning Resources
Data & Educational Resources
o Data: Library for good Business data, Kaggle for Prediction type data, Use Google Data Search for everything else
o Tableau/JMP: Online video library by Tableau, Download JMP books from the Help menu
o Analytics: is a (foreign) Language. Knowing basics and using the right software(s) will take you far “Use-it” or “loose-it” principle applies
o Key is learning-by-doing