Converseon.AI Machine Learning CapabilitiesWorking with Walmart, using our auto MLaaS platform and...

Converseon.AI Machine Learning Capabilities

Campaign Measurement Values Based Measurement: Social Trust Index & Benchmarking

Confidential

NYK Conference

Demystifying Artificial Intelligence and the Role of Machines within Data and Discovery

May 9, 2019

© 2019 Converseon Inc | PROPRIETARY + CONFIDENTIAL

Who we are

2

Kris Russell • Director-Insights & Analytics,

Public Affairs.• Brandwatch client since • Has worked with Converseon

since 2017

Rob Key• Founder & CEO. • Machine Learning as a Service for

advanced Social and VoC data analysis (since 2008) to help clients find powerful insights in unstructured text data.

• Brandwatch partner since 2015.


Once Upon a Time…the world was more simple

.

3

The demands on social listening were pretty basic:

• Sentiment-focused• Limited metrics• Brand specific • Keyword-driven • Indicative not predictive• Uneven data quality


Today it’s more complex… Brands face unprecedented challenges–

PolarizationTribalism

Loss of trustConsumer activism

Brands in the crosshairs


But language is complex - it differs by domain and context

5

“Unpredictable” Movie is Positive“Unpredictable” Braking…

…Not So Much

A lot of meaning is implicit and absent specific keywords

“I spent my entire lunch hour waiting in line for my prescription...”

8%23%

84%37%

7%

40%

0%10%20%30%40%50%60%70%80%90%

100%

Walmart Brand Sentiment: January 2017 – April 2019“Document-Level” “Target-Level”

Data are based on population of >35M social media mentions

“Target-level” analysis gen 60% more signals than “document”

There are also often multiple opinions in a single document

Industry-specific advanced “core” model

Negative CX

Positive Walmart

Negative Target Brand

Positive CXPositive Amazon


Semi-Supervised “active machine learning” enables us to classify language “like humans do” with great sophistication

8

“Computers simply don’t have the brains to do some critical tasks unless we lend them ours.” Dr. Philip Resnik


But even then building highly effective models can be complex

9

“80% of the time enterprises spend on AI projects goes toward preparing, cleaning and labeling data. Specifically, the report finds that the many steps involved in data collection, aggregation, filtering, cleaning, deduping, enhancing, selecting and labeling data far outnumber the steps on the data science, model building, and deployment sides.” Tech Target


Walmart Trust Analysis

• Challenge: how can we leverage advanced

human-centered machine learning analysis to

understand “trust?”

• Map and measure “brand trust” and “distrust”

across 19 brands across 9 industries

• Enable Walmart to understand the expressed

trust conversations on Walmart and benchmark

against key cross industry leaders

10


How we look at trust

11

Are they acting on these values?

Do their values match mine?

Are they good to society?

Are the products reliable and meet requirements?

Corporate Citizenship

Reliability/CX

Social Issues

Brand Purpose

Risk & Value


ML models classify trust when keywords not present

12

Can we accurately capture posts about Trust & Distrust…

…using rule-based and Boolean Query Approaches alone?

Irrelevant to Trust, but captured by Boolean Query:

“Let's say you don’t Have a very Porsche or Audi to sell? We are still a fantastic option for selling your late model luxurious or performance vehicle. We are a trusted member of the Group and are serving our shoppers with the very same area for over thirty yrs.”

Relevant to Trust or Distrust, but missed by Boolean Query:

“I'm very fond of my Mazda3 — zippy car, fun to drive, and better mpg than most comparables.”

“If you are looking for trouble And problems Buy Kia Cars”

Boolean Query Approach: Example(I OR Ive OR “I’ve” OR me OR my OR our OR we) NEAR/3 (trust OR trusted OR trusts OR trustworthy OR trusty OR rely OR relied OR reliable OR reliability) Po

or P

reci

sion

Poor

Rec

all


13

Working with Walmart, using our auto MLaaS platform and services trust models were built and programmatically deployed directly into Brandwatch

Auto MLaaS Brandwatch

Conversus,AI Active machine learning training

Fully integrated into Brandwatch

Target level, not keyword specific


• How well does the algorithm match human “gold standard” agreement? (precision)

• How much of the relevant conversation is captured and analyzed? (recall)

• How well does the model work with data it hasn’t “seen” before? (generalization)

• How many ”signals” and annotations are available from a certain data set? (granularity)

14

All models were automatically scored and validated before deployment


15

Walmart Trust Analysis Initiatives Evolving into the first Social Trust Index

80 brands across industriesAnd growing


Comprehensive library of advanced prebuilt models now available via Brandwatch

16

• “While many open source machine learning platforms are available, the cost to successfully implement them and produce useful models can range into the hundreds of thousands or even millions of dollars... Using prebuilt models from cloud-based platforms can be much more cost-effective.” Forrester Research

• “Setting up an accurate model is long, technical and tedious. Converseon offers the algorithmic equivalent of fast food – a menu of pre-trained models which can then be let loose on a specific dataset and adapted to it with human help.”

Greenbook


17

Current “library” examples

Insurance Automotive Financial services Telecom Consumer Tech Enterprise Tech Food & Beverage QSR Retail Energy Airlines Luxury Goods

Customer Experience Intensity Trust Distrust Innovation Customer Care Advocacy Detraction Loyalty

Brand Relevancy Brand Attributes Brand Specific

Taxonomies

Industries Advanced Models Custom Models


Custom Model ExampleUber tracks key brand attributes such as “People-First” in real time for rapid insight and improvements

“The Custom AI models allow us to clearly separate signals from the noise, accelerate insight and allow Uber to move rapidly to improve service and build trust in our brand on ’real time’ global basis.”

I-COM Data Creativity Award Finalist 2019

18

134 models across 10 countries


Custom Model ExampleGlobal food company used ”motives” custom model to predict sales six weeks in advance

1919

Smoothed time series

Topic + Sentiment / Emotion = .57

Topic + Sentiment / Emotion + Motive = .86

Key Variables that explain Sales*correlation

© 2019 Converseon Inc | PROPRIETARY + CONFIDENTIAL 20

Social Maturity

Characteristics

Custom

Business Value Created

“Core” Advanced Enterprise Enablement

• Sentiment, emotion• Brightview custom

classifiers• Analyst driven

• Research grade turnkey advanced models customized to specific frameworks definitions, needs

• Highest level performance and generalization

• Customized models specific to broad organizational needs

• DIY model development Data integrated into business processes

• Social + VoC data• Data/model governance

CIO, Chief Data Officer, AI Strategy, Data Scientists, SMEs

CIO, Chief Data Officer, Market Research, brand tracking, Insights, Data Science

Brand management/ health, CX, Consumer Insights, Customer Care

Social listening, consumer insights media monitoring

Beneficiaries

• More complex industry focused pre-built models

• Target level analysis• Performance validation

(.75+ f1 est)• Rapid, low cost

deployment

1 2 3 4

Cost Included Per millionPer query

Depends on complexity Scoped to requirements

Key Value Proposition Clean Data Expand Use Drive insight Take Control & Integrate

The Maturity Curve: Align the Solutions to Your Organizational Needs

Data accuracy and speed to insight


Take-Aways

• Advanced ML models provide “human level” data classification at massive scale to unlock deeper and more meaningful insight from social data

• These models becoming essential to brand tracking/trust, CX, advocacy (NPS), predictive analytics, etc.

• Effective models require understanding of classification performance (precision/recall) and need to be designed to generalize effectively so they can be used with confidence on ongoing basis.

• Pre-built models can accelerate adoption and use. Complex custom models need to be built with care (clear definitions, intercoder reliability, ongoing human-in-loop iteration) to avoid potential inadvertent bias and poor performance

• Eventually, “automated MLaaS” technologies will increasingly allow even non data scientists to rapidly build and deploy advanced models for maximum impact by putting the technology in the hands of those who have deepest domain knowledge.

21


Rob Key, [email protected] Russell, [email protected]

22

Thank You!

mailto:[email protected]

Date post:	03-Jun-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Converseon.AI Machine Learning CapabilitiesWorking with Walmart, using our auto MLaaS platform and...

Documents