#ILTAG50

Artificial Intelligence in Law:

AI in Action(Part 2 of 3)

Stephen Allen

Head of Legal

Service Delivery

Hogan Lovells

International LLP

Katie DeBord

Partner and Chief

Innovation Officer

Bryan Cave

Neil Cameron

Chief Consulting

Officer

NCCG

MODERATOR: CEO & Co-

Founder

ROSS

Intelligence

SPEAKERS:

Andrew Arruda

Steve Harmon

VP and Deputy

General Counsel,

Legal

Cisco

Ron Friedmann

Partner

Fireman and

Company

Artificial Intelligence in Law:

Top Products in Action

Why we’re here and where we’re going.

CEO & Co-Founder

ROSS Intelligence

Andrew Arruda

Artificial Intelligence in Law:

Top Products in Action

What the Surveys Tell Us about AI

Ron Friedmann

Partner

Fireman and

Company

AI IN THE GLOBAL ECONOMYMCKINSEY 2017

“The current AI wave is poised to finally break through”

“Five elements of successful AI transformations”

“Six characteristics of early AI adopters”

ADOPTION OF AI IN LEGAL:SURVEY RESULTS

Altman Weil, Law Firms in Transition, 2017

Altman Weil, Law Firms in Transition, 2017

Survey, June 2017, ILTA KM Virtual Roundtable (37 responses)

Altman Weil

Law Firms in

Transition, 2015

AI IMPACT IN LEGAL

Aderant

Innovation

in Global

Legal

Business,

2017

That’s the high level view

Now for the AI in practice

Artificial Intelligence in Law:

Top Products in Action

Katie DeBord

Partner and Chief

Innovation Officer

Bryan Cave

Artificial Intelligence: A New Force Of Nature?

BRYAN CAVE’S AI APPROACH: UNDERSTAND, EXPERIMENT, AND APPLY

ACT I: UNDERSTAND

CT and PEG

• Combination of statisticians and technologists tasked to figure out how we could use leverage data better and use technology to do so.

TechX:

• Group of attorneys and allied professionals worldwide that experiment with advanced legal tech based on their practice areas.

ACT II: EXPERIMENT

Two primary areas of focus for BC’s machine learning/AI experimentation:

•Internal proofs of concept using high volume of unstructured data that coalesces under a common theme

•External proofs of concept using off the shelf research, document review, litigation analytics, and decision-making support tools

EXPERIMENTATION WITH OFF THE SHELF TOOLS

•Due Diligence (Kira, eBrevia)

•KM (RAVN)

•Legal Research (ROSS)

•Decision-making Support (Neota Logic)

•Patent/Litigation Analytics (Lex Machina)

Through TechX, we have a process to

regularly pilot and obtain

feedback on AI-Based tools

for:

•Interoperability with other tools

•Benefit of value add vs. risk of tech fatigue

•Security

Considerations Include:

ACT III: APPLY

USE AI/Machine Learning To

Categorize, Harness, and Understand

Unstructured Data Within Own Systems.

APPLY MACHINE LEARNING/AI BASED TECH TO PRACTICE

WHERE APPROPRIATE

MACHINE LEARNING APPLIED TO UNSTRUCTURED DATA TO CREATE “TASKER TOOL”

Method: Applied Bayesian Classifier to unstructured time entry narratives.

Goal: Enhance AFA pricing capabilities by more accurately understanding the levels of effort associated with tasks/phases within matters.

TASKER TOOL TRAINING INTERFACE

EXTENSION OF TASKER TOOL: USING WORD CLOUDS TO DRIVE BETTER BEHAVIOR

MACHINE LEARNING APPLIED TO SERIAL LITIGATION

Problem Statement: 5% of matters

account for over 50% of overall portfolio cost. Can these

matters be identified and resolved earlier

in their lifecycle?

IDENTIFIED VARIABLES IN PORTFOLIO

1. Variance from 2.5 kids

2. Body Mass Index

3. GMAT Scores

4. Cholesterol level

5. Zillow Zestimate

6. Sleep Number

7. LinkedIn Profile Completion %

8. United Miles

9. Daily Average Steps Walked

10.# Cats Owned

11.Etc.

Linear equation explaining some of the variability in total cost

Statistician

Dozens of Variables Minitab

RESULTS

Client’s litigation costs reduced by 2/3

•Outcomes at least constant if not improved

•Generated numerous best practices around knowledge management, staffing models, settlement strategies, etc. that now get leveraged broadly

• Identified and managed emerging legal theories and adopted common strategies for repeat claims types and counsel

AI-BASED “OFF THE SHELF” TOOLS

Opportunity Drivers:

•High Volume Low Margin Work

•Contract Management Applications

•Highly Regulated

•TECHX Attorney-Sponsored

ROI:

•Escalating benefit to end-user

•More consistent delivery of specific layers of legal advice and service

•Attorney participation and thought leadership

•Serendipitous Ideation

How and When

We Make a

Decision to

Deploy

ISSUE SOLUTION

Legal department of national retail brand

bogged down with requests for term

sheets

Application created allowing client to

generate standard, pre-approved term

sheet after answering a series of

questions

Client requires high volume of NDA’s,

which caused delays in business line

Business line can now self-generate

NDA’s, saving time and creating better

work flow

Need to streamline process of updating

franchise disclosure documents

Questionnaire makes process easy,

lowers risk and increases efficiency

DECISION-MAKING SUPPORT USE CASES

COMMON THEMES

Change is Hard (but it’s worth it):

✓ Need to carve out time to focus

✓ Invest in process

✓ Invest in content

✓ Capture and analyze data

✓ Be entrepreneurial – take risks and fail quickly

✓ Don’t let the perfect be the enemy of the good

Artificial Intelligence in Law:

Top Products in Action

Making the complex simple

and the simple

effective……..and dodging

silver bullets…Stephen Allen

Global Head of

Legal Service

Delivery

Hogan Lovells

MACHINE ASSISTED DOCUMENT REVIEW - AS A

SERVICE

MACHINE LEARNING AND UNDERSTANDING OUR

BUSINESS

MACHINE LEARNING AND UNDERSTANDING OUR

BUSINESS

MACHINE LEARNING AND UNDERSTANDING OUR

BUSINESS

MACHINE LEARNING AND UNDERSTANDING OUR

BUSINESS

Artificial Intelligence in Law:

Top Products in Action

Case Study: PoC for ‘AI Matters’ in Bird & Bird

Chief Consulting

Officer

NCCG

Neil Cameron

‘AI MATTERS’

The Business Problem

• Matter fee forecasts – budgets – are increasingly being

demanded by clients and courts

• Attorneys are not good at this – they suffer from all the

’normal’ planning cognitive biases – and some new ones

– In particular, they conflate ‘costing’ and ‘pricing’

– they do not ‘learn’ to improve their estimates

• The results are reductions in matter realisation, firm

profitability, and client satisfaction

Write-Offs

• If fee estimate is exceeded attorneys tend to write off the

difference, or ask the client for more money, or both

• Unless they have managed the client’s expectations

throughout, they will usually write off much of it

• Before the GFC many law firms used to operate on 80%-85%

realisation – now this tends to be nearer 70%

• A firm turning over $1b pa, will have written off $428m

• A 2% improvement in realisation on that would be $28m

Initiatives to Improve Forecasting

• Attorney Training

• Matter Breakdown

• Matter Templates

• ‘Reference Class Forecasting’

Reference Class Forecasting - Wikipedia

• “Human judgment is generally optimistic due to overconfidence and

insufficient consideration of distributional information about outcomes.

Therefore, people tend to underestimate the costs, completion times,

and risks of planned actions.

• “Reference class forecasting predicts the outcome of a planned action

based on actual outcomes in a reference class of similar actions to that

being forecast.

• “The theories behind reference class forecasting were developed by

Daniel Kahneman, the theoretical work helped Kahneman win the Nobel

Prize in Economics.”

‘AI Matters’

• Take a ‘class’ of matters – e.g. M&A

• Identify the key ‘complexity factors’

• Run algorithms against source data and those factors against previous

matter fees

• Train the system on the relationship between them

• Use the system to forecast new matter fee estimates based on known

and assumed complexity factor

• Do this in partnership with the client

‘AI Matters’

• Take a ‘class’ of matters – e.g. M&A

• Identify the key ‘complexity factors’

• Run algorithms against source data and those factors against previous

matter fees

• Train the system on the relationship between them

• Use the system to forecast new matter fee estimates based on known

and assumed complexity factor

• Do this in partnership with the client

Bird & Bird PoC

• Started with M&A

• Worked with lawyers to identify the most likely complexity factors

• Ran models, refined and re-ran models, and again

• Attorneys predict accuracy of fees to a correlation factor of 0.25

• Our current model predicts fees to an accuracy of 0.75

• Phase 2 will work on refining current complexity factors, working with

attorneys to identify new ones, and building a user interface

Humans

(Attorneys)

• 0.25

AI Matters

• 0.75

Conclusion

• Next steps:

– Enhance and develop further accuracy

– Move on to other types of work

– Use it to ‘fingerprint’ matters for knowledge management and quality assurance

purposes

• Before we start using AI to replace what attorneys do; let’s start using it

to undertake what attorneys can’t, or won’t, do

Artificial Intelligence in Law:

Let’s Discuss our AI in Action

Stephen Allen

Head of Legal

Service Delivery

Hogan Lovells

International LLP

Katie DeBord

Partner and Chief

Innovation Officer

Bryan Cave

Neil Cameron

Chief Consulting

Officer

NCCG

Steve Harmon

VP and Deputy

General Counsel,

Legal

Cisco

Ron Friedmann

Partner

Fireman and

Company