More Chocolate Chip Muffins vs. More Chihuahuas and how AI and Machine Learning are impacting the “Search User Experience”
Don Miller
BA Insight
Intelligent Search: Connected, Search-Driven & Mobile
Don Miller – 2 Rules and an Observation
• Information - Consistency equals transparency, transparency allows you to surface and find the right information
• Relevance is subjective, but consistent metadata makes everything equal even if it is over tagged.
• You need to immediately start looking at how Machine Learning can impact your business. This is not a typical Gartner Hype Cycle
3
Observation - Machine Learning will not go into trough of disillusionment
▪
–
▪
▪
▪
▪
About BA Insight
4
Over 3.5 Million Users
AutoClassifier
70+ Enterprise Systems
Searc
h I
nte
rface
Query
Engin
e
SearchIndex(es)
Machine Learning
AI PlatformsGoogle Cloud AI
SmartHubConnectivityHub
Microsoft Cognitive Services
BA Insight Product Portfolio
▪
▪
–
–
▪
–
Our Technology Strategy
8
▪ Natural Language Query
▪ Bots
▪ Intelligent Query Engine
▪ Search UI (HTML)
▪ Analytics: Reporting and Recommendations
▪ API: Custom UI Development
SmartHub: AI-Driven Platform
9
▪ Standalone with Elastic and Azure Search and Solr
▪ Integrated with SharePoint/O365
▪ Being Integrated With Existing Applications:
– SiteCore, Outlook, Teams, Browsers, Netdocs, ServiceNow
SmartHub Configurations
10
▪ Automated Metadata Generation and Tagging
▪ Multiple Tagging Methods:
– Rule-based engine for Auto-tagging, Pattern recognition (PII,
GDPR)
– AI / Machine Learning
o Natural Language Processing: Entity extraction, Document
Summary Generation
o Image Analysis & OCR, Video Analysis, Sentiment Analysis
– Extensible Enrichment
AutoClassifier
11
▪ Tight Integration with:
– ConnectivityHub: Tagging of content from all sources
– Microsoft / Google AI suites
▪ Plug-Ins for In-Place Tagging of Other Systems
▪ Supports SharePoint, Elastic, Azure Search, ClarityNow
AutoClassifier: Architecture
Connectors to Many Enterprise Systems
• Aderant
• Adobe Experience Manager
• Amazon Aurora
• Amazon RDS
• Amazon S3
• Alfresco
• Azure SQL
• Bentley - Assetwise
• Biomax
• Box
• Confluence
• CuadraSTAR
• Deltek Vision
• Elite / 3E
• File Shares
• Google Cloud
• Google Drive
• HP Consolidated Archive
• (EAS, aka Zantaz)
• HPE Records Manager/
HP TRIM
• IBM Connections
• IBM Content Manager
• IBM DB2
• IBM FileNet P8
• IBM Lotus Notes
• IBM WebSphere
• iManage Work
• Jive
• Kaltura
• LegalKEY
• LexisNexis Interaction
• Lotus Notes Databases
• MediaPlatform
• Microsoft Dynamics CRM
• Microsoft Exchange
• Exchange Public Folders
• Microsoft SQL Server
• MySQL
• NetDocuments
• Neudesic The Firm Directory
• Objective
• OneDrive
• Oracle Database
• OpenText Documentum
• OpenText eDOCS DM
• OpenText eRoom
• OpenText LiveLink/RM
• OpenText Media Manager
• Oracle WebCenter
• PLC/Practical Law
• PostgreSQL
• ProLaw
• Salesforce.com
• SAP ERP
• SAP HANA
• ServiceNow
• SharePoint Online
• SharePoint 2016
• SharePoint 2013
• SharePoint 2010
• SharePoint 2007
• Sitecore
• Slack
• Any SQL-based CRM system
• Veeva Vault
• Veritas Enterprise Vault
(Symantec eVault)
• Website Crawler
• West KM
• Workplace by Facebook
• Xerox DocuShare
• DocuShare
• Yammer
Plus a proven process for creating new connectors to complex systems
15
Agenda – Today’s topic is how to impact search
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
▪
2019 Microsoft Search - Ignite
16
▪
▪
▪ SharePoint Multi-Geo
▪
▪
Hub Sites – Go big provide meaning in title
17
18
Graph and Delve – Cloud based
▪
–
▪
–
–
–
o
o
•
▪
–
▪
▪
Siri or audio conversion
19
Why does search matter
▪
▪
▪
▪
▪
▪
Customer Use Cases/Applications
21
▪
–
–
▪
Metadata is context and structure about information
22
Consistency equals
transparency,
transparency allows
you to surface and
find the right
information
Relevance is
subjective, but
consistent metadata
makes everything
equal even if it is
over tagged.
Internet Sets the Bar for Enterprise Search
WWW leading the way: Search & Refine
Categorisation
Map
Related Content
Refiners & Sorting
Results
with
Metadata
&
Imagery
Search
Navigation
Find
▪ Large Fortune 500 Pharma
– Auto tag metadata but spent manual time going through content sources to figure out what
could be leveraged as a rule to assign consistent metadata, then wrote scripts in alignment with
content sources to automate the rule building process
o Manpower plus cash and platform investment
▪ Large Fortune 500 Oil and Gas
– Leverage picture tagging to assign metadata in alignment with rig maintenance
o Primarily cash plus platform investment
▪ Large Top 50 Legal
– Auto tag via entity extraction
o Primarily cash and platform investment
Examples of automating tagging process (Manual is to PAINFUL!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
25
26
She no worka….
Language and Entity Extraction
28
Video Image and Audio with Timestamp
29
Facial Recognition
30
Signature Recognition
31
Sentiment Analysis
32
Language Detection
33
34
Where is the Intranet heading?
35
Ewe ☺
The 4 Must Dos
37
The missing concierge – Where do your searches/information requests fail?
High Value
– Manual
Tag
High Value
– Auto Tag
Low Value
– Manual
Tag
Low Value
– Auto Tag
High Value
– Multi
System
Search
High Value
– Single
Search
Low Value –
Multi
System
Search
Low Value –
Single
Search
Nearly every Search will need to add
1. Connectivity
Connectors to content & systems outside of O365
2. Structure
Information Architecture & Tags for navigation & findability
3. Context
Personalization through behavioural information
Connectors to Many Enterprise Systems
• Aderant
• Adobe Experience Manager
• Amazon Aurora
• Amazon RDS
• Amazon S3
• Alfresco
• Azure SQL
• Bentley - Assetwise
• Biomax
• Box
• Confluence
• CuadraSTAR
• Deltek Vision
• Elite / 3E
• File Shares
• Google Cloud
• Google Drive
• HP Consolidated Archive
• (EAS, aka Zantaz)
• HPE Records Manager/
HP TRIM
• IBM Connections
• IBM Content Manager
• IBM DB2
• IBM FileNet P8
• IBM Lotus Notes
• IBM WebSphere
• iManage Work
• Jive
• Kaltura
• LegalKEY
• LexisNexis Interaction
• Lotus Notes Databases
• MediaPlatform
• Microsoft Dynamics CRM
• Microsoft Exchange
• Exchange Public Folders
• Microsoft SQL Server
• MySQL
• NetDocuments
• Neudesic The Firm Directory
• Objective
• OneDrive
• Oracle Database
• OpenText Documentum
• OpenText eDOCS DM
• OpenText eRoom
• OpenText LiveLink/RM
• OpenText Media Manager
• Oracle WebCenter
• PLC/Practical Law
• PostgreSQL
• ProLaw
• Salesforce.com
• SAP ERP
• SAP HANA
• ServiceNow
• SharePoint Online
• SharePoint 2016
• SharePoint 2013
• SharePoint 2010
• SharePoint 2007
• Sitecore
• Slack
• Any SQL-based CRM system
• Veeva Vault
• Veritas Enterprise Vault
(Symantec eVault)
• Website Crawler
• West KM
• Workplace by Facebook
• Xerox DocuShare
• DocuShare
• Yammer
Plus a proven process for creating new connectors to complex systems
Only SharePoint and OneDrive Content Surfaces in Delve,
InfoPedia, etc. via the Office Graph
SAP / Oracle
OpenText/Filenet/
Documentum
Content from Other Systems Surfaces in Delve, InfoPedia, etc. via the Office Graph
Nearly every Search will need to add
1. Connectivity
Connectors to content & systems outside of O365
2. Structure
Information Architecture & Tags for navigation & findability
3. Context
Personalization through behavioural information
AutoClassifierrules-driven core, combined with learning-based modules
Enhanced Content
Enriched with
Metadata and
Content Types
Search Visualization Workflow
Nearly every Search will need to add
1. Connectivity
Connectors to content & systems outside of O365
2. Structure
Information Architecture & Tags for navigation & findability
3. Context
Personalization through behavioral information
45
SmartHub - Context equal personalization
The fourth must do for search success
1. Connectivity
Connectors to content & systems outside of O365
2. Structure
Information Architecture & Tags for navigation & findability
3. Context
Personalization through behavioral information
4. Measure
Personalization through behavioral information
47
Agent Bots Cognitive Machine Learning Augmented RealityMicrosoft Cortana Bot Framework Cognitive Services
- Vision- Speech- Language- Knowledge- Search- Labs
Azure MLCognitive Toolkit
Hololens
Google Google Assistant API.AI Cloud Vision APICloud Video IntelligenceCloud Speech APICloud Natural Language APIGoogle Knowledge GraphGoogle Custom SearchML Advanced Solutions Lab
Cloud ML EngineTensorFlow
Glass at WorkDaydream/Tango
IBM Watson Virtual Agent Watson Conversation Watson APIs- Vision- Speech- Language- Data Insights
Watson ML ServiceApache SystemML
Amazon Alexa Amazon Lex RekognitionPolly
Amazon ML
Apple Siri SiriKitSpeech
Core ML ARKit
Facebook Facebook M ParlAI FastTextCommAI
Torch Oculus Rift
The Cognitive Arms Race
Today’s Session – Recap
52
▪
More examples
53
AutoClassifier
70+ Enterprise Systems
Searc
h I
nte
rface
Query
Engin
e
SearchIndex(es)
Machine Learning
AI PlatformsGoogle Cloud AI
SmartHubConnectivityHub
Microsoft Cognitive Services
BA Insight Product Portfolio
Content
ContentProcessing
SearchIndex
/ Graph
Search UI
Enhancing Content with Cognitive Services
QueryProcessing
Content
Enrichment
AutoClassifierrules-driven core, combined with learning-based modules
Enhanced Content
Enriched with
Metadata and
Content Types
Search Visualization Workflow
Adding Text Analytics to ContentUser
Uploads Content Event
Fires
Content Text
Analysis
Sentiment - Is text positive or negative?
Key Phrases - What are people discussing in a single article?
Topics - What are people discussing across many articles?
Language - What language is text written in?
Add metadata to item
Shows in O365 search
Adding Translation to ContentUser
Uploads Content Event
Fires
Content Text
Analysis
Translation – Copy existing content and translate to specified language
save in SharePoint
Shows in O365 search
Create new file/content for specific language
Image SearchUser
Uploads Content Event
Fires
Computer Vision
Image Processing – recognize objects and text inside images
Shows in O365 search
Add metadata to item
Extract image(s)
from documents
Image Classifier and Search
project libraries
and
issue detection
Search Visualization Workflow
61
Where is the Intranet heading?
Search Provides A Unified View
Making O365 search intelligent using Azure Cognitive Services
more
ˇ
Start with understanding how it works
Matching IntentContent
Context
ContentEnrichment
UX
Connections
QueryAugmentation
Internet Sets the Bar for Enterprise Search, How do we make search happen on the intranet?
Insight Data Sets
Trending Documents
People I Work With
UsedDocuments
Files
Contacts
Groups
Meetings
Org
Tasks
File Enrichments
ItemMetadata
# ActivityHistory
myGraph
Sites I Use
Suggested Sites
Usage Statistics
How does the search box work?
Used
Documents
Trending
Documents
Sites
I Use
Suggested
Sites
Beautiful
Bergen
Contoso
Norway
Kathrine
Hammervold
Metadata Drives Great User ExperiencesDocuments from many sourcesAll client or matter-relevant documents are integrated. Rich MetaData
Content annotated automatically – concepts,
categories, citations, matters, clients, etc
Navigation ControlsExplore, Discover, Drill-down
PersonalizationIt knows what to present to whom
Where is the Metadata in Delve?
So….I don’t need metadata, right?
Delve doesn’t have refinement,
so I don’t need metadata, right?
Chart courtesy of xkcd
70
AutoClassifier
70+ Enterprise Systems
Searc
h I
nte
rface
Query
Engin
e
SearchIndex(es)
Machine Learning
AI PlatformsGoogle Cloud AI
SmartHubConnectivityHub
Microsoft Cognitive Services
BA Insight Product Portfolio
72
The End
▪
▪
▪
▪
Addendum 1- Chihuahuas for Precision and Recall
How quickly can we improve from 30%
▪
Chihuahuas for Precision and Recall
▪
Search for Chihuahua
▪
▪
▪
Provides these 4 as Results
How quickly can we improve to 90%
For More Information
▪ Check out resources at:
www.BAinsight.com
▪ Reach me via email at:
▪ Connect with me on LinkedIn at:
https://www.linkedin.com/in/donaldtmiller/