Date post: | 15-Apr-2017 |
Category: |
Data & Analytics |
Upload: | esther-valle |
View: | 326 times |
Download: | 1 times |
Welcome toWatson Innovation Day
Lauri SaftDirector, Partner RecruitmentIBM Watson Ecosystem
2 © 2014 International Business Machines Corporation© 2014 International Business Machines Corporation
2
The Watson Ecosystem is about you, and this Watson Innovation Day is for you
Here’s what we hope you get out of today:
• Inspire you with a clearer understanding of how IBM Watson can propel your business to a new level
• Help shape the vision for your Watson cognitive application with an actionable success criteria
• Equip you with a pragmatic sense for the process to build a Watson application and the enabling technology that is available to you
• Give you a tangible feel for how IBM will partner with you, from inception through go-to-market and beyond to help you make money
3 © 2014 International Business Machines Corporation3
What’s in it for you in the IBM Watson Ecosystem partnership?
• Growth through new revenue opportunities and reach to new clients
• Branding your cognitive application; “Powered by Watson”
• Build a competitive edge to disrupt your market
• Align with IBM Watson Research as we move build the cognitive era
• Collaborative IBM/Partner presentations at industry & IBM customer events
• Define a new industry standard
4 © 2014 International Business Machines Corporation
4
Time Agenda Speaker
8:00am – 8:30am Registration / Breakfast / Networking
8:30a-9:30am IBM Watson Level Set: What is Watson / Cognitive computing, What does the Ecosystem have to offer you? Where is IBM taking Watson in the future?
Lauri SaftDirector, IBM Watson Partner Recruiting
9:30am - 10:15am Getting Ready for Watson – How to Evaluate the Business and Technical Synergy and Chart Your Course for a Watson Cognitive Application
Jodie SasseDirector, IBM Watson Partner Engagement
10:15am - 10:30am
Break
10:30am - 11:00pm
Our Journey with WatsonJeff CohenCo-Founder, WellTok
11:00am – 11:45am Getting Started with Watson: An Overview of the “Powered by Watson” Development Process with the Watson Developer Cloud
Sridhar SudarsanCTO, IBM Watson Ecosystem
11:45am – 12:00pm
Wrap up: Where Do I Go From Here?Lauri Saft
12:00pm – 1:00pm
Lunch and Networking Chicago room
1:00pm – 3:00pm1:1 Consultations with IBM Watson ExecutivesHands on “Work with Watson” Sessions with IBM Watson Technologists
Breakout rooms
Agenda
What is Watson andCognitive Computing?
6 © 2014 International Business Machines Corporation
On February 14, 2011
made history
7 © 2014 International Business Machines Corporation
Watson is ushering in a new era of computing
TabulatingSystems Era
ProgrammableSystems Era
CognitiveSystems Era
1900 1950 2011
8 © 2014 International Business Machines Corporation© 20
Watson has opened up a world of new possibilities
siz
e
speed
corp
us
flexib
ility
#users
siz
e
speed
corp
us
flexib
ility
#users
2880 cores
Single user system
2-3 sentences input
5+ days to retrain
Wikipedia, general corpus
Single Power 750, 240% faster
1000s of users
20 pages of input
< Day to ingest and train
Medical corpus
Watson cloud service
Millions of users
Dialog chaining
Few hours
Broad industry corpus
s sp
corp
flexib
i
#users
9 © 2014 International Business Machines Corporation
2013 recap
Watson Engagement
Advisor - May 21st
�Watson Engagement Advisor announced
�Cross-industry and cross-geography play
January April July October
First Two Commercial
Offerings
February 8th
First 2 healthcare products were announced with MSK and WellPoint
2014
Watson Ecosystem
November 14th�Watson Ecosystem
announced�Driving innovation and
fostering new possibilities
Watson Healthcare Update – Oct. 18�MD Anderson launches Expert Oncology Advisor powered by Watson�Watson used to bridge clinical practice and medical research
10 © 2014 International Business Machines Corporation
Watson has the potential to transform how industries and individuals interact and make every day decisions. Watson helps make individuals, enterprises and society smarter by
engaging the collective knowledge and insight of the market.
IBM is investing in Watson’s future – January, 2014 announce
• IBM Watson Group•2000+ professionals•$1B+ invested over next few years•New Watson HQ – NY (Silicon Alley)•New Watson Offerings•$100M equity investment in the Watson
Ecosystem
11 © 2014 International Business Machines Corporation
Understands natural language and human style communication
Adapts and learnsfrom training, interaction, and outcomes
Generates and evaluates evidence-based hypothesis
1 2
3
Watson combines transformational capabilities to deliver a new world experience
Watson understands me.
Watson engages me.
Watson learns and improves over time.
Watson helps me discover.
Watson establishes trust.
Watson has endless capacity for insight.
Watson operates in a timely fashion.
12 © 2014 International Business Machines Corporation
Watson makes sense of natural language, like:
Welch ran this?
“If leadership is an art then surely Jack Welch has proved himself a master painter during his tenure at GE.”
Person Organization
L. Gerstner IBM
J. Welch GE
W. Gates Microsoft
Structured Data Unstructured Data
Understanding language is critical to 21st century computing
• Noses that run and feet that smell?
• Ship by truck and send cargo by ship?
• How can a slim chance and a fat chance be the same, while a wise man and a wise guy are opposites?
• How can a house burn up as it burns down?
• Why do we fill in a form by filling it out?
• How does an alarm go off by going on?
13 © 2014 International Business Machines Corporation
Deep natural language processing is NOT keyword search
� Watson is not about rules-based search or speech processing
� Traditional systems use keyword and synonym matching to find documents
� Expert systems need rules to demonstrate expertise
� These systems trade off precision for accuracy outside of what they are programmed to recognize
� Cognitive systems, like humans, know how to apply context, fill in the gaps, resolve ambiguity and uncertainty, and accurately interpret language
Keyword search and expert systems are
not able to recognize the subtleties,
idiosyncrasies, and ambiguities inherent in common human
language
14 © 2014 International Business Machines Corporation
Answering complex questions requires more than keyword evidence
explorer
India
In May
1898
India
In May
celebrated
anniversary
in Portugal
In May, Gary arrived in India
after he celebrated his
anniversary in Portugal
Portugal
400th
anniversary
celebrated
Gary
In May 1898 Portugal celebrated
the 400th anniversary of this
explorer’’’’s arrival in India
This evidence suggests “Gary” is the answer BUT the system must learn that keyword matching may be
weak relative to other types of evidence
arrived in
arrival in
Legend
Keyword ““““Hit””””
Reference Text
Answer
Weak evidenceRed Text
Question: Supporting Evidence:
15 © 2014 International Business Machines Corporation
Watson leverages multiple algorithms to perform deeper analysis
27th May
1498
Vasco da
Gama
landed in
arrival
in
explorer
India
Para-phrases
Geo-KB
DateMatch Stronger evidence can be
much harder to find and score…
� Search far and wide
� Explore many hypotheses
� Find judge evidence
� Many inference algorithms
On the 27th of May 1498, Vasco
da Gama landed in Kappad
Beach
400th anniversary
Portugal
May 1898
celebrated
In May 1898 Portugal celebrated the
400th anniversary of this explorer’’’’s arrival in India.
Kappad
Beach
Legend
Temporal Reasoning
Reference Text
Answer
Statistical Paraphrasing
GeoSpatial Reasoning
Question: Supporting Evidence:
What is the IBM Watson Ecosystem?
17 © 2014 International Business Machines Corporation
Enhance customer / member / Enhance customer / member / Enhance customer / member / Enhance customer / member /
citizen / patients experiencecitizen / patients experiencecitizen / patients experiencecitizen / patients experience
Watson Developer Cloud delivers the tools, methodologies, software developer kits and API(s) for App Partners to build the next generation of cognitive applications.
Watson Talent
Hub bridges App Partnersresource gaps by providing a marketplace for critical cognitive, big data, UI, and mobility skills.
IBM Watson Ecosystem – a community of complementary, interconnected partners
Watson Content
Store provides sources of free and fee based
content including public, subscribed and
enterprise
content.
App
Partner
Talent
Partner
Content
Provider/Partner
Watson
Content
Store
Watson Watson Watson Watson
Developer Developer Developer Developer
CloudCloudCloudCloud
Watson
Platform
& Tools
18 © 2014 International Business Machines Corporation
A community for ongoing dialogue, collaboration & interaction
� Developer Q&A Forum: DeveloperWorks
� Active forums and blogs
� https://www.ibmdw.net/watson/
� Engagement with IBM Watson technologists and other SME’s
� “Office Hours” on the dW forum12-1pm EST through April 14, 2014
� Watson “Tech Talks” web conferences –coming in Q2, 2014
� Growing social dialogue
� Twitter: @IBMWatson
� LinkedIn: Watson Advocates Group
� Facebook: facebook.com/ibmwatson
� In process
� Single community with tools, tips, tricks, ideas and conversation
� Meet-ups and hangouts with Cognitive business & tech leaders
� Content, Interaction, and events
19 © 2014 International Business Machines Corporation1
• Work with a handful of partners in target industries (healthcare, retail, travel & transportation)• Harden platform & tools• Refine content ingestion • Build scalable approach to the broader partner marketplace
2Q 2014
• First “Powered by Watson” cognitive application partners commercialize products and bring to market
• Expand content partnerships for general domain and industry specific content stores• Talent partnerships expanded to meet needs of new partners and new markets
2H 2014
• Expansion into English speaking international markets• Streamlined self-service tools deployed through Watson Experience Manager• Additional Watson capabilities/services brought online for Ecosystem Partners
1H 2015
• Developer Cloud opened to a broaderpartner marketplace
The Partner Ecosystem will expand aggressively each quarter
1H 2014
20 © 2014 International Business Machines Corporation
A clearing house of unique and varying sources of data –general knowledge, industry specific content, and subject
matter expertise – from a variety of sources.
The Watson Content Store offers fuel for apps
20© 2014 International Business Machines Corporation
ISVs
• Fuel your application with a variety of content
•Easily access multiple sources•Quickly get value from content
Content Providers
•Distribute via a new channel•Generate revenue•Control price & use•Gain insight into adoption
Benefits
21 © 2014 International Business Machines Corporation
Healthcare Example using IBM Watson Content Store
Clients
Types of Content
• Content Store: General, Domain Specific
• App: Partner Specific, Client Specific
Content Models
• Subscription
• Transaction
• Other
MedInsights Procedures
MMG Best Practices
Domain Content
Partner App
Client & Partner Content
Healthcare Platform
General Content
Watson Healthcare Cloud
MidWest Medical
Group
Excel Physicians Practice
MedInsights powered by IBM Watson
Watson Platform
Medical Journals
22 © 2014 International Business Machines Corporation
IBM Watson works with talent partners to make their skills available in the Watson Talent Hub to assist Development Partners
Watson SMEs provide input and coaching to the Talent Partners
Talent Partners develop Watson-validated talent
Talent Partners build individual landing pages
Talent Partners engage App
Partners to learn about their needs
Talent Partner receive project
requests from App Partners
App Partners agree to terms and provide
basic information
App Providers gain access to
Watson-validated talent
App
Partners
Talent
Partners
Watson hosts a Talent Hub within the Experience
Manager
App Partners find Talent Partner landing pages through WEM
Talent Hub
23 © 2014 International Business Machines Corporation
Talent groups follow the cognitive application development lifecycle. NLP and AI skills are not required to launch a “Powered by Watson app
Content Management
User Experience Design
Cross-Platform / Web
Mobile
Front-end Development
Web Crawling &
Content Scraping
Web Scripting
Parsing
User Experience
User Interface Web Framework Libraries
Programming
Talent Groups
Specialty Fields
Talent groups launching in 2Q 2014
24 © 2014 International Business Machines Corporation
Access to talent is simplified and streamlined
25 © 2014 International Business Machines Corporation
Talent validation will evolve into a formal Certification Program
Content Management
Situational Exam
Watson Fundamentals
Multiple Choice
User Experience
Situational ExamFront-end Development
Situational Exam
Content CollectionContent Curation
FormattingCleansing
Recurring feeds
UI/UX DesignVisualCopy
Cross-platform Application Development
Mobile Application Development
IdeationPrototyping
Broad-based knowledge of Watson
Talent Partner Qualified Talent
Level II: Demonstrated domain expertise as it relates to Watson
Level I: Demonstrated general domain expertise and understanding of Watson
26 © 2014 International Business Machines Corporation
Take the Watson Developer Challenge:
• Opportunity for Mobile Developers to create the next great cognitive mobile app –announced at Mobile World Congress by Ginni Rometty
• Finalists get access to Watson for one business week to prototype their Watson instance and mobile app
• The top 5 prototypes present live to IBM Executives
• Winners join the Watson Ecosystem and get access to IBM Interactive design Services
27 © 2014 International Business Machines Corporation
Contest OverviewSubmissions: 2/26/2014 – 4/15/2014Finalist selection: 4/28Prototyping Period: 4/28 – 5/20Winners Announced: 5/31
• Submission phase: Applicant submit their application concepts.
• Judging is based on:
• Target market that is clearly defined
• Compelling use case for Watson
• Can be delivered via mobile
• Access to data to provide insight (English prose in text form)
• Existing customer base
28 © 2014 International Business Machines Corporation
1:1 business and technical consultations are available today from 1-3pm. Sign up at registration desk.
Getting Ready for WatsonEvaluating the Business and Technical Synergy and Charting Your Course
Jodie SasseProgram Director, EngagementsIBM Watson Ecosystem
30 © 2014 International Business Machines Corporation
How do I start my journey with IBM Watson?
1.What is the business value to the “Powered by Watson”
application you intend to build?
2.Does the application draw upon the unique characteristics
and capabilities of a cognitive offering?
3.Has the content that will fuel the application been
identified and can it be secured/licensed for the intended use?
Have a question and answer interaction pattern, with questions posed in natural language
Seek answers and insights from a defined content repository comprised largely of unstructured data
Need transparency and supporting
evidence for confidence weighted responses to questions and queries
The best use cases must…..
31 © 2014 International Business Machines Corporation
Content
Capabilities
ConceptCommitment
Clients
• Ownership / Access to content • Form - unstructured• Format – ingestible
• Use case alignment w/technology
• Leverages cognitive strengths
• Market potential • Market impact (disruptive)
• Exec sponsorship• Priority capability• Resource commitment
• Anchor client potential
• Installed base
Requirements for rapid success
32 © 2014 International Business Machines Corporation
What do I get as an IBM Watson Ecosystem Application Partner?
� Use of the Watson Developer Cloud, including: �Watson Experience Manger and other tooling
�API to access the QA service to Watson
�Documentation and tutorials
�Workspace to enrich Watson with content
�Workspace to train and test your application
�Customizable user interface to embed into your application
�Guidelines on using Watson elements in the application
� Upload content into the Watson Content Store or source through Ecosystem partners
� Access to a network of partners, customers, content and talent
� Support from a collaborative community of IBM technical, marketing and sales resources
33 © 2014 International Business Machines Corporation
� Work with IBM team to solidify use case strategy
� Define business case & content strategy
� Access the Watson Dev Cloud through tools and API
� Build the prototype solution
� Socialize in market
� Build a commercial application, enrich, train and test application
� Final testing before product release
� Execute on Go to Market
� Deploy the product “Powered by Watson”
� Expand usage in future releases
Planning your Watson application roadmap in 12 – 16 weeks
• Align use case with rapid start criteria
� Sign up IBM PartnerWorld
� Submit Ecosystem application
Ready Engage Prototype Build Deploy
Go-to-Market Plan and Execute
34 © 2014 International Business Machines Corporation
1. Explore use case for Watson business and technical alignment
2. Submit Watson Ecosystem application https://ibm.biz/watsonecosystemapply
3. Join the IBM PartnerWorld community at http://www.ibm.com/PartnerWorld
Step 1: Ready Phase
Ready Engage Prototype Build Deploy
35 © 2014 International Business Machines Corporation
� Access the Watson Developer Cloud
� Enrich Watson : Acquire, Upload, Ingest, Deploy content using the Watson Experience Manager
� Create application prototype with Watson as a component
� Identify and begin acquiring content
� Create user experience mockups
� Develop Q&A pairs
� Test
� Socialize prototype with client base
Phase 2: Prototype
Ready Engage Prototype Build Deploy
36 © 2014 International Business Machines Corporation
� Capture market requirements through advanced discussions with early clients
� Create a minimum viable product “Powered by Watson”
� Continue to refine the architecture, design for interaction and integration patterns with Watson
� Sign beta clients
� Develop application Iteratively:
� Enrich Watson further with content
� Train Watson using Subject Matter Experts
� Test functionally and non-functionally
� Plan for go-to-market (sales, support, marketing)
� Formalize content strategy
Phase 3: Build
Ready Engage Prototype Build Deploy
37 © 2014 International Business Machines Corporation
� Prepare for commercialization, launch and other go-to-market activities
� Plan for application rollout with initial set of users
� Set up support, system usage metrics, and other parameters for application monitoring
� Execute on plan to expand users, application functionality and content
� Based on market feedback and usage, iteratively plan and build out scenario(s) further and more broadly
Phase 4: Deploy
Ready Engage Prototype Build Deploy
38 © 2014 International Business Machines Corporation
What is the cost to be part of the IBM Watson Ecosystem?
Application Partners
IBM Developer Cloud
Engage, Prototype, Build Phase: Free
Deploy phase : Revenue share
Deploy phase,: Application Hosting thru Softlayer Optional) Contracted and billed separately
IBM Watson Content Store
• Free to upload own data• Content available via Content Store with standard
terms: Free and Fee
IBM Watson Talent Hub
• Certified talent resources: Fee based determined by talent provider
Content Partners
• Free to upload and process content on the IBM Watson Content Store
• Revenue share for content used
Talent Partners
• Free participation (qualified) in IBM Watson Talent Hub• Revenue share for talent services provided
Getting Started with WatsonAn Overview of the Watson Development Process
Sridhar SudarsanChief Technical OfficerIBM Watson Ecosystem
40 © 2014 International Business Machines Corporation
Access Watson Developer Cloudusing Watson Experience Manager
Develop app
“Powered by
Watson”
using APIs
Enrich Watsonwith content
Train Watson using tools and experts
Test app
functional and
non-functional
Deploy application
The process for building your “Powered by Watson” app
41 © 2014 International Business Machines Corporation
Access Watson Developer Cloudusing Watson Experience Manager
Develop app
“Powered by
Watson”
using APIs
Enrich Watsonwith content
Train Watson using tools and experts
Test appfunctional andnon-functional
Deploy application
The process for building your “Powered by Watson” app
42 © 2014 International Business Machines Corporation
Design and Develop “Powered by Watson”apps using the QA API
1. Design interaction and integration patterns with Watson and implement proof points as necessary
2. Establish the architecture with Watson and other components
3. Implement and model solution to scale as a product with customization
Key points
Access Watson Developer Cloud
using Watson Experience Manager
Develop app “Powered by Watson”using APIs
Enrich
Watsonwith
content
Train
Watson using tools
and experts
Test app
functional and
non-functional
Deploy
application
43 © 2014 International Business Machines Corporation
Application powered by Watson using QAAPI
Pose Natural language question
Communicates with Watson using
QAAPI
Get response
Optionally, further apply Watson response
and/or other metadata
An example solution pattern used with Watson
• Design a solution with Watson as a key component
• Model the interaction patterns with Watson
• Create user experience mockups for the solution
44 © 2014 International Business Machines Corporation
Guidelines for interaction patterns with Watson
Use the following guidelines when evaluating the suitability of a specific solution:
•Tips for the “Ask” service
•Answers must be found in the content resources ingested in Watson •Answers must be explicit in the content; Watson cannot synthesize answers•Answers must be discrete – Watson cannot combine information from various documents or create an answer that is a deduction from passages it finds (future)•Questions must not require Watson to make judgments
•Descriptive questions
•Questions can be approximately one to three sentences long•Questions should require answers of a small paragraph (about three sentences or less)
45 © 2014 International Business Machines Corporation
Watson works with different types of interactions
Question Type Description Example
Factoid( only supported question capability in
Jeopardy )
Simple fact regarding time, location, reference, quantity
Q: What fees are charged with XXX fund?
Descriptive Description about a topicQ: How are Roth and traditional IRAs different?
Yes /NoSimple one response answers
Q: Are Roth IRA contributions tax deductible?
Procedural – How ToHow to question that returns list of steps
Q: How do I plan for college education?
Procedural –Troubleshooting
Troubleshooting questionsQ: Why did I have a fee charged on my brokerage?
Definition Definition of industry terms Q: What is a 529 plan?
Training on each type of question enables Watson to provide a high quality experience for each user
46 © 2014 International Business Machines Corporation
Using the QAAPI with Watson
� Integrate and Interact with Watson from Mobile or other Self Care Apps using a RESTful API
� Response, confidence evidence and other response metadata can be manipulated, enriched and used part of a conversation; or to a component in the system downstream for further processing
Configure parameters,
including Basic Auth using
the user id and password
for Watson Developer
Cloud, POST question;
Receive question, obtain
response JSON
Return JSON response
and authentication token
Receive response from
Watson
Send response to user or
further processing
47 © 2014 International Business Machines Corporation
Integration with Watson – Sample JSON request / response
REQUEST
RESPONSEConfidence
Evidence
Answer #1
48 © 2014 International Business Machines Corporation
Access the Watson Developer Cloud using the Watson Experience Manager (WEM)
With Watson Experience Manager:
• Developers can use APIs to access Watson and test “Powered by Watson” apps
• Data Scientists can manage their content used to enrich Watson
• User experience developers can customize or create user interaction models with Watson
• Domain experts can train and test their “Powered by Watson” apps
WEM provides a role based set of tools for SME, Watson administrators, and Domain Experts
49 © 2014 International Business Machines Corporation
� Administration
� Manage User Roles
� View Usage Reports
� Configuration
� Configure, Style, & Theme out of the box UI
� Define confidence threshold
Watson Experience Manager – Configuration
50 © 2014 International Business Machines Corporation
Enrich Watsonwith content
1. Enrich Watson with relevant content to fuel the applications
2. Use Watson Experience Manager to upload, ingest, deploy content
3. Leverage the Watson Content Store to acquire public, subscribed or enterprise content
Key points
Access Watson Developer Cloud
using Watson Experience Manager
Develop app
“Powered by
Watson”using APIs
Enrich Watsonwith content
Train
Watson using tools
and experts
Test app
functional and
non-functional
Deploy
application
51 © 2014 International Business Machines Corporation
Proprietary Content
Acquire
Cleanse
Curate
Aggregate
Contract
Identify
Public Domain & Social
IBM Watson
Validate
Application Partners
Source Prepare
Ingest
Deploy
Select Content
Build Apps w/API’s
Leverage
App Users
Unstructured Data From Enterprise Apps
CRM eComm
Workforce HR
Product
Catalog
Trouble
Ticket
Domain Specific Content Providers
Email, Collaboration,
Open Data
ECM KMS
End Users
The content journey
Publish
Publish
Watson Content Store
52 © 2014 International Business Machines Corporation
Content guidelines in detail
� IBM Watson requires a knowledge database (corpus) that contains information from which to formulate correct answers
� Content ingested into IBM Watson means that content is processed by the system such that correct segments of data can be quickly retrieved
� Content can come from a variety of sources and will have to be physically made available
� Content requires a level of cleansing and curation for Watson to be effective
Proprietary Content
Acquire
Cleanse
Curate
Aggregate
Contract
Identify
Public Domain & Social
Validate
Unstructured Data From Enterprise Apps
CRM eComm
Workforce HR
Product
Catalog
Trouble
Ticket
Domain Specific Content Providers
Email, Collaboration,
Open Data
ECM KMS
53 © 2014 International Business Machines Corporation
Content ingestion – Supported contentTypes of Unstructured Data that can be Ingested in Watson
� HTML
� Websites
� Domain Independent Content
� FAQ pages
� Forums
� Product Descriptions
� Product Catalog Export
� Product Reviews
� Social Network Extracts
�Microsoft Word
� Operational Runbook’s
� Policies & Procedures
� Contracts & Agreements
� Terms & Conditions
– Product Manuals
– Financial Reports (sans graphs)
– Analyst Research
– User Guides
– Troubleshooting Guide
– Terms & Conditions
– Setup & Configuration Guides
– Magazines & Journals
� Text
– Extracted Interaction Logs from CRM, Trouble Ticket, Care, Media Asset Management applications
54 © 2014 International Business Machines Corporation
� Understand graphics (images, videos, sound …)
� Ingest directly from a structured data source (Database, Datawarehouse)
� There are architectural patterns and capabilities in Watson to be able to integrate with structured data sources
�Nested tabular data
�Call data records, sensor logs, meter data, etc.
What Watson cannot ingest yet ….
55 © 2014 International Business Machines Corporation
What Works?
Documents that are relevant to the use case under consideration
Understanding the validity of documents & frequency of changes
Well formed semantics – titles, sections, headings, sub-headings
What Does Not?
Access to glossary, abbreviations would be helpful and preferred
If HTML – removing headers, footers, sidebars improves responses
Corpus contain answers to the intended questions
Scanned Docs (OCR), Charts, Non-English, Password Protected, Noisy Chat Logs
Nested Tables; Image right document with no descriptions; Log & Sensor Data
Mismatch in the content ingested and the questions end users are asking
General guidance on content selection
56 © 2014 International Business Machines Corporation
Content crawling & curation – Items to consider
�Content Acquisition
� Crawling
� Integration with ECM/KMS/DMS systems
� Open Services for Lifecycle Collaboration
�Content Curation
� Cleansing, Preparation, Aggregation & Enrichment
�Content Lifecycle Management
� Stale content
� Refreshing content
�Content Metadata, Licenses & Attribution
57 © 2014 International Business Machines Corporation
Content crawling & curation – Example content
� Based on this content one might be interested in understanding the following.
� Various domain (medical health in this case) terms or concepts such as diseases, symptoms (headache, sore throat etc.), associated health conditions (asthma, diabetes etc.), vaccinations etc.
� Concepts or terms expressed in different ways (flu Vs. influenza, running nose Vs. cold etc.)
� Timing in terms of when the disease is more prevalent during a given year.
� Statistics in terms of how many people were affected over the years.
� Locations and other geo references made in the document with respect to the concepts or terms.
� Co-references or relationships or associated side effects with other health conditions (Guillain–Barré Syndrome, chronic lung disease etc.), vaccination options etc.
58 © 2014 International Business Machines Corporation
� Based on this content one might be interested in understanding the following.
� Various domain (telecom in this case) terms or concepts such as equipment types (gateway, router, receiver), problems (connection issues, no audio etc.), associated conditions (correct cabling, receiver configuration, etc.), abbreviations etc.
� Concepts or terms expressed in different ways (receiver Vs STB Vs DVR, router Vs gateway etc.)
� Timing in terms of when a particular problem is more prevalent post service install & activation
� Statistics in terms of how many people were affected over the years.
� Locations and other geo references made in the document with respect to the concepts or terms.
� Co-references or relationships or associated service problems with other conditions or issues (DVR Recording & Memory Capacity, Data Features & Connectivity etc.), upgrades, value added options etc.
Content crawling & curation – Example content
59 © 2014 International Business Machines Corporation
Watson Experience Manager – Corpus management
� Content:
� Content Upload & Management – Not an ECM tool
� View Ingested Corpus
� Allows for on-demand corpus pre-processing & ingestionNumber of
Docs Ingested
as Corpus
View Ingested
Document &
CAS
Tag Each
Document
Ingest into
Corpus on-
demand
Upload &
Delete
Documents
60 © 2014 International Business Machines Corporation
Train Watsonwith tools and experts
1. Watson is a cognitive system that is trained by experts and users and learns over time
2. Watson Experience Manager allows domain experts to provide training data like question and answer pairs
3. Watson is trained iteratively – follows a cycle of collecting realistic training data, running training experiments, testing, error analysis.
Key points
Access Watson Developer Cloud
using Watson Experience Manager
Develop app
“Powered by
Watson”using APIs
Enrich Watsonwith content
Train Watson using tools and experts
Test app
functional and
non-functional
Deploy
application
61 © 2014 International Business Machines Corporation
� Training:
� Allow SME’s to enter & manage “Question & Answer” pairs
� Review & Approve Q&A Pairs
� Forms the basis for training data to help train Watson
� Goal is to build a representative set that enables Watson to understand the domain's language characteristics
Drive towards a
goal
Approve, Edit,
Add Comments
Sample Q&A
Pair Instance
Watson Experience Manager – Create Q&A pairs to help train Watson
62 © 2014 International Business Machines Corporation
� Suite of tools for training and testing Watson
� Test, train, and improve the accuracy of the system
� SDK and a set of extensions points to develop tools to add to the platform
Training
Experiment,
Error Analysis
Key tools & capabilities – Watson common tooling platform
63 © 2014 International Business Machines Corporation
Test “Powered by Watson” apps (Functional and non-functional)
1. Watson Experience Manager offers ways to test the responses from Watson
2. Testing is an iterative process to improve responses; and test integration with Watson
3. In addition to functional tests, plan to perform stress tests based on user ramps, and other non functional tests.
Key points
Access Watson Developer Cloud
using Watson Experience Manager
Develop app
“Powered by
Watson”using APIs
Enrich
Watsonwith
content
Train
Watson using tools
and experts
Test app
functional andnon-functional
Deploy
application
64 © 2014 International Business Machines Corporation
Watson Experience Manager – Testing
� Testing
� Test system by asking questions
Additional
Considerations
Confidence of
the response
Evidence -
where the
answer
passage was
provided
65 © 2014 International Business Machines Corporation
Deploy “Powered by Watson”applications
1. Roll out the application using the production topology
2. Set up support teams and processes for handling user queries
3. Establish a set of metrics to be collected, with tools to support the process.
Key points
Access Watson Developer Cloud
using Watson Experience Manager
Develop app
“Powered by
Watson”using APIs
Enrich Watsonwith content
Train
Watson using tools
and experts
Test app
functional and
non-functional
Deploy
application
66 © 2014 International Business Machines Corporation
Deploying your “Powered by Watson” solution
• Rollout iteratively
•Create a rollout plan with the user base
•Size the system based on peak and scaled usage, failover and availability requirements and response times
•Establish SLAs and other metrics needed from the Watson Developer Cloud
•Determine any additional security provision needed
•Promote the powered by Watson application to the production zone in the Watson Developer Cloud
67 © 2014 International Business Machines Corporation
Summary - 3 key takeaways
1
2
3Committed to partner success. New business unit, HQ in NY,
brings together research, development, service and go-to-market and features Innovation Hub with incubator, client solutions center and design lab
Consider the possible interaction patterns when designing your Watson app to create the optimal user experience (including factoids, descriptive, Yes/No, procedural, procedural troubleshooting and definitions).
The Watson Experience Manager (WEM), provides a platform to upload content, train Watson with question and answer pairs, and
test the system by asking questions.
The process for building a Watson app involves an iterative process of integrating Watson capabilities using the QAAPI, uploading content, training Watson, and testing the application for the appropriate response.
Wrap up: Where Do I Go From Here?
69 © 2014 International Business Machines Corporation
Where do I go from here?
Apply
https://ibm.biz/watsonecosystemapply
Share:
https://ibm.biz/watsondeveloper
Learn:
https://ibm.biz/watsonecosystem-youtube