Date post: | 21-Oct-2014 |
Category: |
Business |
View: | 1,807 times |
Download: | 2 times |
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
BayesiaLab’s Knowledge Elicitation Environment
An innovative Brainstorming Tool
Dr. Lionel JOUFFE
May 2010
1
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission2
All models are wrong; the practical question is how wrong do they have to be to not be useful (Box&Draper 87)
MODELING BY BRAINSTORMINGMODELING BY BRAINSTORMING
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission3
Designing a Model for Decision Support
Every Company is faced to complex decisions that need to be rationally supported
Sometime, there are too few data available, or no data at all, to allow using data mining and data analysis technics to automatically build a Decision Support System
Experts have gathered invaluable Tacit Knowledge through their experience
We need to Convert this Tacit Knowledge into Explicit Knowledge and use it to build a model
We want actionable models to allow What-if scenarios (simulation and/or diagnosis), drivers analysis, ...
Bayesian Belief Networks (BBNs) are ideal models for such problematics: their graphical representation allows a manual design by using expert knowledge, and their probabilistic engines offer powerful simulation capabilities
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission4
BBNs are made of Two Distinct Parts
StructureDirected Acyclic Graph (DAG), i.e. no directed loop
Nodes represent the variables
Each node has a set of exclusive states (e.g.: Young, Adult, Aged)
Arcs represent the direct probabilistic influences between the variables (possibly causal)
ParametersProbability distributions are associated to each node, usually by using tables
CONDITIONAL PROBABILITY DISTRIBUTION
A smoker has a 60% of risk of suffering from a Bronchitis, whereas the risk of
a non smoker is 30% only
MARGINAL PROBABILITY DISTRIBUTIONWe consider a population made
of 40% of Adults
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission5
BBNs are Powerful Inference Engines
We get some evidence on the states of a subset of variables: Hard positive and negative evidence, Likelihoods, Probability distributions, Mean values
We take these findings into account in a rigorous way to update our belief on the states of all the other variables
Probability distributions on their values
Multi-Directional Inference (Simulation and/or Diagnosis)
The evidence on Smoker (a new probability distribution) allows to update the probability distribution of Age (Diagnosis) and Bronchitis (Simulation)
Prior Distribution Posterior Distribution
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission6
BBN Modeling by Brainstorming
Clear definition of the BBN’s objective(s) (e.g.: Improvement of the Product/Service Quality, improvement of the Purchase Intent, improvement of the Company’s performance, ...)
Identification of the conceptual dimensions that are linked to those objectives (e.g.: Human resources, Management, Production, Marketing, ...)
Definition of the group of experts that will fully cover all the dimensions (and the different geographical zones), with a small redundancy to allow fruitful debates
Brain Storming Sessions with this group of Experts to manually build the BBN
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The StructureThe Directed Acyclic Graph
7
For each identified conceptual dimension
Definition of the main variables
Definition of the exclusive states of those variables
Creation of one node per identified variable
Brainstorming to define the direct relationships between the variables, and addition of the corresponding arcs between those dependent variables
The structure elicitation is probably the simplest task of the Brainstorming session
For each root node, i.e. without incoming arc, definition of the marginal probability distribution over the defined states
For each node with incoming arc(s), definition of the conditional probability distribution over the defined states, for each combination of the states of its connected nodes
Each expert gives his/her belief on the distributions
There are various kinds of biases to be aware of
Cognitive (Plausibility, Control, Availability, Anchoring) Emotional (Mood, Motivation) Group (Anchoring, Herding) Facilitator (can be biased toward charismatic experts or toward
the last expressed opinion)
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The ParametersProbability Distributions
8
Probabilities do not have to be exact to be useful
☛ Use the new BayesiaLab’s Knowledge Elicitation environment to reduce these biases, to improve traceability, to gather all the
useful knowledge, ....
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission9
BAYESIALAB 5.0Knowledge Elicitation Environment
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Experts
Definition of the group of Experts
10
- Group of experts can be Imported/Exported- The Open Session button allows opening an Online Brainstorming Session*
- The Generate Tables button allows generating a Bayesian network by using the assessments of the selected experts only
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Experts
* Available on subscription only
This Expert Editor allows defining:The Expert’s name, its Credibility (that will be use globally during the consensus computation), her/his Picture, a Comment to describe her/his area of expertise. The last field contains the number of assessments realized by the expert on the
current network
11
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Experts’ Assessments
Selecting a cell in the probability table activates the Assessment button for assessing the question corresponding to the selected line, i.e. what is the marginal probability
distribution of Age over the 3 defined states?
12
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Experts’ Assessments
Pressing the Assessment button opens the Assessment Editor that allows the Facilitator to manually add, delete and modify Experts’ Assessments.
The Post Assessment button can be used by the Facilitator to Post the question to the BayesiaLab’s secured website for an online assessment
13
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Expert Online Assessment Tool
Once logged in, the Expert is waiting for a
question
The secured website
The Expert’s name, case sensitive!
The session name
14
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Expert Online Assessment Tool:Example with the 3 states variable Age
Once the Facilitator has posted the question with the Assessment Editor, the question is displayed on the Expert’s webpage.
There is no context (root node). This is then a
marginal probability
Check box for fixing the probability of the
state
The question is relative to the node “Age”, that has 3 states: Young,
Adult and Aged.There are then 3 sliders for the probability
distribution assessment, and another one for the confidence
Pie Chart representing the
probability distribution specified with the sliders
The label corresponds to the
Confidence level the expert has specified with the
Confidence Slider (ranging from “I Do not Know” to “I
am Certain”)
The comment field can be
used for explaining the assessment
15
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Expert Online Assessment Tool:Example with the binary variable Cancer
16
The context variables in the BBN
This question is relative to node Cancer, and the specific Context is “Age = Adult” and
“Smoker = Yes”
Hovering over the context variables returns the comment
associated to the corresponding node, if any
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Facilitator’s tool
Once the Expert validates her/his assessment, this assessment is sent to the BayesiaLab’s server and the Facilitator’s listener is automatically updated
This listener allows following the status of the
Experts’ assessments
Clicking on OK makes BayesiaLab harvesting the
assessments. Closing the window cleared the question from the webpage of the Experts that do not have
17
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Facilitator’s tool
This gray part corresponds to the Experts’ probability distribution
assessments
This second part contains the Expert’s name, the Assessment’s
Confidence, the associated Comment and the Time (in second) for validating
the assessment
18
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Facilitator’s tool
The content of this editor is sortable by each column just by clicking on the corresponding header
It is sorted here in the ascending order on the probabilities assessed for the
state Young
Selecting the line allows displaying the Expert’s picture
Sorting the assessments by state probabilities can be used for:- detecting Experts’ misunderstanding
- Knowledge sharing, especially by making the 2 “extremes” Experts debate
If some useful knowledge comes out from the debate, the Facilitator can post again the question for a new Expert Assessment. Each Expert will then be
allowed to update her/his assessment online (each Experts’ webpage is initialized with the information she/he set in the previous round)
19
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Consensus
A small icon is added at the left of each probability to graphically
represent the consensus degree:from a full transparency when there
all the Experts agree on the probability, to no transparency when the range of the assessments is 1
Once the assessments validated, a Mathematical consensus is computed by using the Experts’ credibility and their assessment’s confidence. This automatic consensus can be manually
modified by the Facilitator to set a Behavioral consensus, i.e. one issued after a fruitful debate
Hovering over this icon returns the minimum and the
maximum assessments, and the number of assessments
20
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Consensus
The probability distributions that have a set of assessments are framed
with a green line
An icon is added to the nodes for indicating the nodes that have Experts assessments. The darker the icon is, the lower the global consensus is
Here is the list of assessments corresponding
to the second line of the table
21
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Consensus
Pressing the “i” key while hovering over the expert icon allows displaying
the information panel below
This information panel contains:- the number of rows ((Conditional) probability distributions) that comes with Experts
assessments- the total number of assessments that have been set in the probability table
- the number of Experts that have assessed at least one probability distribution in the table - a measure of the global disagreement that takes into account the deviations from the
mathematical consensus- the maximum disagreement corresponding to the greatest difference between two
assessments in the probability table
22
iiiiiii
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
The Assessment Report
Right clicking on the Expert Icon in the lower left corner of the Graph window allows generating the
following HTML report. This report first gives information on the Experts,
then returns a sorted list of the nodes wrt the global disagreements, and another one wrt the
maximal disagreements.Finally, for each node, a summary contains all the
global information on the assessments of the (Conditional) Probability Table
All these informations can be useful for the Model Validation, e.g. by checking first the nodes based
on their associated disagreements (global and maximal), then based on the time for the
assessments (that can reflect a difficulty, or, on the contrary, too prompt assessments)
23
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of a Bayesian Network per Expert
This exportation tool allows to create a Bayesian Belief Network for each Expert.
The parameters (probabilities) are those assessed by the Expert. If the Expert has not assessed all the probabilities, the model will use either the
consensual probabilities, or those manually entered by the Facilitator
24
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of the Probability Assessments
This exportation tool allows creating a CSV file with all the assessments of the
probabilities. There is one column per variable to describe the context (yellow), one
column to indicate the assessed Node (green), the other columns describing the assessed probability, the confidence level,
the Expert, and the assessment time.Each line describes one assessment of a
(Conditional) Probability Table cell
MitA/TiPo MiAt TiPo Node Probability Confidence Expert Time
Weak Weak Strong MitA/TiPo 0,97 1 Hiro 56
Strong Weak Strong MitA/TiPo 0,03 1 Hiro 56
Weak Weak Strong MitA/TiPo 0,95 1 Haitien 210
Strong Weak Strong MitA/TiPo 0,05 1 Haitien 210
Weak Weak Strong MitA/TiPo 0,8 0,58 Claire 145
Strong Weak Strong MitA/TiPo 0,2 0,58 Claire 145
Weak Weak Strong MitA/TiPo 0,85 0,77 Matt 65
Strong Weak Strong MitA/TiPo 0,15 0,77 Matt 65
Weak Weak Strong MitA/TiPo 0,4 0,8 Mohinder 76
Strong Weak Strong MitA/TiPo 0,6 0,8 Mohinder 76
Weak Weak Strong MitA/TiPo 0,75 0,9 Nathan 50
Strong Weak Strong MitA/TiPo 0,25 0,9 Nathan 50
... ... ... ... ... ... ...
Weak MiAt 0,7 0,2 Noah 76
Strong MiAt 0,3 0,2 Noah 76
Weak MiAt 0,75 1 Matt 90
Strong MiAt 0,25 1 Matt 9025
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Exportation of the Expert Assessments
This exportation tool generates a CSV file with all the assessments of the Experts.
There is one column per Expert, one column per Expert’s Confidence (yellow), the last column indicating the weight of the line (1/number of states of the assessed variable)
(green).Each line describes the Experts’ assessment
of a(Conditional) Probability Table cell
Hiro Hiro Confidence
Haitien Haitien Confidence
.... Noah Noah Confidence
Weight
0,97 1 0,95 1 .... 0,7 0,8 0,5
0,03 1 0,05 1 .... 0,3 0,8 0,5
0,3 0,81 0,05 1 .... 0,3 0,7 0,5
0,7 0,81 0,05 1 .... 0,7 0,7 0,5
0 1 0 1 .... 0 0,79 0,5
1 1 1 1 .... 1 0,79 0,5
0,65 0,79 0,71 1 .... 0,7 0,2 0,5
0,35 0,79 0,29 1 .... 0,3 0,2 0,5
26
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Analysis of the Expert Assessments
Each node represents the discretized probabilities
assessed by the Expert
27
We then can used this file to analyze the direct probabilistic relationships that hold between the Experts’ assessments
We then can used this file to analyze the direct probabilistic relationships that hold between the Experts’ assessments
This network has been automatically learned with one of
the BayesiaLab’s Association Discovering algorithms on a set of 120 Experts’ assessments
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Automatic Segmentation of the Experts
Each color corresponds to a cluster. Three segments of Experts have
been induced in that example. The real experts behind those anonymized
experts have indeed three different profiles (functionally and geographically)
Dendrogram corresponding to that
segmentation
Based on the obtained Expert Segments, one Bayesian network per segment can be generated (by using the Expert Editor). This can be useful for analyzing the sensibility of the model, but also to get specific networks (depending on the geographical localization
for example)
28
Based on the obtained network, Experts can be clustered into homogeneous groups by using the BayesiaLab’s Variable Clustering algorithm
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Parameter Sensibility Analysis
BayesiaLab also comes with an Assessment Sensitivity Analysis tool that allows measuring the uncertainty associated to the consensus.
The general idea is to generate a set of networks by randomly drawing Experts’ assessments, and then measuring the uncertainty
associated to each probability distribution.
29
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Parameter Sensibility Analysis
Three kinds of analysis are available, depending on the Random selection policy that is chosen to generate the set of networks (1000 networks in the above example):
- One Expert per network: each network generated is parametrized by using the selected Expert (or the consensual probability if the selected Expert has not been involved in the
assessment)- One Expert per node: each network generated is parametrized by selecting for each node one Expert. If the selected Expert is not involved, the consensual probability if the selected
- One assessment per Conditional Probability Table’s row (if any)
30
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Parameter Sensibility Analysis
States of the analyzed variable
Marginal probability distribution of the Target node computed with
all the consensus
Density function illustrating the uncertainty associated to this node. The Mean over the 1000 networks (one Expert
per network) is 70.62% (versus 70.46% in the monitor), the Standard
Deviation 2.13%. There are 62% of chance of having a
probability comprise between 70 and 72%
31
Tabs for the selection of the variable under analysis
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Parameter Sensibility Analysis
One Expert per node
One Expert per Conditional Probability row
32
©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express
written permission
Plan
Modeling by Brainstorming
BAYESIALAB 5.0Knowledge Elicitation Environment
Contact
33
6 rue Léonard de Vinci BP0119
53001 LAVAL CedexFRANCE
Dr. Lionel JOUFFEPresident / CEO
Tel.: +33(0)243 49 75 58Fax: +33(0)243 49 75 83