of 12
8/10/2019 Chapter 10 Validation and Verification
1/12
Validation and Verification
Adapted from JerryBanks
8/10/2019 Chapter 10 Validation and Verification
2/12
Verification
Concerned with building the modelright
Comparison of conceptual model andcomputer representation
Is the model implemented correctlyin the computer?
Are the inputs and logicalparameters represented properly?
8/10/2019 Chapter 10 Validation and Verification
3/12
Validation
Concerned with building the rightmodel
Accurate representation of the realsystem
This is achieved through thecalibration of the model
Iterative process until accuracy isacceptable
8/10/2019 Chapter 10 Validation and Verification
4/12
ModelBuilding,Verification,
andValidation
REAL SYSTEM
Conceptual Model
1Assumptions on system components
2Structural Assumptions (defines the interactions
between the system components)
3Input parameters and data assumptions
Operational Model(Computer
Representation)
Conceptual
Validation
Model
Verification
Calibration and
Validation
8/10/2019 Chapter 10 Validation and Verification
5/12
Common sense suggestionsfor verification
Have someone check thecomputerized model
Make a flow diagram (with logicalactions for each possible event)
Examine model output forreasonableness
Print the input parameters at theend of the simulation
8/10/2019 Chapter 10 Validation and Verification
6/12
8/10/2019 Chapter 10 Validation and Verification
7/12
Three Classes of Techniquesfor Verification
Common sense techniques
Thorough documentation
Traces
8/10/2019 Chapter 10 Validation and Verification
8/12
Calibration and Validation
Validation is the overall process ofcomparing the model and its
behavior to the real system and itsbehavior
Calibration is the iterative process of
comparing the model to the realsystem and making adjustments tothe model, and so on.
8/10/2019 Chapter 10 Validation and Verification
9/12
Iterative Process ofCalibration
REAL SYSTEM
Initial Model
Second
Revision of
Model
First Revision of
Model
Compare Model to
Reality
Compare RevisedModel to Reality
Compare second
Revised Model to
Reality
8/10/2019 Chapter 10 Validation and Verification
10/12
3 Step Approach by Naylorand Finger (1967)
Build a model with high face validity
Validate model assumptions
Compare the model input-outputtransformations to correspondinginput-output transformations of the
real system
P ibl lid ti t h i i
8/10/2019 Chapter 10 Validation and Verification
11/12
Possible validation techniques inorder of increasing cost-valueratio by Van Horn (1971)
High face validity. Use previousresearch/studies/observation/experience
Conduct statistical test for datahomogeneity, randomness, and goodnessof fit test
Conduct Turing test. Have a group ofexperts compare model output versussystem output and detect the difference
Compare model output to system outputusing statistical tests
P ibl lid ti t h i i
8/10/2019 Chapter 10 Validation and Verification
12/12
Possible validation techniques inorder of increasing cost-valueratio by Van Horn (1971)
After model development, collectnew data and apply previous 3 tests
Build a new system or redesign theold one based on simulation resultsand use this data to validate the
model Do little or no validation. Implement
results without validating
