Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | marcia-phelps |
View: | 232 times |
Download: | 13 times |
TMTM
Exposure Data Quality and Exposure Data Quality and Catastrophe ModelingCatastrophe Modeling
Rick AndersonRick Anderson
February 28, 2002February 28, 2002
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Quality IssuesData Quality Issues
Is the insurance to value being accurately reflected? Is the insurance to value being accurately reflected? Does my data capture my actual exposure on a Does my data capture my actual exposure on a
regional and peril basis? regional and peril basis? Do I understand the default assumptions in my data? Do I understand the default assumptions in my data? Do I know that the information my brokers and Do I know that the information my brokers and
agents are providing me is correct? agents are providing me is correct? Am I capturing my aggregate information correctly?Am I capturing my aggregate information correctly?
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Statement of the ProblemStatement of the Problem
What is the impact of poor data quality on:What is the impact of poor data quality on:– Exposure data valuesExposure data values– Uncertainty in modeled lossesUncertainty in modeled losses– Business decisions (external and internal)Business decisions (external and internal)
How do I quantify / score data qualityHow do I quantify / score data quality– On a location basisOn a location basis– On a policy basisOn a policy basis– On an aggregate portfolio basisOn an aggregate portfolio basis
How do I optimize data quality given my current business How do I optimize data quality given my current business constraints?constraints?
What improvements should I be making?What improvements should I be making?
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Tackling the ProblemTackling the Problem
Close working relationship with business partnersClose working relationship with business partners
– AgentsAgents
– ReinsurersReinsurers
– ModelerModeler Development of a structured data quality assessmentDevelopment of a structured data quality assessment Ability to identify specific data quality issues and their Ability to identify specific data quality issues and their
impact on portfolio risk assessment at all levels.impact on portfolio risk assessment at all levels. Development of a consistent independent data Development of a consistent independent data
quality measurequality measure
– Data Quality Index (DQI)Data Quality Index (DQI)
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Quality in the Context of Data Flow Data Quality in the Context of Data Flow Primary Insurance Company PerspectivePrimary Insurance Company Perspective
Pricing,Reinsurance,
Cap. Allocation,etc.
Exposure Database
ProductionStream
Cat ModelAnalysis
Data Acquisition(Source Data)
Data Resolution Analysis
Process/ Operational Accuracy Analysis
Data Acquisition Accuracy Analysis
What does it mean?What matters?
DA
TA
QU
AL
ITY
DA
TA
FL
OW
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Components of Data QualityComponents of Data Quality
Accuracy componentAccuracy component Resolution componentResolution component
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data AccuracyData Accuracy
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Examining the Components of Examining the Components of Exposure Data Quality: Exposure Data Quality: Data AccuracyData Accuracy
How accurately is my data being captured and How accurately is my data being captured and processed?processed?
Examination of processes through interviews and Examination of processes through interviews and exposure data queriesexposure data queries – Data acquisitionData acquisition– Data processingData processing– Operations / systemsOperations / systems
Market dependentMarket dependent
Logic tree assessment framework Logic tree assessment framework
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Accuracy ComponentsData Accuracy Components
Data acquisition (source of data origination)Data acquisition (source of data origination)– Conditional on type of source and line of Conditional on type of source and line of
businessbusiness– Source reputation / biasSource reputation / bias– Source data vintage / validity / consistency / Source data vintage / validity / consistency /
interpretabilityinterpretability Data processingData processing
– Conditional on line of businessConditional on line of business– Bias / vintage / validity / consistency / Bias / vintage / validity / consistency /
interpretabilityinterpretability Operations / systemsOperations / systems
– Data accessibility / data integration / systems Data accessibility / data integration / systems process / operations value to costprocess / operations value to cost
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Accuracy Component of Data Quality Accuracy Component of Data Quality Assessment FrameworkAssessment Framework
1. Data Acquisition
2. Data Processing
3. Operations
Accuracy Component Data Quality
Questionnaire 1
Questionnaire 3
Questionnaire 2
W3
W2
W1
Warning flags from queries of the exposure database
Peril and LOB specific On-Site Questions
Components of Data Flow
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Example Logic Tree with Data Accuracy CriteriaExample Logic Tree with Data Accuracy Criteria
Data Data AcquisitionAcquisitionAccuracy Accuracy ScoreScore
Reputation/BiasReputation/Bias
Data Data
ReputationReputation
BiasBias
VintageVintage
ValidityValidity
ConsistencyConsistency
InterpretabilityInterpretability
Question 1Question 1Question2Question2
..
..
..
Question 11Question 11Question 12Question 12
..
..
..
Question 29Question 29Question 30Question 30
..
..
..
0.30.3
0.70.7
0.30.3
0.50.5
0.50.5
0.20.2
0.40.4
0.10.1
DirectDirectIndependent AgentIndependent AgentWholesale BrokerWholesale BrokerRetail BrokerRetail BrokerRisk Retention GroupRisk Retention Group
Integrated Data SubmissionIntegrated Data SubmissionCatastrophe Model EDMCatastrophe Model EDMDigital (Spreadsheet, Word Doc, etc.)Digital (Spreadsheet, Word Doc, etc.)Paper SubmissionPaper Submission
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Development of Data Accuracy Criteria Relative Development of Data Accuracy Criteria Relative Importance WeightsImportance Weights
Assessed as relative impact on modeled losses and Assessed as relative impact on modeled losses and key data quality issueskey data quality issues
Based on:Based on:– Extensive interviews with Cat managers, Extensive interviews with Cat managers,
underwriters and systems personnelunderwriters and systems personnel– Results of relative parameter impact analyses on Results of relative parameter impact analyses on
AAL (data validity criteria)AAL (data validity criteria)– Availability of other information from which to draw Availability of other information from which to draw
assumptionsassumptions
Line of business, peril, and region dependentLine of business, peril, and region dependent
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Accuracy Criteria Data Accuracy Criteria Development of QuestionnaireDevelopment of Questionnaire
Questionnaire is administered through interview processQuestionnaire is administered through interview process
Questions are multiple choiceQuestions are multiple choice– Yes / NoYes / No– Always / Most of the Time / Occasionally / NeverAlways / Most of the Time / Occasionally / Never
Number and content of questions designed to Number and content of questions designed to adequately assess how criteria are addressed at adequately assess how criteria are addressed at companycompany
Normalized relative importance weighting applied to Normalized relative importance weighting applied to questions within each criteriaquestions within each criteria
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Warning Flags Summaries from DB QueriesWarning Flags Summaries from DB Queries
Used as supporting information in answering Used as supporting information in answering questionnairequestionnaire
Warning flagsWarning flags– Data consistencyData consistency
• Address entryAddress entry• ValuesValues• Construction and occupancy class/schemaConstruction and occupancy class/schema
– Data vintageData vintage– Data biasData bias
• Secondary characteristicsSecondary characteristics• Primary characteristicsPrimary characteristics
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Warning Flags Summaries from DB Queries – Warning Flags Summaries from DB Queries – Sample ResultsSample Results
Data Vintage – Use of Policy Status FlagData Vintage – Use of Policy Status Flag
StatusStatus # of Policies# of Policies % of Total Policies% of Total Policies
BOOKBOOK 11 16.7%16.7%
No Status No Status 55 83.3%83.3%
Data Consistency – Value Entry CheckData Consistency – Value Entry Check
AddressAddress Total Total TotalTotal AverageAverage MatchMatch LocationsLocations ValueValue ValueValue
Street LevelStreet Level 5 5 $1,270,000 $1,270,000 $254,020$254,020
Zip LevelZip Level 1 1 $147,500 $147,500 $147,500$147,500
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
1. Data Acquisition Accuracy1. Data Acquisition Accuracy
Conditional on type of provider of data, data Conditional on type of provider of data, data format, submission process, and line of businessformat, submission process, and line of business
Data acquisition accuracy componentsData acquisition accuracy components
– ValidityValidity
– VintageVintage
– Data provider bias Data provider bias
– Data provider reputation Data provider reputation
– ConsistencyConsistency
– InterpretabilityInterpretability
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Acquisition Criteria: Relative ImportanceAcquisition Criteria: Relative Importance
Data vintageData vintage Location validity checksLocation validity checks Default value treatmentDefault value treatment Data acquisition biasData acquisition bias Data validity checksData validity checks Use of data alteration flagsUse of data alteration flags Data aggregationData aggregation Location entry consistencyLocation entry consistency Reputation of data providerReputation of data provider Secondary construction characteristics Secondary construction characteristics
treatmenttreatment
HighHigh
LowLow
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
2. Data Processing2. Data Processing
Conditional on database format, platform, and Conditional on database format, platform, and line of businessline of business
Incorporates results from queries of exposure Incorporates results from queries of exposure databasedatabase
Data processing accuracy componentsData processing accuracy components– BiasBias– ValidityValidity– InterpretabilityInterpretability– VintageVintage– ConsistencyConsistency
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
3. Systems / Operations Accuracy3. Systems / Operations Accuracy
Processing / operations data quality componentsProcessing / operations data quality components
– Data accessibility and storageData accessibility and storage
– Data integration and linkingData integration and linking
– Technology systems process/flowTechnology systems process/flow
– Operational value-to-costOperational value-to-cost
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Accuracy - SummaryData Accuracy - Summary
Assessment of how closely processes arrive at the Assessment of how closely processes arrive at the true and accepted valuetrue and accepted value
Structured and consistent approachStructured and consistent approach
Ability to assess the contribution of individual Ability to assess the contribution of individual components to overall data accuracy score components to overall data accuracy score
Periodic assessment is valuable for internal process Periodic assessment is valuable for internal process reviewreview
Integral component to overall data qualityIntegral component to overall data quality
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data ResolutionData Resolution
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Examining the Components Of Examining the Components Of Exposure Data Quality: Exposure Data Quality: Data ResolutionData Resolution
What data am I capturing and at what level?What data am I capturing and at what level?
Direct query of exposure data parametersDirect query of exposure data parameters – GeocodingGeocoding– ConstructionConstruction – OccupancyOccupancy – Year builtYear built – Building heightBuilding height – Construction modifiersConstruction modifiers
Peril, region and market dependentPeril, region and market dependent
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution Analysis Tree - Data Resolution Analysis Tree - California Earthquake ResidentialCalifornia Earthquake Residential
Cladding
HAZARD VULNERABILITY
Coordinate
Zip Code
County
Location Resolution
Const. Scheme
Occupancy Class
Secondary Characteristics
Construction Class
Year Built Number of Stories
Frame Bolted Down
Soft Story
Unknown
URM Chimney
Cripple Walls
UnknownInventoryRMS
ISO Fire Known
Unknown
Known
UnknownMFW Frame
SFW Frame
SF House
MF Housing
LOCATION
ATC
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Florida HU – Location Sampling (10 km. Grid)Florida HU – Location Sampling (10 km. Grid)
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Florida HU – Location Sampling (1 km. Grid)Florida HU – Location Sampling (1 km. Grid)
COLLIER
MONROE DADE
BROWARD
PALM BEACH
MARTIN
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Development of Category WeightsDevelopment of Category Weights
Weights for individual categories are determined Weights for individual categories are determined through numerical simulation (analysis) of the impact through numerical simulation (analysis) of the impact of a given category on losses for the geography, of a given category on losses for the geography, peril, and LOB under considerationperil, and LOB under consideration
Final weights are normalized across the applicable Final weights are normalized across the applicable categoriescategories
CategoryCategory HighHigh Med.Med. LowLow
GeocodingGeocoding ww1a1a w w1b1b w w1c1c
Cons. SchemeCons. Scheme ww2a2a w w2b2b w w2c2c
Year BuiltYear Built ww5a5a w w5b5b w w5c5c
22ndnd. Char.. Char. ww6a6a w w6b6b w w6c6c
Extensive testing, validation, and benchmarkingExtensive testing, validation, and benchmarkingperformedperformed
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Florida HU – Scoring RegionsFlorida HU – Scoring Regions
Scoring Region by Hazard
Very HighHighMediumLow
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
California EQ – Scoring RegionsCalifornia EQ – Scoring Regions
I-5
Scoring Region by Hazard
HighMediumLow
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Resolution Geocoding Scores by Hazard Level Resolution Geocoding Scores by Hazard Level California Earthquake Residential California Earthquake Residential
100
75
25
100
80
30
100
85
40
0
10
20
30
40
50
60
70
80
90
100
Sco
re b
y G
eoco
din
g L
evel
High Hazard Medium Hazard Low hazardHazard Level
coordinates
street
zip
city
county
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution Category Score Summary – Data Resolution Category Score Summary – California Earthquake Residential High Hazard RegionCalifornia Earthquake Residential High Hazard Region
Category Attribute
Name Attribute
Score Catergory
Weight Weighted
Score Unknown 0 0.75 0.00 Coordinate 100 0.75 75.00 Street Address 95 0.75 71.25 Postal Code 80 0.75 60.00 City 25 0.75 18.75 County 25 0.75 18.75 State 0 0.75 0.00
Location Resolution
Cresta 0 0.75 0.00 RMS 100 0.01 1.00 ATC 100 0.01 1.00 ISO 85 0.01 0.85
Construction Scheme
ISO FIRE 75 0.01 0.75 Known 100 0.05 5.00 Construction Class Unknown 75 0.05 3.75 Known 100 0.04 4.00 Occupancy Class Unknown 70 0.04 2.80 Known 100 0.05 5.00 Year Built Unknown 50 0.05 2.50 Known 100 0.10 10.00 Few Unknown 60 0.10 6.00
Secondary Characteristics
Unknown 25 0.10 2.50
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution Category Weights Data Resolution Category Weights
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
High
Med
Low
Sei
smic
Reg
ion
s
Score (%)
Geocoding
Construction Scheme
Construction Class
Occupancy Class
Year Built
Secondary Chars.
California Earthquake Residential
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution Category WeightsData Resolution Category Weights
Florida Hurricane Commercial
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Very High
High
Med
Low
Ha
zard
R
eg
ion
s
Score (%)
Geocoding
Construction SchemeConstructionClassOccupancy Class
Number of Stories
Secondary Chars.
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution - Aggregation MethodologyData Resolution - Aggregation MethodologyProgression of Data Resolution ScoringProgression of Data Resolution Scoring
Account 1Account 1 Account 2Account 2 Account 3Account 3
Commercial Commercial PortfolioPortfolio
LocationLocation
PortfoliPortfolioo
PP11
ScoreScore
AA11 AA33AA22
LL11
LL22
LLnn
LL11
LL22
LLnn
LL11
LL22
LLnn
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Data Resolution - Data Resolution - Development of Relative “Importance” FactorsDevelopment of Relative “Importance” Factors
Relative importance is an approximation of the AAL.Relative importance is an approximation of the AAL. Ground-up AAL approximated at ZIP code level Ground-up AAL approximated at ZIP code level
based on insurance industry exposure.based on insurance industry exposure. Gross AAL approximated by average ratio of gross / Gross AAL approximated by average ratio of gross /
ground-up AAL per attachment point.ground-up AAL per attachment point.
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Layer contribution to AALLayer contribution to AAL
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 10% 20% 30% 40% 50%
Attachment Point (% of Value)
Per
cent
of
AA
L
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Sample Resolution ScoresSample Resolution Scores
- 10 20 30 40 50 60 70 80 90 100
Company C
Company B
Company A
Score (%)
Geocoding
Construction Scheme
Construction Class
Occupancy Class
Number of Stories
Secondary Chars.
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Improving Data Resolution – Improving Data Resolution – Leveraging Account Data Resolution ScoresLeveraging Account Data Resolution Scores
Identify accounts with score less than than target Identify accounts with score less than than target scorescore
Determine account potential for improvement as:Determine account potential for improvement as:
(Score Difference) * (Account Importance)(Score Difference) * (Account Importance)
Identify accounts with biggest improvement potential Identify accounts with biggest improvement potential and decide on strategy for data improvementand decide on strategy for data improvement
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Improving Data Resolution – Targeting AccountsImproving Data Resolution – Targeting Accounts
Score Difference = Target Score - Account ScoreScore Difference = Target Score - Account Score Potential Improvement = (Score Difference) * (Importance)Potential Improvement = (Score Difference) * (Importance)
Target score 82.25
Account ScoreScore
Difference ImportancePotential
ImprovementSV Office Center 80.21 2.04 23,517,279 47,975,249So Cal Management 79.92 2.33 16,329,576 38,047,912Crown Ltd 80.54 1.71 9,816,495 16,786,206Putt Putt Motors 86.86 -4.61 1,460,773 -6,734,164Beach Apartments 87.78 -5.53 6,425,708 -35,534,165Palo Alto Condos 86.88 -4.63 13,022,979 -60,296,393
Average 82.25
© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.
Combining the ComponentsCombining the Components