Presenters Vincent J. Granieri, FSA, EA, MAAA Leonard ... Albert Jeffrey Moore, ASA, MAAA Presenters...

Moderator:

Albert Jeffrey Moore, ASA, MAAA

Presenters Vincent J. Granieri, FSA, EA, MAAA

Leonard Mangini, FSA, FALU, FRM, MAAA Albert Jeffrey Moore, ASA, MAAA

Neil Raden Paul C. Ramirez, FSA, MAAA

1 11

Managing the Big Data OpportunityManaging the Big Data OpportunitySponsored by the Technology Section of the Society of Actuaries

May 6, 2015New York, NY

Sponsored by the Technology Section of the Society of Actuaries

May 6, 2015New York, NY

2 22

Session 1: What’s the Big Deal About Big Data

Paul Ramirez FSA, MAAA Neil Radon

What’s the Big Deal About Big Data?

Paul Ramirez, FSA, MAAA

Managing the Big Data Opportunity

May 6th, 2015

4

What Does “Big Data” Mean?

• “Big Data” has become a nebulous term

• Senior management want to take advantage of big data, but do they really know what this means?

Any volunteers on taking a stab at what big data means to them?

5


• Big Data refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze – McKinsey Global Institute

• 90% of the world’s data was created in the last two years – IBM

• The definition implies that the concept of big data is transient.

6


• High Volume• Very “deep” data – hundreds of characteristics about a

person, for example

• High Velocity• Frequently arriving data. This presents a challenge in both

analysis and storage

• High Variety• Unstructured• From multiple data sources• Images, text, video, audio, weblogs, browsing data


• Often an automated data source• Created a sensor or machine

• Many times a new source of data• Web‐browsing data, tracking data

Now we know what big data is, so what do we do with it?

2‐Oct‐2013 7

8

Types of Analytics

• Descriptive Analytics• These are “traditional” analytics. Most analytics being performed

today fall under descriptive analytics. This is using descriptive statistics to analyze demographics, financial results, etc. The focus here is on describing the past and the present.

• Predictive • Analytics that are intended to forecast the future, based on past

experience. Much growth in this area has happened with new analysis methods and also a focus on receiving results real‐time.

• Prescriptive• An extension of predictive analytics that is focused on selecting a

strategy. This includes simulation and optimization. Given an objective, requirements, and constraints, prescriptive modeling can help provide a solution.

9

Classes of Predictive Analytics

• Anomaly Detection• The search for data items that do not match an expected pattern or

behavior

• Association Rule Learning• Discovery of interesting relationships between different variables in

a dataset, as well as hidden patterns in data

• Clustering• Groups points in a data set that are similar together to understand

different “classes” as well an unexpectedly similar data points

• Classification• Assigns each data point to a mutually exclusive class based on

existing data

• Regression• Forecasting a future value based on historical data

10

Case Study: “Target”ing Expectant Mothers

Business Need

• Target wanted to identify pregnant mothers as early as possible

• Research indicates that consumers often develop shopping habits, but that life events, particularly the arrival of a new baby, are one of the rare opportunities when habits can be “reset”

• By targeting consumers in their second trimester, there is a strong chance of keeping a consumer for years

11


Available Data

• Target has records for each guest that makes a purchase, including• Credit Card Usage• Coupon Usage• Purchase History• Survey Results• Demographics• Estimated Salary

12


Analysis

• Using their baby‐shower registry as a starting point, Target’s statisticians were able to develop buying patterns for pregnant women

• Some of the links are intuitive in retrospect, some are not:• Unscented lotion, vitamin supplements, cotton balls, etc.

• Using their analysis, a “pregnancy perdition” score was developed, including an estimate of due date within a small window

13


Actions

• Now having a list of candidates who are likely pregnant, Target devised a targeted coupon mailing

• In order to ease fears, Target “disguises” intent by including random items, such as a lawnmowers, wineglasses, etc.

14


Results

• From 2002 to 2010, revenues grew from $44 M to $67M

• No published results on how much is attributable to this work, but Target executives have commented publicly that they attribute much of the growth to this program.

From Charles Duhigg’s The Power of Habit

Other Examples

• Hewlett Packard uses predictive analytics with their HR data. By analyzing data such as performance reviews, salaries, raises, job rotations, etc., they are able to develop a “flight risk” score.

• Assurant uses predictive analytics to perform dynamic queuing in their call center. By comparing attributes of the customer and the phone representative, they have been able to triple customer retention rates• Their results indicate that pairing a well‐fit customer and phone rep

is incredibly effective, so much so that it is worth keeping a customer on hold longer to do so

15

Challenges for Insurance Companies

• Insurance companies are older, mature organizations. Often, the underlying data systems are legacy systems that are ill‐equipped for data analysis

• Compared to an industry such as retail, there are infrequent customer touch points• In many cases, the only customer touch points are underwriting and

issue process, and the claims process

• Do insurance companies naturally collect big data? • Often, data at insurance companies is large in breadth, but really

fairly shallow

• Capturing big data and using analytics can required significant front‐end overhead

• Privacy and security concerns are coming to a forefront with regards to data

16

17

How To Become a Data‐Driven Company

• Prepare for change – becoming an analytical company can take years

• Reevaluate business processes throughout the company. Are there opportunities to capture data with a change in business processes?

• Leverage external resources• Hire consultants to bridge the expertise gap• Purchase external data to jumpstart the process

18

How Do Actuaries Fit In?

• Actuaries are uniquely positioned to shape the path for analytics at our respective companies• Possess understanding of the underlying business and

insurance products• Skilled at quantitative analysis• Many actuaries have high technical proficiency

• How will data and analytics fit into insurance organizations?• Is this an actuarial function or IT function?

What’s the Big Deal about Big Data…for Actuaries?

Neil RadenFounder, Hired Brains Research

Twitter: @NeilRaden

Blog: http://hiredbrains.wordpress.comWebsite: http://www.hiredbrains.com

Mail: [email protected]: http://www.linkedin.com/in/neilraden

Neil RadenNeil Raden is the founder and Principal Analyst at Hired Brains Research LLC, , a provider of consulting and implementation services to many Global 2000 companies since 1985, providing research and advisory services focusing on Big Data, Analytics, Decision Management and Business Intelligence. He began his career as a Property & Casualty actuary with AIG in New York before moving into predictive analytics services, software engineering, and systems integration with experience in delivering environments for decision making.

He is the co‐author of the book “Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions,” 2007, Prentice Hall. His blogs appear atInformationWeek, SmartDataCollective and http://hiredbrains.wordpress.com. He is a regular contributor to Forbes, LinkedIn Groups, Focus, Quora and eBizQ and was also an early Wikipedia editor and administrator in areas of technology, health care and mathematics.

EMAIL: [email protected] USERNAME: @neilradenLINKEDIN PROFILE: http://www.linkedin.com/in/neilraden

Copyright 2015 Neil Raden and Hired Brains Research LLC 20

21

1950 1960 1970 1980 1990 2000

Batch Reporting

CICS/OLTP

C/S OLTP

Y2K/ERP

4GL/PC/SS DW/BI

Convergence

Convergence is Here

2010

Operational BI

Composite Apps

BPM

Semantics

DecisionAutomation

History of the Rift Between Operational and Analytical Processing

Copyright 2015 Neil Raden and Hired Brains Research LLC

Big Is RelativeThis Pace Isn’t New, Just Magnitude


Though Volume is interesting, it isn’t what distinguishes Big Data

Moore’s Law


Different Way to Visualize It


No More Managing from Scarcity


Data Warehouse and HadoopData Warehouse Hadoop

Characteristics

Use Cases

Characteristics

• High performance analytics and complex joins

• High concurrency• SQL (ANSI and ACID compliant)

• Advanced workload mgmt.

• High Availability• Data Governance• Emerging Late Binding• Fine Grain Security• One‐stop support

• Fast Data Landing and Refinement

• Processing Flexibility• Emerging SQL/SQL‐like interfaces

• Batch‐oriented processing• Low workload concurrency• Multi‐structured and file based data

• Late Binding• Open Source Community

• Low $/TB• Long‐Term Raw Data Storage

• ETL• Reporting• Deep Analytics


Even Big Data Doesn’t Speak for Itself


• Incomplete• Behaviors under-

represented• Anonymizing

disasters• Single source of

data inadequate• Harmonization

Not a crystal ball

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.

John Tukey


Decisions: A Miracle Happens?


40 years with decision support and BI. Are we making better decisions

Will Data Science Lead Us to Better Decision Processes?

Getting to a culture of decision making requires you to have real, solid wins using analytics to make people care from top to bottom.

What Is Data Science?• Discovering what we don’t know from data• Getting predictive and/or actionable insight • Development of data products that have clear

business value• Providing value to the organization through

sharing and learning• Using techniques like storytelling and

metaphor to explain concepts• Building confidence in decisions

Do You Know This Number?


2.718281828459...

Why is this important

Euler Gave Us the Tools


Contribution Example

Graph Theory Graph & Ontology Databases

Infinitesimal Calculus Everything

Topology Topological Data Analysis

Number Theory Encryption

Nothing we do in Big Data would be possible without Euler

But Euler Got One Thing Wrong


• Tobias Mayer• A contemporary of Euler• Famous for his observations of the libration of the moon

• TONS of observations• Figured out how to group them

Famous quote:Because these observation were derived from nine times as many observations, one can therefore conclude that they are nine times more accurate”

Euler Not a Data Scientist


Euler:“By the combination of two or more equations, the errors of the combinations and the calculations multiply themselves.”

The greatest mathematician of all time pre‐dated the concept of statistical error

One Way to Become a Data Scientist:Mugging injury turns man into math genius

A brutal beating outside a club left college dropout Jason Padgett with brain damage.

But the furniture store worker discovered he could draw diagrams, turning mathematical formulae into stunning works of art

Don’t try this at home


Why Does This Matter?


Because Data Science is not the realm of the most brilliantmathematicians

It’s for people who know how to do it and who have the correct training and tools to do it themselves

Remainder of Presentation Continued in the afternoon Session

3:30 – 4:00


38 3838

Session 2a: How Big Data Changes the IT Relationship (This topic was not covered because of time constraints)

39 3939

Session 2b: Actuarial Standards of Practice

Leonard Mangini, FSA, FRM, FALU, MAAA

ACTUARIAL PROFESSIONALISMIN A DATA SCIENCE CONTEXT

Leonard Mangini, FSA, FRM, FALU, MAAAMangini Actuarial and Risk Advisory LLC

Managing the Big Data Opportunity‐May 6, 2015

Liability Disclaimer, Copyright, Use of Slides

Although I’ve attempted to capture the letter and spirit of the Code of Conduct, ASOPs, andQualification standards faithfully‐ you have a personal professional duty to familiarize yourself withthe original source material and apply professional judgment as to its specific application to yourown work and those working under your direction as you perform covered Actuarial Services.

None of the content in this presentation is intended to be legal or professional advice or an ActuarialOpinion by the Society of Actuaries, Leonard Mangini, or Mangini Actuarial and Risk Advisory LLC.

The nature of your work, and other professional designations you hold, may require you to be boundby additional professional requirements from other professional organizations as well..

Much of the original source material on Professionalism is copyrighted material of the AmericanAcademy of Actuaries. This presentation paraphrases these for educational purposes to capture theintent of the standards, and every attempt has been made to identify and cite the originals.

These slides may NOT be copied, redistributed, or otherwise furnished to any party without the prior written consent of Mangini Actuarial and Risk Advisory LLC, other than as may be required to comply with an audit of the attendee’s annual CPE compliance.

Managing the Big Data Opportunity Mangini Actuarial and Risk Advisory LLC May 6, 2015 41

Overview

• Qualification Standards• Code of Conduct‐ Precepts, Discussion Papers• Actuarial Standards of Practice‐ ASOPs• Reproducible Analyses/Literate Programming• Resources/Guidance• General Discussion


Qualification Standards

Managing the Big Data Opportunity Mangini Actuarial and Risk Advisory LLC May 6, 2015

43

Qualification‐What is an SAO?

• Eff. Jan 1, 2008‐ supersedes prior document’s statutory/regulatory focus

• Broadened definition of Statement of Actuarial Opinion:

“For purposes of the Qualification Standards, a Statement of Actuarial Opinion (SAO) is an opinion expressed by an actuary in the course of performing Actuarial Services and intended by that actuary to be relied upon by the person or organization to which the opinion is addressed. ‘Actuarial Services’ are defined in the Code of Professional Conduct as “Professional services provided to a Principal (client or employer) by an individual acting in the capacity of an actuary. Such services include the rendering of advice, recommendations, findings, or opinions based upon actuarial considerations.”

• Changed the Basic Education and Experience requirements

• Significantly changed Continuing Education to 30 “50 minute hours” per year

Source: Qualification Standards for Issuing Statements of Actuarial Opinion in the United States‐ including Continuing EducationRequirements‐ American Academy of Actuaries‐ available on Academy of Actuaries Website


Qualification Standards‐ Scope

• Precept 2 of Academy’s Code of Conduct requiresmembers to:

“perform Actuarial Services only when qualified to do so on the basis of basic and continuing educationexperience and only when they satisfy applicable qualification standards”

• Common Code of Conduct adopted by ASPPA, CAS, CCA, SOA– Applies regardless of whether or not the actuary is also member of Academy

• Also required to observe qualification standards of Recognized Actuarial Organization, as defined in Code of Conduct, for jurisdiction in which actuary renders Actuarial Services

• NOT intended for actuaries in non‐actuarial roles (management) even if actuarial aspects:

“If it is common for persons holding comparable positions to issue such statements, whether or notthey happen to be actuaries, this is evidence the Qualification Standards are not intended to apply.”



Qualification Standards‐ Basic

• General Qualification‐ Basic Education, Experience, General CE for most SAOs• Specific Qualification‐ additional requirements for particular SAOs by Practice Area• Areas: ERM, Finance, General, Health, Group Life/Managed Care, Investments, Individual Life/Annuity,

Retirement Benefits. Section 4.3 provides guidance for emerging or non‐traditional practice area

• Basic Education and Experience‐ General SAOs– Academy Member, Associate/Fellow CAS/SOA, Fellow CCA, Member/Fellow ASPPA, fully qualified IAA Member– 3 years responsible actuarial experience‐ defined as work requiring knowledge and skill solving actuarial problems– Knowledgeable through exam or documented professional development of Code of Conduct applicable to SAOs

• Basic Education/Experience for SAOs in Specialty Track‐must have ONE of the following:– (1) Highest designation in IAA Member/Specialty Track of SAO, OR– (2) Highest designation IAA Member, min 1 year responsible experience in area under actuary qualified for SAO– (3) Minimum 3 years responsible experience in area of SAO under supervision of actuary qualified for such SAO

Source: Qualification Standards for Issuing Statements of Actuarial Opinion in the United States‐ including Continuing Education Requirements‐ American Academy of Actuaries‐ available on Academy of Actuaries Website


Qualification Standards‐ General CE• General Continuing Education (CE) Requirement

– Complete/Document at least 30 hours each calendar year of relevant CE– At least 3 hours Professionalism, at least 6 hours Organized Activities, at most 3 hours General Business– 30 hours calendar year prior to SAO, shortfall can be earned same year if prior to SAO‐ can’t count to current year– Basic Education (exams) counts towards CE if earned in year prior to SAO– 30 hour requirement includes hours obtained towards CE in Specific Qualification– If practicing in more than one area‐ still 30 hours total‐ actuary good judgment obtaining CE in all areas of practice

• CE is relevant if:– (1) Broadens/deepens actuary’s understanding of one or more aspects of the work actuary does;– (2) Material expands an actuary’s knowledge of practice in related disciplines bear directly on actuary’s work; OR– (3) Facilitates an actuary’s entry into a new area of practice– Actuary’s reasonable, good‐faith determination CE will enhance ability to practice in a desired field– Relevant CE‐ Technical Topics, Business/Consulting/Communication Skills, Professionalism, Review ASOPs/Code

• Organized Activities‐ Involve interaction with actuaries or other professionals of different organizations– Conferences, Seminars, Webcasts, In‐person/online courses, Committee Work directly relevant to SAO, in‐house if outside speaker

• Other Activities‐ reading or writing books/papers/articles, reading statutes/regulation, listening to recorded actuarial meetings/seminars, in‐house meetings, studying for actuarial exams, preparing CE activities

Source: Qualification Standards for Issuing Statements of Actuarial Opinion in the United States‐ including Continuing Education Requirements‐ American Academy of Actuaries‐ available on Academy of Actuaries Website


Qualification Standards‐ Specific

• Specific Standards apply only to certain SAOs with adopted standards– Section 3.1.1.1: NAIC Life and A&H Annual Statement– Section 3.1.1.2: NAIC Property and Casualty Annual Statement – Section 3.1.1.3: NAIC Health Annual Statement

• Life Topics: policy forms, coverages, dividends, reinsurance, investments, asset valuation, asset liability cash flows, stat accounting, valuation of liabilities, and valuation and non‐forfeiture laws

• Experience‐ at least 3 years relevant to SAO, under review by actuary qualified to issue SAO

• Specific Continuing Education– 15 hours/calendar year of CE directly relevant to the topics identified in Section 3.1.1– Minimum 6 of 15 hours involving interactions with outside actuaries other professionals– Specific Qualification CE may be used towards General Qualification, excess carry ahead one year.



48

Actuarial Code of ConductPrecepts/Annotations


49

Code of Conduct‐ General

• Eff. Jan 1, 2001 all US actuarial organizations• Precepts of Professional and Ethical Conduct and Annotations within Precepts• Material Violations subject to Discipline• Jurisdiction of Intended Use of Work Product may have additional standards• Nature of work my have additional standards‐ SEC filings, Investment‐related

work by CFA Charter‐holder, Risk management by GARP Certified FRM etc.• Law takes precedence over Code

• Source: American Academy of Actuaries, Yearbook and Leadership Manual 2009


50

Code of Conduct‐ Definitions

• Actuarial Communication‐ written, electronic, oral re Actuarial Services• Actuarial Services‐ Professional Services provided to a Principal by an

individual acting in capacity of an actuary– Advice, Recommendations, Findings, Opinions

• Actuary‐ individual admitted to actuarial organization that adopted Code• Confidential Information‐ Not in public domain which Actuary becomes

aware of providing Actuarial Services• Law‐ Statutes, Regulations, Judicial Decisions with legal authority• Principal‐ Client or Employer of Actuary• Recognized Actuarial Organization‐ IAA full member

Source: American Academy of Actuaries, Yearbook and Leadership Manual 2009


Code of Conduct‐ Precepts

• Precept 1‐ Honesty and Integrity– Skill and Care in Actuarial Services– Not Violate or Evade Law or damage profession’s reputation– No dishonesty, fraud, misrepresentation

• Precept 2‐ Qualification– Perform Actuarial Service if qualified by Basic or Specific and Continuing Ed– Standards of Recognized Actuarial Organization of jurisdiction– Absence of Standards does not relieve duty

• Precept 3‐ Standards of Practice– Actuarial Services performed by, or under your direction, meet ASOPs– Professional judgment, general practices, regarding applicability– Justify material departures from these standards and practices




• Precept 4‐ Actuarial Communication– Clear and Appropriate to circumstances, audience, satisfies ASOPs– Responsibility for work, availability to explain

• Precept 5‐ Principal and Capacity– Identify the Principal for whom Actuarial Communication intended– Describe capacity in which Actuary serves

• Precept 6‐ Independence– Disclose to Principal material direct/indirect 3rd party compensation

• Precept 7‐ Conflict of Interest– Actuary should NOT perform Actuarial Services with actual/potential conflict unless

ability to act fairly unimpaired, disclosed, Principals agree in writing



53


• Precept 8‐ Control of Work Product– Actuary must take reasonable steps to ensure Actuarial Services are NOT being

used to mislead 3rd parties– Clear and Fair Actuarial Communication– Place limits in distribution and utilization to prevent misuse

• Precept 9‐ Confidentiality– NOT disclose Confidential Information unless Principal Authorizes or required by

to do so by Law

• Precept 10‐ Courtesy and Cooperation– Objective/courteous discussion of differences in assumptions/methods




• Precept 13‐ Knowledge of Material Unresolved Code Violations– Actuary with knowledge of apparent, material, unresolved violation of

code by another Actuary should consider discussing with other Actuary and attempt to resolve apparent violation

– If not attempted or resolved disclose to ABCD except if such disclosure is contrary to Law or would divulge Confidential Information

– Material if important or impacts outcomes, NOT mere form– Not expected to discuss if prohibited by Law or in adversarial role

• Precept 14‐ Prompt, Truthful, and Full Cooperation with ABCD



Academy Discussion Papers

• Precept 13‐Material Code Violations‐ 2013• Relationship to Users of Work Product‐ 2003• Materiality‐ 2006• Selection and Application of Models‐ 2006• Principles/Practices in Developing Areas‐ 2004

Committee on Professional Responsibility– NOT official ASB or ABCD, NOT guidance– Developed to encourage discussion self‐regulation


Precept 13 Discussion Paper‐ 2013

• Discusses “spirit” of Code of Conduct and importance of Self‐Regulation– Integrity/Public Trust, Technical Nature of Work, Maintain Independence– “If you see something, say something”– Actuary has no obligation to investigation‐ that is role of ABCD– Requirement to “Discuss first” intended to clarify differences in interpretation

• Paper discusses– One‐on‐ One Resolution‐ to understand circumstances and judgment– Subjective meaning of “Apparent”, “Unresolved”, and “Material”– Precept 9 on Confidentiality‐ can “bad behavior” be reported without breaching Confidentiality– ASOP 41 Section 4.4‐ where an actuary self‐discloses deviations from other ASOPs– Professional Judgment as defined in ASOP 1 Section 2.9– Reporting Violations without falsely accusing and violating Precepts 1 and Precept 10 yourself– A Precept 13 Process “Flow Chart”

Source: American Academy of Actuaries, The Application of Precept 13 of the Code of Professional Conduct, 2013


57

Relationship to Users of Work Product‐ 2003

• Ensure actuaries assist Principals with “Spirit” not just “Letter” of Regulations– Implied indirect relationship to regulators, policyholders, plan participants, public– Intent to encourage discussion of who else is a “user” of actuarial Work Product– Does NOT advocate mandatory practices beyond Code/ASOPs/Qualification‐ just judgment– If no ASOP‐ Precept 1 still requires honesty, integrity, and competence– Committee feels that Pro Bono work still is covered by Code of Conduct

• Code does NOT render Actuary responsible to 3rd parties beyond Law, notes benefit of Precepts– Precept 5‐ Identifying Principals and the capacity actuary serves‐ informs other potential users– Precepts 6 and 7‐ identifying and disclosing potential conflicts‐ informs all users‐ assent to proceed– Precept 2‐ Consider Qualifications to perform work upfront and decline work if NOT– Precept 3‐ Adequately self‐supervise and supervise those under your direction‐ preserves quality– Precept 4‐ Adequate communication of results and observations‐ limitations on use, distribution, consultation on results– Precept 8‐ Non misleading‐ benefits Others and Principal

• Precepts 13 and 14 are important in self‐policing

Source: American Academy of Actuaries, The Actuary’s Relationship With Users of a Work Product, 2003


58

Materiality Discussion Paper‐ 2006

• Materiality – simply whether something is important– Inclusion‐ should it be considered– Refinement‐ whether a calculation is accurate enough to convey the intended message or conclusion– Context‐ does something need to be disclosed

• No ASOP devoted to materiality itself– Defined within scope of ASOP 5 (Health and Disability Claims) and ASOP 17 (Expert Testimony)– Used in more than a dozen ASOPs‐ left to professional judgment

• Provides suggested approach to thinking about the subject– If omission, understatement, overstatement in work product alters Principal’s decisions/expectations– Materiality is NOT the inherent variability range or uncertainty surrounding an estimate– Consider other standards‐ SEC, FASB, IASB, NAIC and other sources in Appendix– ASOP 41 is clear that actuary is NOT responsible to unintended users but is responsible to assure not misused– If user is Auditor/Examiner then ASOP 21 guides materiality to those specific users– Financial Reporting typically involves disclosing materiality standards

Source: American Academy of Actuaries, Materiality Discussion Paper, 2006


59

Selection/Application of Models‐ 2006• Discussion Paper released in 2006 due to already evolving industry/role of actuaries

• Principals (Clients/Employers) and other users may not understand judgment involved in selecting models, in selecting assumptions used, uncertainties of outputs and that two actuaries following acceptable principles can get different results from same data by using different models/assumptions

• ASOP 38: Using Models Outside the Actuaries’ Area of Expertise: actuarial and non‐actuarial models

• Actuaries should clearly communicate that:– Deterministic models produce “one answer” but use uncertain inputs and model only approximate phenomena– Stochastic models involve uncertainty in occurrence, timing, and severity– Regardless of complexity‐ a model is still a model and not all contingencies are quantifiable

• Encourages Disclosure of Model Limitations:– Model reasonability to intended purpose, reflects actuarial principles and accepted practice– Usability given the existing, available, data– Whether output is consistent with Principal’s purpose– Precepts 1, 3, 8: Advice based on integrity, skill, care; complying with ASOPs, not misleading or evade law

• Encourages Review of Data per ASOP 23 on Data Quality Disclosing Impact on Assumptions/Conclusions

Source: American Academy of Actuaries, Role of Actuary in Selection/Application of Actuarial Models Discussion Paper, 2006


60

Principles/Practices in Developing Areas ‐ 2004• Discussion Paper released in 2004 when actuaries were already entering “new” practice areas

• Assist with complying with Precept 3‐ Actuarial Services must satisfy applicable standards and with Annotation 3‐2 that if standards are unclear or non‐existent actuaries should exercise ”professional judgment, taking into account generally accepted actuarial principles and practices”

• Assist with complying with Precept 2‐ requiring actuaries to be qualified to perform Actuarial Services and Annotation 2‐2 that requirement remains , even in absence of applicable Qualification Standards

• Actuary must decide when working in a developing practice area the extent to which existing ASOP standards apply and the extent to which existing practices and principles generally recognized by the profession can be applied‐ with or without adaptation

• Precept 1 requires actuary to act honestly, with integrity and competence, fulfilling profession’s responsibility to the public, reputation of the profession. Main theme of other Precepts is honesty and integrity in communications/disclosures, confidentiality, conflicts, not misleading, reporting violations

Source: American Academy of Actuaries, The Application of Principles and Practices for Actuaries Working in Developing Areas Discussion Paper, 2004


61

Principles/Practices in Developing Areas ‐ 2004• Determining Qualifications in an Emerging Area‐ Precept 2 and Qualification for PSAOs

– Basic Education‐ sufficiently comprehensive knowledge to determine and apply actuarial concepts– Experience‐ experience relevant to the Prescribed Statement of Actuarial Opinion– Continuing Ed‐ sufficient to maintain current knowledge of applicable standards, principles of practice – Consider experience in one or more recognized practice areas related to the new area– Unlikely that PSAOs required in new area when ASB has not issued ASOPs for that area– ASB is being proactive‐ ASOP 46 and 47 on ERM, ASOP on PBR developed ahead of ORSA/PBR

• Determining Applicability of Existing ASOPs– ASOPs 1‐ Intro , ASOP 23 on Data Quality and ASOP 41 Actuarial Communications apply to all areas– If actuary is not able to determine if other existing ASOP might apply this may indicate that the actuary

lacks the necessary experience to provide Actuarial Services in the new practice area– Can consult with other qualified actuaries practicing in the emerging area– Applicable Laws or regulations can provide guidance



62

Principles/Practices in Developing Areas ‐ 2004

• Principles and Practices Generally Accepted by Profession– If Principles and Practices consistent with ASOPs they can be considered generally accepted– Practices NOT consistent with ASOPs, NOT established by precedent/usage cannot despite outcome– Consider SOA/CAS journals, accepted textbooks, study materials, Academy Practice Notes, other countries– Consider Employer practices, accountants, auditors and other technical professionals– Request advice from ABCD on sources of guidance. ABCD will NOT provide specific guidance on particulars– Principles of other professions involved in new area‐ accountants, attorneys, economists, financial

engineers, mathematicians, risk managers‐ their journals, research, meetings, codes of conduct

• Professional informed judgment applied with integrity and honesty

• Disclose per ASOP 41, that standards don’t exist, show reasoning used, and warn that there is risk due to newness of area that work might not meet standards



63

64 6464

Session 3: Tools and Techniques for Actuaries and Business AnalystsVincent J. Granieri FSA, MAAA

Advisory Services with Integrity

Managing the Big Data Opportunity

Vincent J. Granieri, FSA, MAAA, MBA, EASOA Spring Meeting

May 6, 2015


Programming Tools ‐ R

•Open source software – it’s free•Comprehensive governance• Extensive validation•5,000 packages – including detailed graphics• Large support network•Well‐suited for data applications• Worldwide use by statisticians, investment bankers, commercial bankers, academics

66


Programming Tools ‐ R

• Learning curve• Sketchy documentation•Memory hog

67


Difference in Means

• Theory• Large, independent populations

• Differences in sample means will be normally distributed• If difference in means of exposed group vs. unexposed group is small, then exposure is not causative

• Relative to standard deviation of the difference in means

68


Difference in Frequencies

• Indications• When one of two possible outcomes are measured• There is a supposed causative factor• Cumulative incidence can be measured

•Disadvantages• Does not adjust for exposure

69


Difference in Frequencies Definitions

• Risk of an event = probability that the event occurs• Difference in Frequency: probability event occurs for one group ‐probability event occurs for another group

70

Has Condition Does Not Have Condition Total

Exposed Group A B A + B

Not Exposed Group C D C + D


Difference in Frequencies Definitions

•Risk of Condition for Exposed = [A/(A+B)] •Risk of Condition for Not Exposed = [C/ (C+D)] •Difference in Frequency= [A/(A+B)] ‐ [C/ (C+D)]

71


Relative Risk

• Indications• When one of two possible outcomes are measured• There is a supposed causative factor• Easy to understand

•Disadvantages• Does not adjust for exposure

72


Relative Risk Definitions

• Risk of an event = probability that the event occurs• Relative Risk: probability event occurs for one group / probability event occurs for another group

73





Relative Risk Definitions

•Risk of Condition for Exposed = [A/(A+B)] •Risk of Condition for Not Exposed = [C/ (C+D)] •Relative Risk = [A/(A+B)] / [C/ (C+D)]

74


Odds Ratio

• Indications• When one of two possible outcomes are measured• There is a supposed causative factor

•Advantages• Can compare across many different study designs• Good for rare conditions

•Disadvantage: ignores level

75


Odds Ratio Definitions

• Odds of an event = probability that the event occurs/probability that the event does not occur

• Odds Ratio: odds of event for exposed group /odds of event for not exposed group

76





Odds Ratio Definitions

•Odds of Condition for Exposed = [A/(A+B)] / [B/(A+B)] = A/B

• Odds of Condition for Not Exposed = [C/(C+D)] / [D/(C+D)] = C/D

• Odds Ratio = [A/B] / [C/D] = [AD] / [BC]

77


Odds Ratio – Confidence Intervals

78

• Odds Ratio distribution is positively skewed• Normally distributed on the log scale

• Calculate the CI on the natural log scale, then convert back to get limits on the original scale

• of ln(OR) = SQRT(1/A + 1/B = 1/C + 1/D)• 95% CI = ln(OR) +/‐ 1.96*• Calculate exponential of the upper and lower CI• If 95% CI does not include 1.0, then association is statistically significant


Odds Ratio ‐ Example

79

Condition Both PresentOnly Cond Only Rx Neither Odds Ratio Drug Schizophrenia 26 2,849 1 11,097 101.27 CLOZAPINE

Condition + ‐Exposed Rx

+ 26 1‐ 2,849 11,097

Odds Ratio Linking Rx & Condition


Odds Ratio Example – Confidence Intervals

80

• Calculate the CI on the natural log scale, then convert back to get limits on the original scale

• of ln(OR) = SQRT(1/26 + 1/1 = 1/2,849 + 1/11,097) = 1.0193• 95% CI = ln(OR) +/‐ 1.96* = 4.6178 +/‐ 1.0193

• [3.5985, 5.6371]

• Calculate exponential of the upper and lower CI on the log scale –[36.54, 280.65]

• Since 95% CI does not include 1.0, then Clozapine’s link to Schizophrenia is statistically significant


Scale of Magnitude ‐Modified Cohen Scale

Trivial Small Moderate Large Very Large Near Perfect Perfect

Correlation 0.0 0.1 0.3 0.5 0.7 0.9 1

Diff in Means 0.0 0.2 0.6 1.2 2.0 4.0 infinite

Frequency Difference

0 10 30 50 70 90 100

Relative Risk 1.0 1.2 1.9 3.0 5.7 19 infinite

Odds Ratio 1.0 1.5 3.5 9.0 32 360 infinite

81


Linear Regression

• Models the relationship between a single dependent variable (DV) and one or more explanatory independent variables (IV)

• Form of equation: y = a + b1x1 + b2x2 + … bnxn• bi are the relative impact of the IVs (x’s)

• Can add cross factors if needed; e.g. bn+1x3x6

82


Linear Regression Example• Predicting Risk Scores – Medicare Advantage

• Given: Claims, Rx history and Hierarchical Conditions (HCC)• Find predicted impact on current year’s risk score of Rx, ICD‐9 and HCCs existent in a previous year and calculate the impact above the average for current year using linear regression

• Find predicted impact on current year’s risk score of Rx, ICD‐9 and HCCs discovered this year and calculate the impact above the average for current year using linear regression

• Calculate an expected incremental uplift by applying MA risk scoring rules to the predicted change in risk score

• Rank members by projected uplift above the average

83


Cox Proportional Hazards

• Introduced in 1972, to study the impact of independent variables on survival

• Can handle right‐censored data• Subjects can leave the study at any time and survive to its end

• Can handle multiple underwritings on a single life•Results are independent of the underlying survival curve

• Blessing and a curse

84


Cox Proportional Hazards

•Results are expressed as the log(hazard)• Hazard is relative to the base curve; e.g. 2 implies double the risk

• Consistent with Gompertz•Assigns a portion of the risk to each variable through a hazard ratio

• elog(hazard) = hazard

85


Cox PH Example•Revising an Underwriting Debit/Credit Model

• Given: 200,000 underwriting events on 80,000 lives• 15 year period• Age, gender, conditions identified, dates of death

• Choose reference population: male, non‐smokers• Define variables and specify continuous/binary

• BMI (continuous) vs. obese (binary)• Define standards/process• Run models

86

87 8787

Session 4: Big Data Case Study #1

Vincent J. Granieri FSA, MAAA


Managing the Big Data Opportunity Case Study: Predictive Modeling Risk

Scores in Medicare Advantage


May 6, 2015


Predictive Modeling With Medicare Advantage Data

• Primer on Medicare Advantage• Issues• Methodology• Objective: Create a targeted member list for intervention by predicting risk scores under Medicare Advantage

• Tools: Odds Ratios, Persistence, Multivariate Linear Regression

89


Primer on Medicare Advantage

• CMS reimburses payors via capitated rates• Base rate based on age• Additional reimbursement based on ‘risk’ (risk score)• Adjustments for comorbidity, claims concentration

• Reimbursement is determined in the year of service• Applied to returning members in the following year (year of payment)

• ICD‐9 codes => condition codes => hierarchical condition codes (HCC)

90



• Every year, the slate is wiped clean• Previous years’ HCCs must be re‐validated

• YOS window is precise and ends on 1/31/y+1• Adjustments are possible thereafter, but no new items

• HCCs are validated by provider events• Doctor visits• CMS also recognizes HCCs generated by home health care assessments (interventions)

91



• Each April, CMS issues final rules for the current YOS• Changes in methods, factors, procedures

• Payors then want to start validation process with interventions

• Here’s where we come in

• Which members are most likely to experience an increase in risk score?

• Challenge: create a prioritized targeted member list

92


Inputs

•Payer historical data for 5‐6 YOS/YOP • Demographics• Spend• Rx claims• ICD‐9 codes

• Treatment codes => Hierarchical Condition Codes

•MA risk scoring rules

93


Issues

•Data• Availability• Consistency• Validity

• Efficiency of intervention•Defining a successful intervention

94


Issues – Data Availability

• Need enough data to do analysis• At least two YOS/YOP

• Can do lag studies – DV is this year’s risk score, IV is last year’s score• Can do current studies – DV is other HCCs which may be also present when a certain HCC (IV) is present

• Possible to forecast a year from a partial year of data• Not ideal but do‐able

95


Issues – Data Consistency

• Risk scoring, age, concentration factors change over time• Convert all risk scores to the 2014 model• Ignore coding concentration, age adjustment factors until the very end

• Ensure that the relationships we discover are due to the conditions, not the rules

• Coding inconsistencies• ‘Acute’ conditions persist• ‘Chronic’ conditions don’t• Created persistency matrix for each condition by year

• Members with condition / members had the condition originally and are still members

96


Sample Data

HCC Description 0 yr

Members_Left 1 Yr 1 yr 1_yr_%



1HIV/AIDS 20 16 16 100.00% 15 14 93.33% 13 13 100.00%

17Diabetes with Acute Complications 65 47 16 34.04% 36 12 33.33% 28 6 21.43%

18Diabetes with Chronic Complications 5298 4673 3773 80.74% 3870 3066 79.22% 3131 2480 79.21%

19Diabetes without Complication 14343 12546 11915 94.97% 9971 9375 94.02% 8279 7742 93.51%

97


Issues – Data Validity

• Cure for quadriplegia?• ICD‐9 code that maps to quadriplegia is very close to another code

• Created a rule that required a code to show up at least twice or it wasn’t counted

• High % of schizophrenics in one population• HCC codes had changed, but descriptors were not updated in the data• Old schizophrenia HCC was the new substance abuse HCC

98


Issues – Intervention Efficiency/Success

• Intervention cannot find certain HCCs• Intervention sometimes misses HCCs it should find• Intervention finds other HCCs as well• Created two indices

• Maximum score with intervention – assumes intervention is perfect• Likely score with intervention – assumes intervention is not• Both the above include other HCCs at current rates of discovery

• Interventions that discover HCCs first are automatically successful• Interventions that validate claims data or initiate claims are good too

99


Methodology•Persistence analytics

• For each HCC ever encountered in the data:• Look back 3 years and give 100% iff HCC was there all 3 years• Otherwise, use the persistence matrix results

• This is the maximum potential from persistence• Adjust for the intervention success rate

100


Methodology• Odds ratio/relative risk analytics

• For each Rx:• Find the odds ratio/relative risk that the Rx is related to an HCC

• Thresholds: odds ratio/relative risk is > 9.0/5.7• For each member with a new Rx claim:

• Apply the implied probability (i.e. OR of 10 is probability 10/11)• For each ICD‐9:

• Find the odds ratio/relative risk that the Rx is related to an HCC• Thresholds: odds ratio/relative risk is > 9.0/5.7

• For each member with an ICD‐9 that is new/documented at least twice:• Apply the implied probability

101


Methodology•Regression Analytics

• Using clinical information, set up linear regression tests:• Onset: predict new HCCs from other HCCs, ICD‐9 groups• Progression: predict higher level HCCs from lower level HCCs, other HCCs, ICD‐9 groups

• Run regressions on YOS 2013 to obtain factors• Adjust regressions based on results

• Test regression results by running final regressions on YOS 2012 or 2014

• Looking to validate results

102


Regression Results for Progression to Metastatic Cancer

Coefficients:Estimate Std. Error t value Pr(>|t|)

(Intercept) ‐0.00025 0.000643 ‐0.383 0.70199c_h_11 0.044105 0.004703 9.378 < 2e‐16 ***

p1_ig_162 0.049351 0.015708 3.142 0.00168 **c_ig_157 ‐0.1987 0.034332 ‐5.788 7.33E‐09 ***c_ig_197 0.824843 0.014244 57.908 < 2e‐16 ***p1_ig_197 0.14173 0.019974 7.096 1.36E‐12 ***c_h_9 0.197338 0.009526 20.715 < 2e‐16 ***p1_h_9 ‐0.15 0.013721 ‐10.932 < 2e‐16 ***p2_h_9 0.043152 0.013508 3.195 0.0014 **c_h_10 0.061559 0.008093 7.606 3.04E‐14 ***p1_h_10 0.062451 0.008871 7.04 2.04E‐12 ***p2_h_10 ‐0.03139 0.011175 ‐2.809 0.00499 **c_h_12 0.027839 0.003749 7.426 1.20E‐13 ***p1_h_12 0.016674 0.004025 4.143 3.46E‐05 ***p2_h_12 0.013054 0.004549 2.87 0.00412 **

103


Methodology – Putting it all Together

•Persistence + Odds Ratio: Rx + Odds Ratio: ICD‐9 + Regression (Onset and Progression) = Raw potential score

•Adjust based on data sources• If forecasting 2015 with 2014 data, apply persistency• If forecasting 2015 with 2015 data, do not

•Adjust for age factors, concentration, interactions•Adjust for intervention efficiency

104


Predictive Model TML Score Sheet

Patient_Id Service_Year Previous_RAF RAF_Potential RAF_Potential_Perfect_I RAF_Potential_Curr_I RAF_Potential_No_Inter HCC HCC_Potential Interactions_Potential Age_Factors Odds Regression PersistenceConcentration

1599 2015 1.192 1.703 1.703 1.686 1.59 0 1.013 0.007 0.683 0 0.038 1.008 0.954

5992 2015 1.511 1.679 1.66 1.639 1.408 0.154 0.969 0.013 0.543 0.236 0.158 0.669 0.954

39978 2015 2.477 4.438 4.438 3.522 1.853 1.193 2.446 0.928 0.543 2.434 0.571 0 0.954

47123 2015 3.172 2.032 1.948 1.948 1.859 0 1.13 0.054 0.848 0.138 0.239 0.858 0.954

51297 2015 2.994 5.187 5.187 4.411 2.476 2.109 3.522 0.203 0.326 2.97 0.674 0.479 0.954

51905 2015 0.502 0.702 0.457 0.457 0.447 0 0.354 0 0.348 0.255 0.099 0.154 0.954

52359 2015 1.122 0.831 0.831 0.831 0.831 0.317 0.383 0.011 0.437 0 0.22 0.269 0.954

68204 2015 3.578 2.717 2.228 2.008 1.479 0.616 1.675 0.005 0.539 1.31 0.155 0.285 0.954

68956 2015 1.908 2.364 2.364 2.198 1.614 0 1.501 0.426 0.437 0.318 0.376 1.016 0.954

100031 2015 1.616 1.563 1.563 1.478 0.815 0 0.956 0.259 0.348 0 0.155 0.956 0.954

105


Thank You!

For complete copy of this presentation, email

[email protected]‐272‐7118

106

107 107107

Session 5: Panel Discussion

108 108108

Session 6: Big Data Case Study #2

Vincent J. Granieri FSA, MAAA


Managing the Big Data Opportunity Case Study: Predictive Modeling Risk

Assessment (U/W) Models


May 6, 2015


Predictive Modeling With Life Underwriting Data

• Primer on Life Settlements• Analytical Approach• Method• Objective: Develop an underwriting system of debits and credits from life settlement data

• Tools: Prevalence, odds ratios, Kaplan‐Meier survival curves, Cox Proportional Hazards Regression Model

110


Primer on Life Settlements

• Life Settlements arose in the mid ‘90s when HIV patients sold their policies to individual investors

• Trading certain smaller payment for greater later payment• Needed $$ for treatment

• Advent of protease inhibitors wrecked that market and the focus shifted to impaired seniors

• Increasing longevity and opportunistic healthy insureds nearly destroyed the market again

111


Primer on Life Settlements

• Independent underwriters have been involved from nearly the beginning

• No such thing as a decline – a life expectancy is generated for every case

• Wide range of risk from preferred to super sub‐standard• Risk is comparable to immediate annuities or pensions

112


College‐Educated Seniors

General SeniorPopulation

Contemplated Settling but Presumed Not Settled

Insured Seniors

Reported Settled


Inputs – Life Settlement Underwriting

•80,000+ unique lives• Underwritten 2+ times on average from 2000‐2012

•10,000+ deaths•200+ variables/conditions including ICD‐9 codes• Follow up through 2012•Desired outcome – an underwriting model to predict life expectancy

114


Analytical Approach: Broad to Narrow

•Broad first cuts – goal: familiarity• Prevalence• Odds ratios

• Transitional• Kaplan‐Meier survival curves – goal: base curves

•Detail• Cox proportional hazards model – goal: debits/credits

115


Kaplan‐Meier vs. Cox Proportional Hazards

• K‐M is purely descriptive – i.e. no regression analysis• K‐M estimates survival curves – tpx• Cox PH model – overwhelming choice for survival regression models

• Makes no assumption regarding underlying survival curve• Force of mortality grows exponentially with age• Apportions hazard among competing independent variables

• Both handle right‐censored data

116


Method

•Clean data and prepare R data base•Calculate prevalence

117


Prevalence

Description Male Total Female Total Male Count Female Count Male % Female %HIV/AIDS 9,096 11,993 421 73 5% 1%Septicemia/Shock 249 235 3% 2%Metastatic Cancer and Acute Leukemia 1,032 981 11% 8%Diabetes with Renal or Peripheral Circulatory Manifestation 3,257 4,032 36% 34%Protein‐Calorie Malnutrition 882 1,353 10% 11%End‐Stage Liver Disease 471 394 5% 3%Intestinal Obstruction/Perforation 337 331 4% 3%Bone/Joint/Muscle Infections/Necrosis 209 253 2% 2%Rheumatoid Arthritis and Inflammatory Connective Tissue Disease 463 1,031 5% 9%Severe Hematological Disorders 289 661 3% 6%

HCC Prevalence in a Health Insurance Population

118


Method

•Clean data and prepare R data base•Calculate prevalence•Calculate odds ratios

119


Odds Ratio

120

HCCGroup Both Present Only HCC Only NDC Neither Odds Ratio Drug Schizophrenia 26 2,849 1 11,097 101.27 CLOZAPINE

outcome + ‐exposure

+ 26 1 ‐ 2,849 11,097

Odds Ratio Linking RX & HCC


Method

•Clean data and prepare R data base•Calculate prevalence•Calculate odds ratios•Run Kaplan‐Meier curves

121


College‐Educated Seniors

General SeniorPopulation

Contemplated Settling but Presumed Not Settled

Insured Seniors

Reported Settled


123


124


Method

•Clean data and prepare R data base•Calculate prevalence•Calculate odds ratios•Run Kaplan‐Meier curves•Determine first cut independent variables

• Continuous & factor•Calculate hazard ratios using Cox

125


Method – cont’d

• Eliminate variables with p > 0.2•Calculate hazard ratios on this smaller set• Eliminate variables with p > 0.1•Calculate hazard ratios on the still smaller set• Eliminate variables with p > 0.05•Analyze and set debit/credit model

126


Cox Model Results for Baseline Variables

127

All Variables Variables (p<=0.2) Variables (p<=0.1) Variables (p<=0.05)

Variable Frequency ln(Hazard) Hazard Lower 95% Upper 95% p Value ln(Hazard) Hazard Lower 95% Upper 95% p Value ln(Hazard) HazardLower 95%

Upper 95% p Value ln(Hazard) Hazard

Lower 95%

Upper 95% p Value

Age 82,164 0.0776 1.0807 1.0753 1.0862 0.00 0.0775 1.0806 1.0753 1.0859 0.00 0.0775 1.0806 1.0753 1.0858 0.00 0.0781 1.0812 1.0760 1.0864 0.00

BMI 60,453 0.0019 1.0019 1.0014 1.0024 0.00 0.0019 1.0019 1.0014 1.0024 0.00 0.0019 1.0019 1.0014 1.0024 0.00 0.0019 1.0019 1.0014 1.0024 0.00

Cancer Recurrent 1,228 0.3362 1.3997 1.1952 1.6391 0.00 0.3526 1.4227 1.2212 1.6576 0.00 0.3515 1.4213 1.2209 1.6546 0.00 0.3683 1.4452 1.2427 1.6808 0.00

Female 31,842 (0.3602) 0.6976 0.6440 0.7556 0.00 (0.3679) 0.6922 0.6428 0.7453 0.00 (0.3582) 0.6990 0.6524 0.7489 0.00 (0.3667) 0.6930 0.6469 0.7423 0.00

Active Lifestyle 15,773 (0.1312) 0.8770 0.8085 0.9513 0.00 (0.1361) 0.8727 0.8049 0.9463 0.00 (0.1466) 0.8636 0.7967 0.9362 0.00 (0.1495) 0.8612 0.7945 0.9335 0.00

Sedentary Lifestyle 1,069 0.1912 1.2107 1.0380 1.4122 0.01 0.2182 1.2438 1.0710 1.4446 0.00 0.2266 1.2543 1.0820 1.4540 0.00 0.2326 1.2619 1.0887 1.4628 0.00

Unknown Lifestyle 26,351 0.1359 1.1455 1.0646 1.2326 0.00 0.1349 1.1444 1.0647 1.2301 0.00 0.1330 1.1422 1.0627 1.2276 0.00 0.1407 1.1511 1.0713 1.2369 0.00

Family Longevity 15,554 (0.0871) 0.9166 0.8547 0.9829 0.01 (0.0927) 0.9114 0.8514 0.9757 0.01 (0.0921) 0.9121 0.8521 0.9762 0.01 (0.1050) 0.9003 0.8416 0.9632 0.00

Family Super Longevity 10,227 (0.2581) 0.7726 0.7073 0.8438 0.00 (0.2568) 0.7736 0.7096 0.8432 0.00 (0.2560) 0.7742 0.7103 0.8438 0.00 (0.2632) 0.7686 0.7055 0.8373 0.00

Current Tobacco User 3,788 0.6112 1.8426 1.6498 2.0580 0.00 0.6187 1.8566 1.6649 2.0703 0.00 0.6209 1.8607 1.6692 2.0741 0.00 0.6242 1.8668 1.6750 2.0807 0.00

Former Tobacco User 29,863 0.1712 1.1867 1.1192 1.2583 0.00 0.1700 1.1853 1.1184 1.2562 0.00 0.1734 1.1893 1.1223 1.2603 0.00 0.1764 1.1929 1.1259 1.2639 0.00

Rare Tobacco User 135 (0.2841) 0.7527 0.2720 2.0833 0.58 (0.2556) 0.7744 0.2886 2.0781 0.61 (0.2635) 0.7684 0.2865 2.0611 0.60 (0.2232) 0.7999 0.2991 2.1395 0.66

Uses Tobacco Replacement 232 0.5298 1.6986 1.1313 2.5506 0.01 0.5283 1.6961 1.1306 2.5445 0.01 0.5125 1.6695 1.1128 2.5047 0.01 0.4922 1.6359 1.0904 2.4544 0.02

Tobacco Use Unknown 13,584 0.1240 1.1320 1.0219 1.2540 0.02 0.1243 1.1324 1.0230 1.2535 0.02 0.1245 1.1326 1.0232 1.2538 0.02 0.1231 1.1310 1.0219 1.2517 0.02


Cox Model Results for Cardiac Risk Factors

128


Variable Frequency ln(Hazard) Hazard Lower 95% Upper 95% p Value ln(Hazard) Hazard Lower 95%Upper 95% p Value ln(Hazard) Hazard Lower 95%Upper 95% p Value ln(Hazard) Hazard Lower 95% Upper 95% p Value

Hypertension 55,899 0.0880 1.0920 1.0254 1.1629 0.01 0.0911 1.0954 1.0291 1.1659 0.00 0.0906 1.0948 1.0286 1.1653 0.00 0.0891 1.0932 1.0272 1.1634 0.01

Hyperlipidemia 64,302 ‐0.2798 0.7559 0.7107 0.8040 ‐ ‐0.2812 0.7549 0.7101 0.8024 0.00 ‐0.2857 0.7515 0.7070 0.7988 0.00 ‐0.2867 0.7508 0.7064 0.7979 0.00

Glucose Intolerance 7,764 0.0657 1.0679 0.9530 1.1966 0.26

Diabetes 14,494 0.2380 1.2687 1.1863 1.3569 0.00 0.2279 1.2559 1.1765 1.3407 0.00 0.2391 1.2701 1.1923 1.3529 0.00 0.2336 1.2632 1.1871 1.3441 0.00

Family History of Premature Cardiovascular Disease 6,377 ‐0.1063 0.8992 0.7436 1.0873 0.27

Elevated homocysteine 1,961 ‐0.1566 0.8551 0.7041 1.0383 0.11 ‐0.1497 0.8610 0.7119 1.0413 0.12


Issues

• Should we age/gender match?•What is the underlying mortality table? •Correlation of ‘independent’ variables•Granularity

• Drawing the line• Sooner or later, the cells are too small to be useful

129


Age/Gender Matching – Assisted Living vs. Life Settlement

130


Cox Results Can Define Age/Gender/Tobacco Relationships

131


Variable Frequency ln(Hazard) HazardLower 95%


Lower 95%


Lower 95%


Lower 95%

Upper 95% p Value

Age 82,164 0.0776 1.0807 1.0753 1.0862 0.00 0.0775 1.0806 1.0753 1.0859 0.00 0.0775 1.0806 1.0753 1.0858 0.00 0.0781 1.0812 1.0760 1.0864 0.00

Female 31,842 (0.3602) 0.6976 0.6440 0.7556 0.00 (0.3679) 0.6922 0.6428 0.7453 0.00 (0.3582) 0.6990 0.6524 0.7489 0.00 (0.3667) 0.6930 0.6469 0.7423 0.00

Current Tobacco User 3,788 0.6112 1.8426 1.6498 2.0580 0.00 0.6187 1.8566 1.6649 2.0703 0.00 0.6209 1.8607 1.6692 2.0741 0.00 0.6242 1.8668 1.6750 2.0807 0.00

Former Tobacco User 29,863 0.1712 1.1867 1.1192 1.2583 0.00 0.1700 1.1853 1.1184 1.2562 0.00 0.1734 1.1893 1.1223 1.2603 0.00 0.1764 1.1929 1.1259 1.2639 0.00

Rare Tobacco User 135 (0.2841) 0.7527 0.2720 2.0833 0.58 (0.2556) 0.7744 0.2886 2.0781 0.61 (0.2635) 0.7684 0.2865 2.0611 0.60 (0.2232) 0.7999 0.2991 2.1395 0.66

Uses Tobacco Replacement 232 0.5298 1.6986 1.1313 2.5506 0.01 0.5283 1.6961 1.1306 2.5445 0.01 0.5125 1.6695 1.1128 2.5047 0.01 0.4922 1.6359 1.0904 2.4544 0.02

Tobacco Use Unknown 13,584 0.1240 1.1320 1.0219 1.2540 0.02 0.1243 1.1324 1.0230 1.2535 0.02 0.1245 1.1326 1.0232 1.2538 0.02 0.1231 1.1310 1.0219 1.2517 0.02


Spurious Results Can Arise if Variables Aren’t Independent

132

All Variables

Variable Frequency ln(Hazard) HazardLower 95%

Upper 95%

Significant stenosis of the Left Main 1,689 0.0830 1.0866 0.9363 1.2610

Significant stenosis of the proximal Left Anterior Descending Coronary Artery 7,728 (0.0231) 0.9772 0.8771 1.0887

Significant stenosis of the proximal Circumflex 3,805 0.0566 1.0582 0.9396 1.1918

Significant stenosis of the proximal Right Coronary Artery 5,670 0.0925 1.0969 0.9815 1.2258

Stenosis affecting one or more mid‐vessel segments or secondary branches 10,288 (0.1534) 0.8578 0.7747 0.9498


Limiters of Predictive Modeling

• Inconsistent underwriting• Better to be consistent than right

•Poorly defined variables• Unknown is not the same as absent

• Tobacco use: yes, no, rarely, smokeless, quit, unknown• Interactions/comorbidity should be tested

• Need 4 variables to test interaction of 2 conditions

•Related conditions• Glucose intolerance/diabetes

133


Thank You!

For complete copy of this presentation, email

[email protected]‐272‐7118

134

135 135135

Session 7: Actuarial Professional ConsiderationsLeonard Mangini, FSA, FRM, FALU, MAAA

Actuarial Standards of Practice


136

ASOPS‐ Starting Off a Project

• When starting any client engagement as a consultant, or new project for an employer, one should check to see which ASOPs already apply and whether any proposed/exposed might kick‐in

• Can look to other jurisdictions‐ e.g. Canada for principles

• Academy website dynamically organizes ASOPs by type of engagement to assist in this process

• Applicability Guideline Spreadsheet– Provides guidance‐ but ultimately own judgment– Has “tab” for Life, Casualty, Health, Pension– Organized by type of project


137

Key ASOPs Related to Data/Modeling

• Depends on context of Actuarial Service and Area or Practice– Appraisal, ERM, Risk Classify, Product Development, Reporting, etc.

• ASOP 1‐ Intro Standard• ASOP 41‐ Actuarial Communication

• ASOP 23‐ Data Quality• ASOP 25‐ Credibility• ASOP 38‐Use of Models Outside Area of Expertise‐ P&C• Proposed ASOP on Modeling• Proposed ASOP on Principles Based Reserves


General ASOPs

• ASOP 1‐ Intro Standard of Practice• ASOP 41‐ Actuarial Communication


139

ASOP 1‐ Intro Standard

• Adopted Mar 2013, eff. Jun 1, 2013‐ revision to prior 2004 and 2008 versions

• ASB defines appropriate level of practice‐ not just codifying current practice

• Reaffirms ASB as the final authority on ASOPs

• “Deviations” communication and disclosure language removed from all ASOPsand moved into ASOP 41 Actuarial Communications

Source: American Academy of Actuaries Website


140

ASOP 1‐ Intro Standard

• ASOP 1 Defines “Must” and “Should”– “Must”‐ ASB judges no reasonable alternative to practice‐ deviation is failure– “Should” – normally appropriate, professional judgment may indicate a

situational alternative, however the deviation must be disclosed and explained per the standard provided in ASOP 41 Actuarial Communications

– “May”‐ indicates the practice in the standard is reasonable and appropriate in many (but not all)) circumstances and actuary should apply judgment

• Certain terms are defined in ASOP 1 and these carry over to all other ASOPs– Actuarial Services, Soundness, Deviation, Known, Materiality– Practical/Practicable, Principal, Professional Judgment, Reasonable, Reliance,

Significance/Significant• Terms defined within other ASOPs are limited to that ASOP• Undefined terms should have their common usage meaning



141

ASOP 1‐ Intro Standard• Defines purpose of ASOPs

– What should be considered, done, documented, disclosed when rendering actuarial services– Standard for appropriate actuarial practice– NOT meant to imply that mere deviation is malpractice– Intended for Precept 2 Qualified Actuaries whose education and experience permits them to

understand the standards and how they would apply situationally• Principles based‐ common situations• Affirms that Actuarial Guidelines carry same weight as ASOPs• Practice Notes, textbooks, articles, sessions at actuarial meetings‐ DO NOT• Breach of ASOP is Breach of Code of Conduct‐ ABCD consequences• Deviations are not Breaches if explained as per ASOP 41• ‘Strained Readings” of ASOPs are not appropriate• Seek ABCD guidance if believe multiple ASOPs conflict



142

ASOP 41‐ Actuarial Communication

• Adopted Dec 2010, eff. May 1, 2011‐ revision to prior 2002 version• Applies to all practice areas, all actuarial communication (e.g. opinions and findings) but NOT

general communication like advertising or invoices

• If other Guidance is additional/inconsistent then THAT one supersedes, but ASOP 41 applies if the other guidance is NOT inconsistent

• Law or Regulation may specify form and content‐ should still comply with ASOP 41 as much as possible while still complying with the law

• ASOP 41, like ASOP1, defines many terms broadly:– Actuarial Communication, Actuarial Document, Actuarial Finding, Actuarial Report, Actuarial Services– Reiterates that Oral Communication is still a communication‐ can’t “escape” by not putting in writing– Intended User: Any person who actuary identifies as able to rely on actuarial findings in communication– Other User: Any recipient of communication who is not an intended user



143


• Content Requirements for Actuarial Communications:– Scope of Work, Methods, Procedures, Assumptions, Data, Other Information– Form, Content, Clarity – Must be appropriate to the circumstances, written

in clear language, and understandable to the intended users– Timeliness

• Identification of Responsible Actuary/Actuaries– Availability to Provide Supplementary Info/Explanations

• Actuarial Report‐ Required if actuary intends findings to be relied upon by ANY intended user. Can be several documents/formats as long as the actuary identifies which documents together comprise the report.



144


• Should state findings, methods, procedures, assumptions, data with sufficient clarity that another qualified actuary in same practice area could make objective appraisal of the reasonableness of work in the report

• Section 3.4 and Section 4 have long list of required disclosures:– Responsible Actuary, Acknowledgment of Qualification, Scope, Principal, Intended Users,

Conflicts, Limitations in Use– Date of Findings, Data, Contingencies and Limitations of Findings, Reliance where disclaim

responsibility– Methods Prescribed by law versus Judgment, Responsibility for Assumptions and Methods

• Section 4.4 Deviations from ASOPs‐ if not required by law‐ nature, rationale, and effect of such deviations



145

Data/Modeling Specific ASOPs

• ASOP 23‐ Data Quality• ASOP 25‐ Credibility• ASOP 38‐ Use of Models outside Expertise• Proposed ASOP on Modeling• Proposed ASOP on Principles Based Reserves


146

ASOP 23‐ Data Quality

• Revised for Deviation Language eff. May 1, 2011‐ prior 2004 and 1993 versions– If ASOP establishes requirements in addition to those required by Law‐ actuary should

satisfy BOTH standard and Law

• Purpose is to give guidance to Actuary in:– Selecting Data underlying Work Product, Relying on Data supplied by others– Reviewing Data, Using Data, Making Appropriate Disclosures on Data Quality

• Scope is providing actuarial services in all practice areas, other ASOPs may contain additional considerations

– Does NOT require Actuary to audit data– If depart from standard to comply with law OR other reason Section 4 applies



147

ASOP 23‐ Data Quality‐ Definitions

• Appropriate Data‐ suitable for the intended purpose, relevant to process• Audit‐ formal systematic accuracy test using techniques employed by auditors• Comprehensive Data‐ inventory/sampling methods sufficient for analysis• Data‐ numerical, census, or classification information, NOT general/qualitative• Assumptions are NOT data‐ data are used in developing actuarial assumptions• Data Element‐ Item of Information such as Date of Birth or Risk Classification• Practical‐ Realistic in approach during time of assignment, given purpose and

nature, and the physical constraints on the project‐ time and cost• Review‐ informal exam of obvious characteristics of selected data for

reasonableness and consistency with purpose



148

ASOP 23‐ Data Quality‐ Selection

• Actuary should consider what data to use given scope and intended use:– Appropriateness for purpose, sufficiently current for date of analysis– Reasonableness and Comprehensiveness‐ internal/external consistency,

Sampling methods, if used to collect data– Known, material limitations– Cost and feasibility of obtaining alternate data in reasonable time– Benefit obtained from alternate data– Balanced against availability, time, cost to compile alternate data



149

ASOP 23‐ Data Quality‐ Accuracy

• Accuracy and Comprehensiveness of others’ data is the responsibility of supplier of data– Disclose reliance on the source in Documentation and Communication– May rely unless material errors or unreliability become apparent during work

• Should review for reasonableness and consistency unless professional judgment not necessary• Should take into account extent of checking, verification, or auditing already performed• Should take into account purpose/nature of the work being done• Should consider definitions, materially questionable values, inconsistencies, prior periods

• Actuary NOT required to detect if data from others falsified, intentionally misleading• Actuary NOT required to audit data



150

ASOP 23‐ Data Quality‐ Bad Data

• Actuary should make professional judgment if data of sufficient quality for analysis or require enhancement

• Can adjust data/assumptions to perform analysis, may choose to complete work even if material bias‐ but should disclose nature and magnitude of bias

• If data are so inadequate to prevent actuary form performing an analysis– Get alternate data, OR– Decline to complete assignment



151

ASOP 23‐ Data Quality‐ Documentation

• ASOP 41 guides documentation in general

• ASOP 23 requires additional documentation regarding:– process to evaluate data– material defects– adjustments/modifications made to data– support for other disclosures that will be communicated



152

ASOP 23‐ Data Quality‐ Communication

• When issuing communications under ASOP 23 should disclose– Sources of data, whether actuary reviewed– Extent of reliance on data and information from others– Any material judgmental adjustments to data by actuary or others – Any limitations on the use of work product due to uncertainty in data quality– Any unresolved concerns about data that could have material effect on work product– Existence of highly uncertain results or material bias due to data quality– Magnitude of uncertainty or bias if reasonably able to estimate

• Should also disclose ASOP 41 Section 4.2, 4.3, 4.4 issues– Material assumptions or methods prescribed by law rather than data– Reliance on material assumptions or methods set by others

• Material deviations from ASOP 23

Source: American Academy of Actuaries WebsiteManaging the Big Data Opportunity Mangini Actuarial and Risk Advisory LLC May

6, 2015153

ASOP 25‐ Credibility‐ Definitions

• Eff. May 1, 2014, extends to all practice areas‐ prior just Health and P&C

• Credibility‐ statistical predictive value actuary attaches to a particular set of data• Credibility Procedure‐process that evaluates subject experience for potential use in setting

assumptions without reference to other data OR identification of relevant experience and selection of implementation method for blending relevant experience with subject experience

• Full Credibility‐ subject experience assigned full predictive value at selected confidence interval• Relevant Experience‐ data other than subject experience that in actuary’s judgment predictive of

parameter under study. Relevant Experience may include subject experience as a subset.• Risk Characteristics‐measurable/observable factors used to assign a risk class• Risk Classification System‐ system used to assign risks to groups based on expected cost/benefits• Subject Experience‐ specific set of data drawn from experience for predicting studied parameter



154

ASOP 25‐ Credibility‐ Scope/Applicability

• When required by law, statute or regulation to evaluate credibility• When actuary chooses to evaluate credibility of subject experience• When actuary communicates that credibility has been evaluated• When actuary has blended subject experience with other experiences• When actuary represents data as statistically/mathematically credible• If actuary sees conflict with ASOP 35 on Pensions it “trumps” ASOP 25• Section 4‐ statute/regulation requires deviation OR deviation appropriate

• Credibility procedures covered by standard have two purposes:– Evaluate subject experience for potential use in setting assumptions without other data– Improving estimates of parameters under study

• May be used for pricing, ratemaking, experience rating, reserving



155

ASOP 25‐ Credibility‐ Procedures

• Selecting Procedure:– Should use appropriate procedure when determine full credibility or when blending– Procedure selected OR developed may vary by practice area and application– May require additional review to satisfy applicable law– Should consider whether procedure expected to produce reasonable results,

appropriate for intended use and purpose, AND practical to implement cost/benefit– Should appropriately consider characteristics of both subject and relevant experience– Should consider statistical predictive value of more recent experience versus past data



156

ASOP 25‐ Credibility‐ Experience

• Selecting Relevant Experience– Should exercise professional judgment/care selecting/using relevant experience– Should have characteristics similar to the subject experience‐ demographics, coverage,

frequency, severity, or other determinable risk characteristics that the actuary expects to be similar to the subject experience

– If proposed relevant experience does not meet and cannot be adjusted to meet such criteria, it should not be used

• Should consider extent subject in relevant experience– Ifmaterial part of relevant use professional judgment in whether/how use relevant

• No relevant experience‐ should use judgment with subject experience• May be acceptable to assign full, partial, zero without a rigorous model• Should consider homogeneity of subject and relevant experience



157

ASOP 25‐ Credibility‐ Disclosure

• Actuary should disclose– Credibility procedures used– Material changes from prior credibility procedures

• Actuary should also include the following, as applicable– ASOP 41 Section 4.2, if material assumption/method prescribed by law– ASOP 41 Section 4.3, reliance on other sources and if disclaiming responsibility

for material assumptions or methods that were selected by others– ASOP 41 Section 4.4, if, in the actuary’s professional judgment, the actuary has

otherwise deviated materially from ASOP 25



158

ASOP 38‐ Using Models Outside Expertise (P&C)

• Eff. May 1, 2011 for deviation‐ originally Dec 15, 2000

• P&C actuaries who use models that incorporate specialized knowledge outside own are of expertise during services– Applies whether model is proprietary in nature or not– Refer to Section 4 if depart from standards due to law or judgment

• Definitions:– Expert‐ one qualified by knowledge, skill, training, or education to

render opinion on matter at hand– Model‐ Information structure such as a set of mathematical

equations, logic, or algorithms used to represent phenomena



159

ASOP 38‐ Practices

• Actuary may find it appropriate to use models that incorporate specialized knowledge outside their own area of expertise

• When using such a model, the actuary should:– Determine appropriate reliance on experts– Have a basic understanding of the model– Evaluate whether model is appropriate for intended application– Determine that appropriate validation has occurred– Determine the appropriate uses of the model

• Level of effort in understanding and evaluating should be consistent with intended use and materiality to results of the actuarial analysis



160

ASOP 38‐ Reliance and Testing

• Actuary may rely on experts concerning aspects of the model that are outside own area of expertise. Experts may be those who provided the model or may be other experts

• To determine the level of reliance, the actuary should consider:– Whether individuals relied upon are experts in the applicable field– Extent to which model has been reviewed or opined on by experts– Known differences of expert opinion in aspects material to use of model– Whether there are standards that apply to model or testing/validation– Whether the model has been certified as having met such standards



161

ASOP 38‐ Understanding Models

• Actuary should:– Be reasonably familiar with basic model components, how they interrelate– Identify fields of expertise used in developing/updating model– Make a reasonable effort to determine if model is based on generally

accepted practices within the applicable fields of expertise– Be reasonably familiar with how the model was tested or validate and the

level of independent expert review and testing that was performed– Understand user input required, including the level of required detail of user input

needed to produce results consistent with intended use– Determine that model output is consistent with actuary’s intended use



162

ASOP 38‐ Appropriateness/Validation

• Appropriateness‐ actuary should:– Evaluate whether model appropriate for particular analysis– Consider limitations, modifications, assumptions needed to use output– Adequacy, if historical data underlies parameters, in representing the range of

reasonably expected outcomes given current knowledge– Make reasonable effort to be aware of significant developments in the relevant

fields of expertise and evaluate if materially impact analysis• Validation‐ actuary should:

– Evaluate quality and availability of input data per ASOP 23 Data Quality– Examine output for reasonableness versus alternate methods, historical observations,

consistency/reasonableness among output results– Consider sensitivity of model output to variations in user input/model assumptions– Decide if any adjustments are required and disclose those adjustments

• Actuary may rely on another actuary to perform the evaluation/validation



163

ASOP 38‐ Documentation/Disclosure

• Standard requires documentation whether or not a legal or regulatory requirement exists– Evaluation of model– Use of model output in analysis– Demonstrate how actuary met all the requirements listed above– How complied with standard if there are proprietary elements

• Standard requires disclosure/communication– Model used– Adjustments made to output– Disclosures per ASOP 41 4.2‐4 for legally mandated assumptions,

reliance on assumptions made by others, deviations from ASOP 38



164

Pending Modeling ASOP

• Original work on standards for models was motivated by catastrophe modeling in P&C which relied on experts who were not actuaries‐ resulted in ASOP 38

• Increasingly, model work is entering directly into financial statements so in 2010 ASB asked Life Committee to work on a an ASOP focused on standards for Models, In December 2012 ASB created two Task forces‐ one for General Models in all practice areas and another to update ASOP 38 focusing exclusively on Catastrophe Models

• General Model ASOP 2nd Exposure Nov 2014, Comment Deadline March 1, 2015• Will be effective for work performed 9 months after adopted by ASB• Revised ASOP 38 Catastrophe Modeling (all practice areas) released concurrently



165

Pending Modeling ASOP: Draft Comments

• First Exposure in June 2013 raised some key clarification requests:– Guidance regarding applicability– Responsibility when actuary is part of a modeling team– Guidance when reviewing models of other actuaries

• Second Exposure redraft focused on– Making clear that actuarial judgment required to determine extent to which full

application of the standards is warranted and when NOT based on intended use– Guidance when actuary relies on colleagues/vendors who may/may not be actuaries– Guidance when a model has material limitations or doesn’t fulfill purpose– Discussion why adding margins to assumptions/parameters might be appropriate



166

Pending Modeling ASOP‐ Scope

• ASOP applies to selecting, designing, building, modifying, developing, using, reviewing, evaluating models when performing Actuarial Services

• “Using a model” includes using the results of a model• Applies to all forms of models in all practice areas• Section 3.1 deals with models where results are not heavily relied upon

or do not have material financial effects‐ acknowledging that in those circumstances not all guidance is necessary or practical

• As usual has a Section 4 for departures to comply with law or for any other reasons to be explained and disclosed by the actuary



167

Pending Modeling ASOP: Definitions‐1

• Assumptions‐ Input that represent expectations or possibilities based on professional judgment or prescribed by law

• Data‐ Inputs that represent facts/information collected from sources such as records, experience, experiments, surveys, or observations

• Granularity‐ level of detail built into a model. Models with greater granularity may provide more precision/flexibility but may require greater effort to design, maintain, assemble, run

• Implementation‐ An executable form of a model

• Input‐ Data, Assumptions, or parameters used by model to produce output

• Intended Application‐ The designer’s planned use of the model



168


• Intended Purpose‐ The intended application or project objective, or both, depending on actuary’s role at the time Actuarial Services are performed.

– Intended Application‐ if includes designing, building, or developing the model or modifying, reviewing/evaluating model before selected or used

– Project objective‐ if Includes selecting/using model in a specific project or if includes modifying, reviewing, evaluating model when being selected/used

• Model‐ Representation among variables, entities, or events using statistical, financial, economic, mathematical, scientific concepts and equations. Models help explain system, study components, derive estimates, guide decisions.

– Consists of specification, implementation, one or more model runs



169


• Modeling‐ Selecting, designing, building, modifying, developing, using, reviewing, or evaluating models.

• Model Risk‐ Risk of adverse consequences from decisions made as a result of a model that doesn’t adequately represent what is being modeled

• Model Run‐ output of a model derived from a given input

• Parameters‐Mathematical, financial, economic, scientific, or statistical input to models that when varied result in different output.



170


• Principal‐ A client or employer of the actuary

• Project objective‐ Specific goal or question actuary is addressing when selecting or using a model to meet the needs of the principal or actuary

• Specification‐ Description of the model identifying inputs, interactions, and output through the use of logic, algorithms, or mathematical formulas



171

Pending Modeling ASOP: Applicability

• Applies to actuarial practice regarding models in all practice areas, subject to:

• Full application is appropriate when, in the actuary’s judgment, intended users of the model rely heavily on results and use of the results of the model has a material financial effect e.g. corporate planning, ratemaking and reserving. Materiality should be guided by ASOP 1 Section 2.6

• Full application may not be necessary‐ results not heavily relied upon or not material financially

• Actuary should use judgment deciding extent of application‐ extent of reliance by intended user and materiality of financial effect within context of use of model results and requirements of principal‐ facts reasonably known by actuary at the time that Actuarial Services are performed.

• If judgment dictates applying some or all guidance not warranted ‐ not a deviation. • If applying some or all guidance is warranted but not followed‐ is considered a deviation• Not following warranted guidance, even if not practical, is considered a deviation



172

Pending Modeling ASOP: Reliance Others

• Model built by colleague/vendor‐may limit ability to understand model

• Actuary shouldmake reasonable effort given the project objective to have a basic understanding of the model, including:– Intended application– General operation– Major sensitivities and dependencies within model– Key strengths and limitations

• If actuary is part of a modeling team, should either personally confirm or may reasonably rely on others who have confirmed that the applicable guidance from this ASOP is followed



173

Pending Modeling ASOP: “Intended Purpose”

• Actuary should select, design, build, modify, develop, or use a model that meets the intended purpose. An actuary reviewing or evaluating a model should confirm that the model meets the intended purpose.

• (*A*) When the actuary designs, builds, develops, reviews, evaluates model should confirm capability is consistent with intended application such as considering granularity of inputs, relationships recognized, ability to identify volatility around expected values

• (*B*) When selecting, reviewing, evaluating, or using the model the actuary should do so in the context of meeting project objective. In using model, efforts to improve model inputs, formulas, documentation, controls, validation, and presentation of results should be consistent with project objective

• When modifying a model to change the intended application or improve its ability to meet intended application, the actuary should be guided by Section 3.21 (*A*). When modifying a model to improve model inputs, formulas an outputs to meet project objective the actuary should be guided by 3.22 (*B*)

• Actuaries responsibility may include expressing opinion, using results, communicating results or preparing documentation. Here the actuary should understand important aspects of model being used, basic operations, important relationships, major sensitivities, strengths, potential weaknesses AND whether, and extent which model can fulfill intended purpose given information, time constraints, and practical considerations.



174

Pending Modeling ASOP: Model Structure

• Actuary should evaluate whether structure of model is appropriate for intended purpose. Considerations should include, where appropriate:

– Which provisions and risks specific to a business segment, contract, or plan are material and appropriate to reflect in the model

– Whether grouping model inputs will produce reasonable results– Whether use of the model requires a particular level of granularity– Whether deterministic, stochastic results, or both are needed, AND– Whether projection of future results might be materially influenced by choices and

options available to the entity being modeled in whole, part, members, counterparts



175

Pending Modeling ASOP: Inputs/Assumptions

• Actuary should refer to ASOP 23 selecting, reviewing/evaluating data to be used‐ directly, deriving assumptions

• Actuary should use assumptions/parameters appropriate to intended purpose:– Basing them on actual experience to extent available, relevant, sufficiently reliable– Other relevant experience properly modified to circumstances if NOT– Professional judgment to modify other available sources of information

• Actuary should determine whether adjusting assumptions or parameters to include a margin having amaterial effect is appropriate when experience data isn’t fully credible, for conservatism, to adjust for bearing risk, or to account for future unpredictability

• Actuary should consider whether range of assumptions/parameters, number of model runs reflect a range of conditions consistent with the intended purpose

• Actuary should use assumptions/parameters consistent with underlying economics‐ disclose if not due to legal prescribed assumptions, conservatism or other reason

• If reusing existing model whether inputs still appropriate or should be refreshed



176

Pending Modeling ASOP: Model Risk

• Should use reasonable/appropriate model risk mitigation‐ validation, governance, controls• Nature and degree of validation should be consistent with complexity, intended purpose• Should use governance and controls to maintain integrity, avoid unintended changes

• Validate each model run/set of runs relied on by intended user– Reconcile model input to actual data‐ documenting material differences– Check formulas, logic, tables‐ depending on context, controls, changes– Test against historical actual results, where applicable

• Depending on project objective, should– Perform analytic tests on model results for reasonableness– Reconcile against prior runs if change assumptions, data, formulas‐ keep reconciliation– Run tests of variations on key assumptions and parameters to ensure run consistently– Compare results to alternative models

• If appropriate, obtain reasonableness peer review of input, construction, model results



177

Pending Modeling ASOP: Documentation

• For model results to be used in Actuarial Communication the actuary should document nature of data used and material Assumptions and Parameters

• Follow ASOP 41 Sections 3.4.1 and 3.4.2 even if NO report created



178

Pending Modeling ASOP: Communication

• Actuary should explain Intended purpose, how address intended users’ needs– Methodology, key assumptions/parameters, Inherent limitations, uncertain results– Material changes since most recent comparable communication– Reconciliation to prior reports– Extent model fails intended purpose due to limited info, time, practical considerations– Any other material limitations and the implications

• Actuary should consider description of optimism or conservatism inherent in model inputs and methodology in relation to expected future experience

– Quantitative, qualitative, or directional

• Actuary should disclose Reliance on Others– Projections or Analysis by Others‐ deeming such as data under ASOP 23– ASOP 41 for responsibility for data, assumptions, parameters, methods



179

Pending ASOP on Principles Based Reserves

• 2nd Exposure June 2014, Comment deadline Dec 15, 2014• Effective for work performed starting 4 months after adopted by ASB

• Under VM‐20, company, NOT actuary responsible to regulators for compliance BUTone or more qualified actuaries is responsible to senior management of company for overseeing calculation of PBR and for signing the PBR Actuarial Report.

• ASOP frequently refers actuary back to specific text of VM‐20 to avoid duplication

• ASOP focuses only on reserves for which company experience is used to some extent in setting assumptions or when cash flow models used



180

Pending ASOP on PBR‐ Scope

• Actuarial Services performed by actuary on behalf of life insurance companies or fraternal benefits

• In connection with developing or opining on principles based reserves for life insurance subject to VM‐20 where such reserves are represented by actuary as being in compliance with Dec 2012 SVL/Valuation Manual

• Actuary should refer to Section 4 if depart for legal or other reasons



181

Pending ASOP on PBR‐ Definitions/Methods

• Section 2, defines 22 different terms for use within the ASOP, several ofwhich relate to data and modeling:

– anticipated experience assumption, credibility, granularity, margin, model segment, modelcell, prudent estimate assumption, qualified actuary, relevant experience, risk factor,scenario, sensitivity test

• Requires actuary to be familiar with SVL and Valuation Manual and bequalified as described in VM‐20 and VM‐31 to sign PBR statements

• Requires calculation of stochastic reserve, deterministic reserve, and netpremium reserve to follow VM‐20 Section 2

• Requires net premium reserve assumptions/methods in VM‐20 Section 3• Requires Exclusion tests to follow VM‐20 Section 6 w.r.t. Grouping• Requires VM‐20 Section 7,8,9 models/assumptions for stochastic and

deterministic reserves



182

Pending ASOP on PBR‐ Validation/Assumptions

• Section 3.4 of ASOP has detailed static and dynamic Model Validation requirements including granularity, historical back‐testing, scenarios and consideration over controls, model changes, assumption changes, refreshing stale data, scenario‐dependent assumptions and dynamic policyholder behavior and management actions

• Points actuary to ASOP 23 Data Quality and ASOP 25 Credibility in setting assumptions and VM‐20 Section 9 for grading into industry tables

• Requires considering whether reasonable to assume the range of policyholder behavior constrained to historical outcomes/experience

• Requires considering policy values/optionality from policyholder view• Requires margins to be added to anticipated experience assumptions for estimation

error and moderately adverse deviations

So all of this needs to be considered when using “Big Data” techniques for valuation



183

ASOPs Based on Area of Practice

• ASOP 2‐ Non‐Guaranteed Elements• ASOP 12‐ Risk Classification• ASOP 24‐ Illustration Model Reg.

• ASOP 7‐ Analysis of Cash Flows• ASOP 22‐ Asset Adequacy Opinions• ASOP 10‐ US GAAP Methods/Assumptions• ASOP 17‐ Expert Testimony• ASOP 21‐ Assisting Auditors/Examiners

• ASOP 46‐ Risk Evaluation in ERM• ASOP 47‐ Risk Treatment in ERM


Pricing/Product Development ASOPs

• ASOP 2‐ Non‐Guaranteed Elements• ASOP 12‐ Risk Classification• ASOP 24‐ Illustration Model Reg.


Financial Reporting ASOPs

• ASOP 7‐ Analysis of Cash Flows• ASOP 22‐ Asset Adequacy Opinions• ASOP 10‐ US GAAP Methods/Assumptions• ASOP 17‐ Expert Testimony• ASOP 21‐ Assisting Auditors/Examiners


186

ERM Related ASOPs

• ASOP 46‐ Risk Evaluation in ERM• ASOP 47‐ Risk Treatment in ERM


187

Doing Data Analysisin a Compliant Way


188

Compliant Data Analysis/Modeling Considerations

• Qualified for Project? CE Compliant for SAOs? Teammates? Staff? • Do you have quality data?

– Relevant, Historic/Current, Comprehensive, Scenario‐Coherent, Credible?– Budget time/people? Relying on others to compliantly check/validate?

• Are you building a model or using one?– Do you have a plan to test/validate basic model? Sensitivity Test?– Enough homogeneous data to “hold back” for testing?– Relying on others to compliantly check/validate?– Are non‐actuaries building/servicing model? Experts? State‐of‐Art? – Is the model locked down, version control, secure?

• How do you plan to adjust inputs/parameters for bias? Margins?• How are you documenting data, models, adjustments, biases, issues?• Are you communicating in a way tailored to Principal/Intended Users?

Managing the Big Data Opportunity Mangini Actuarial and Risk Advisory

LLC May 6, 2015189

Reproducible Data AnalysisAnd Literate Programming


190

Reproducibility/Communication of Results

• Common practice‐ inefficient validation and compliance– Considerable time/effort to manually document– Data, Analysis, Reports typically separate documents/processes– Relink documents for changes? Version control?– Non‐standard data formats and proprietary storage– Different tools, platforms, sources: Admin, Reinsurance, Actuarial

Projection Software, EXCEL, SAS, SQL/DBs, PowerPoint/Word etc.

• Complicates Supervisor/Peer Review, Internal/External Audit, Responding to Regulators, ORSA/ERM‐manual reconstruction


191

Reproducibility Tools

• Actuarial Analyses too complicated to “replicate”‐ scientific experiment• ASOPs in a nutshell‐ require “reproducibility” not replication• Free Tools such as R, Python, Octave can permit high power data

analysis and machine learning while fostering “reproducibility”

• Excellent 25‐minute video from Prof. Roger Peng from John Hopkins’ Bloomberg School of Biostatistics that you should watch on the concept of reproducibility in general, and how R‐Studio/R can facilitate:

http://www.r‐bloggers.com/literate‐statistical‐programming‐with‐knitr‐creating‐reproducible‐analysis‐in‐r/


192

Literate Programming in R

• A data‐based work product as a stream of English text and programming code– Data retrieval code‐ static websites, data feeds– Analysis code‐ chunks‐ inferential statistics, regression, machine learning, actuarial – Presentation code‐ formats results into tables, charts, etc– Document text explains data and actuarial analysis (ASOP 23 and ASOP 41)

• Literate programs “weaved” to produce human‐ readable documents• Literate programs “tangled” to produce machine‐readable documents• Need a documentation language and a programming language, Weave/Tangle Tool

– R markdown for documentation, R for Analysis, Knitr Tool to weave/tangle

Sources:http://en.wikipedia.org/wiki/Literate_programminghttp://www.r‐bloggers.com/literate‐statistical‐programming‐with‐knitr‐creating‐reproducible‐analysis‐in‐r/


193

Literate Programming‐ R Tools

• R Studio‐ freeware, open‐source environment for R with enormous support community

– http://www.rstudio.com– Works with Windows, Mac, Linux– “Shiny”‐ creates interactive web apps without HTML, CSS, Java knowledge

• “knitr” R Package weaves and tangles text and programs built into R Studio– Supports LaTeX, R Markdown, HTML as document platforms– Exports to PDF for “security” or HTML for publishing

• R Markdown is simplified HTML Markup language for dynamic docs with easy commands: http://rmarkdown.rstudio.com


194

Literate Programming MOOCs

• Highly recommend the Coursera “Data Science” MOOC Specialization which covers using R to retrieve/clean data, analyze data, literate/reproducible work

• Can take the classes “free” if don’t want ID verified certificates‐ run every month• Free Courses in R, Python, SQL also available online‐ IBM’s Big Data University• http://bigdatauniversity.com


Academy Professionalism Resources

• American Academy of Actuaries Website:• http://www.actuary.org/content/professionalism


QUESTIONS?

[email protected]: 516‐418‐2549


197

198 198198

Session 8: Completion of Big Data Discussion

Neil Raden

The Data Scientist

• Term invented by Yahoo• Super‐tech, super‐quant• Business expert too• Orientation: Search and Web• We used to call them quants• Few and far between• How do you find/train them?• Hint: like actuaries



Chief Actuary of GeoSpatial Analytics and ModelingChief Analytic OfficerChief Analytics & Algorithms OfficerChief Analytics OfficerChief Credit & Analytics OfficerChief Data and Analytics OfficerChief Research & Analytics OfficerChief Scientist, Global Head of AnalyticsChief Scientist, VP of AnalyticsChief Technology Officer, Enterprise Information Management & AnalyticsClient Director, Business AnalyticsDirector ‐ Advanced AnalyticsDirector ‐ Analytic ScienceDirector – Analytics DeliveryDirector ‐ BI & AnalyticsDirector ‐ Fraud Analytics & R&DDirector ‐ Predictive AnalyticsDirector (Analytics and Creative Strategy)Director (Marketing Analytics)Director : Digital AnalyticsDirector Analytics Strategy, JMPDirector Marketing AnalyticsDirector of Advanced AnalyticsDirector of Analytic Consulting, Product/Data Loyalty AnalyticsDirector of Analytic SolutionsDirector of AnalyticsDirector of Analytics (consultant)Director of Data Analytics and Advertising PlatformsDirector of Digital Analytics and Customer InsightDirector of Health AnalyticsDirector of Innovation, Big Data AnalyticsDirector of Product, AnalyticsDirector of Risk Analytics and PolicyDirector of Science & Analytics for Enterprise Marketing Management (EMM)Director of Web Analytics and OptimizationDirector, Advanced AnalyticsDirector, Advanced Analytics, HumanaOneDirector, Advanced Strategic Analytics

Director, Analytic ScienceDirector, Analytic StrategyDirector, Analytical ServicesDirector, AnalyticsDirector, Big Data Analytics and SegmentationDirector, Business AnalyticsDirector, Business Analytics & Decision Management StrategyDirector, Business Intelligence & Analytics, PogoDirector, Business Intelligence and AnalyticsDirector, Business Planning & AnalyticsDirector, Center for Business Analytics, Stern School of BusinessDirector, Clinical AnalyticsDirector, Customer AnalyticsDirector, Customer Analytics & PricingDirector, Customer Insights and Business AnalyticsDirector, Data AnalyticsDirector, Data Science & Analytics PracticeDirector, Data Warehousing & AnalyticsDirector, Database Marketing & Analytics (Marketing)Director, DVD BI and AnalyticsDirector, Gamification Analytics Platform, Information Analytics & InnovationDirector, Global Digital Marketing AnalyticsDirector, Group AnalyticsDirector, Head of Forensic Data AnalyticsDirector, Marketing AnalyticsDirector, Marketing Analytics for Bing Product GroupDirector, Oracle Database Advanced AnalyticsDirector, Predictive Analytic ApplicationsDirector, Reporting/AnalyticsDirector, Risk & AnalyticsDirector, Risk and Business AnalyticsDirector, Statistical Modeling and AnalyticsDirector, Statistics and Project Analytics / Senior Analytic ConsultantDirector, Strategic AnalyticsDirector, Web AnalyticsDirector/Head of AnalyticsDirector/Principal, Analytics

This Is Getting Ridiculous

201


Types of AnalyticsData Mining

XX

XX

X

X

X XX

XX

X

X

XXX X

XX

XXXXX X

X

X XX

X X

XX

X

XX

X

XX

XXXXX X X

X XX

X X

XX

XX

X XX

X

X

XX X

X

X

X

X

XX

Who are my best/worst customers? How do I turn my data into rules for better decisions?

Predictive Analytics

How are those customers likely to behave in the future? How do they react to the myriad ways I can “touch” them?

Optimization

How do make the best possible decisions given my constraints?

Knowledge - Description Action - Prescription

Business Intelligence

How do I use data to learn about my customers? What has been happening in my business?

Descriptive Analytics ‐ Improve Rules


**

*

*

**

* ***

*

***

**

*

*

*

**

**

**

*

**

*

**

*

*

***

**

** * *

***

*

* *

**

***

**Low‐moderateincome, young

HighIncome High income,

low‐moderate education

Moderate‐high educationlow‐moderate income

High

Moderate education,low income, middle‐aged

Low education,low income

Education

High

Predictive Analytics – Add Insight


10

20

30

40

Member completes treatment

Member fails to complete treatment


Impact May Take Time to Play Out

Stat Tools Can Be Dangerous


• Tests are not the event • Tests are flawedTests detect things that don’t exist

• Tests give test probabilities not the real probabilities • False positives skew results • People prefer natural numbers• Even Science is a test

Anscombe’s Quartet


Spurious Correlation


Texas Sharpshooter Fallacy


Types of AnalysisDescriptive Title Quantitative

Sophistication/NumeracySample Roles

Type I Quantitative R&D PhD or equivalent Creation of theory, development of algorithms. Academic /research. Work in business/government for very specialized roles

Type II Data Scientist or Quantitative Analyst

Advanced Math/Stat, not necessarily PhD

Internal expert in statistical and mathematical modelling and development, with solid business domain knowledge.

Type III Operational Analytics Good business domain, background in statistics optional

Running and managing analytical models. Strong skills in and/or project management of analytical systems implementation

Type IV Business Intelligence/ Discovery

Data and numbers oriented, but no special advanced statistical skills

Reporting, dashboard, OLAP and visualization, some design, posterior analysis of results from quantitative methods. Spreadsheets,“business discovery tools”


Analytic Types

Types of AnalysisDescriptive Title Quantitative

Sophistication/NumeracySample Roles

Type I Quantitative R&D PhD or equivalent Creation of theory, development of algorithms. Academic /research. Work in business/government for very specialized roles

Type II Data Scientist or Quantitative Analyst

Advanced Math/Stat, not necessarily PhD

Internal expert in statistical and mathematical modelling and development, with solid business domain knowledge.

Type III Operational Analytics Good business domain, background in statistics optional

Running and managing analytical models. Strong skills in and/or project management of analytical systems implementation

Type IV Business Intelligence/ Discovery

Data and numbers oriented, but no special advanced statistical skills

Reporting, dashboard, OLAP and visualization, some design, posterior analysis of results from quantitative methods. Spreadsheets,“business discovery tools”


Analytic Types

Type VBetter BI/Viz/Disco

Training/Mentoring/Apps

Training/Mentoring/Apps

3rd Party Services

Type Shifting

A Typical Day• Basic data manipulations to wrangle data

and fit a variety of standard models ‐40%• Translate a business problem into the

design of a data analysis strategy ‐ 5%• Graphically explore data to motivate

modeling choices and improvements– 10%• Interpret and critically examine standard

model output – 5%• Test the performance of models on

holdout data ‐ 10%• Go to meetings – 30%


70% is not Data Scientist work

Type Shifting


• As much as 80% of “Data Scientist” work can be done by others

• Data gathering, cleansing, profiling, parsing and loading

• Data and process stewardship• Platform availability• Providing organizational and market domain

expertise• Creation of presentation material

Analytics is hard and takes resourcesAnalytics takes effort to create and assimilate Focus analytics on key leverage points of business

UPS focuses on where the package is

Marriott focuses on yield management

If you try to do everything, won’t do anything well.


Analytics Is Hard

A Final Thought About AnalyticsThe challenge of analytics is communication and

creating a shared understanding.

It’s about focusing on high impact areas, moving forward one step at a time, being skeptical, being

creative, searching for the truth.

Any company can“Compete on Analytics.”

But not like this


Stock Market Returns for the “Competing on Analytics” Cohort

‐80%

‐40%

0%

40%

80%

120%

Amazon

Marrio

tt

Hond

a

Intel

Novartis

Wal‐M

art

UPS

Veriz

on

P & G

Progressive

Capital O

ne

Yaho

o

Dell

Barclays

Average Stock Market Return

Five Things to Remember

• Data is an “asset,” people make it valuable• Your data scientists may well be a team• Communication, insight and reason more important than math

• You have lurking data scientists in your firm• Start with what matters, build confidence


Thank You


Neil RadenFounder, Hired Brains ResearchTwitter: NeilRadenBlog: http://hiredbrains.wordpress.comWebsite: http://www.hiredbrains.comMail: [email protected]: http://www.linkedin.com/in/neilraden

Apparently Life Insurance Is Ahead of All Other Industries in Big Data*


2/3’s of Life companies deployed big data analytics < 5 years ago

33% claim full‐scale operations for > a decade

½ using big data analytics for six or more functions including:

• marketing initiatives,• sales lead generation,• underwriting,• claims/fraud detection and prevention

* http://www.lifehealthpro.com/2014/12/02/two‐thirds‐of‐life‐insurers‐use‐big‐data‐analytics

BigDataRelatedResourcesforSessionParticipants

Open‐SourceDataScienceSoftware‐withrobustusergroupsandsupportnetworksR:DownloadServerforRandR‐StudioInterface:http://cran.r‐project.org/R‐BloggersUserCommunity:http://www.r‐bloggers.com/Python:DownloadSitefor“Canopy”Interface:https://store.enthought.com/Octave(FreeVersionofMatLab):GNUDownloadSite:https://www.gnu.org/software/octave/ProM(ProcessMiningSoftware):Downloadsiteandusergroups:http://www.processmining.org/FreeorLow‐CostSelf‐DirectedandMassiveOpenOnlineCourses(MOOCs):BigDataUniversity‐withcoursesonR,Python,SQL,Hadoop,MachineLearning:http://bigdatauniversity.com/CourseraandEdX‐withcoursesonMachineLearning,R,Python,ProcessMining:MachineLearningcoursefromStanford:https://www.coursera.org/course/mlProcessMiningCoursefromTechnischeUniversityEindhoven:https://www.coursera.org/course/procminDataScienceSpecializationTrackfromJohnHopkins:https://ww.coursera.org/specialization/jhudatascience/1DataMiningSpecializationTrackfromUniversityofIllinoisatUrbana‐Champaign:www.coursera.org/specialization/datamining/20?utm_medium=listingPageCalTechJPLpost‐grad/PhDSummerSchoolonBigDataAnalytics(veryadvanced):https://www.coursera.org/course/bigdataschoolPythonCoursesfromUniversityofMichigan(Coursera)andMIT(EdX):Basic:https://www.coursera.org/course/pythonlearnAdvanced:https://www.edx.org/course/introduction‐computer‐science‐mitx‐6‐00‐1x‐0Stanford,Courserahttps://class.stanford.edu/courses/Engineering/db/2014_1/abouthttp://www.coursera.org/course/dbDatabaseSystemsMITOpenCoursewarehttp://ocw.mit.edu/courses/electrical‐engineering‐and‐computer‐science/6‐830‐database‐systems‐fall‐2010/Syllabus

Date post:	14-Apr-2018
Category:	Documents
Upload:	lemien
View:	218 times
Download:	3 times

Presenters Vincent J. Granieri, FSA, EA, MAAA Leonard ... Albert Jeffrey Moore, ASA, MAAA Presenters...

Documents