DESIGN FEATURES OF NCHSSURVEYS

By Iris Shimizu

Mathematical Statistician

Office of Research and Methodology, NCHS

Disclaimer: The opinions in this presentation are those of the presenter and not necessarily those of NCHS.

OUTLINE• DESIGN FEATURES OF ESTABLISHMENT

SURVEYSo SAMPLE DESIGNo DATA WEIGHTS

• FEATURES COMMON TO ALL COMPLEX SAMPLE SURVEYS

(both establishment and population surveys)

3

ESTABLISHMENT SURVEYS

TARGETED ANALYSIS UNITS: • CLIENTS OF ESTABLISHMENT

• EVENTS OCCURRING AT/WITH ESTABLISHMENT

• ESTABLISHMENTS THEMSELVES

4

National Health Care Survey

• National Hospital Discharge Survey• National Survey of Ambulatory Surgery• National Ambulatory Medical Care Survey• National Hospital Ambulatory Medical Care Survey• National Nursing Home Survey• National Home and Hospice Care Survey

5

FEATURES

• MULTI-STAGE SAMPLING

• PRIMARY SAMPLING UNITS (PSUs)o ESTABLISHMENTSo AREAS (USED TO SAVE COSTS)

6

FEATURES (CONTINUED)

• STRATIFICATIONo GEOGRAPHYo PROVIDER SPECIALTYo SIZE (INPATIENT BEDS, VISIT VOLUME)o ESTABLISHMENT TYPEo OWNERSHIP TYPE

• SELECTION WITH PROBABILITY PROPORTIONAL TO SIZE (PPS)

7

FEATURES (CONTINUED)

• SAMPLING FREQUENCYo EVERY YEAR FOR PHYSICIANSo PERIODICALLY FOR OTHER

ESTABLISHMENTS BASIC SAMPLE –NEW DESIGN

UPDATES PERIODICALLY

8

FEATURES (CONTINUED)

• WITHIN ESTABLISHMENT SAMPLINGo TIME SAMPLEo VISIT SAMPLE –FROM FRAME PROVIDED

BY ESTABLISHMENTo STRATIFICATIONo SYSTEMATIC RANDOM SAMPLINGo PPS FOR SELECTING SERVICE AREAS

9

OVERALL PROBABILITY

• PRODUCT OF PROBABILITIES AT EACH SAMPLING STAGE

• ACCOUNTS FOR SAMPLING DESIGN FEATURES

10

DATA WEIGHTS

• INVERSE OF SELECTION PROBABILITIES

• ADJUSTMENT FOR UNIT NON-RESPONSE

• CALIBRATION – USES DATA FROM NON-

SAMPLE SOURCE FOR UNIVERSE

11

VARIANCES USING PUBLIC USE FILES

• REFER TO DATA FILE DOCUMENTATION

• FOR RECENT YEARS AND BARRING RISKS,

NEEDED DESIGN VARIABLES ARE IN FILES

• RESEARCH DATA CENTER

12

SUMMARY FOR ESTABLISHMENT SURVEY DESIGN

• DESIGNS USE MULTI-STAGE STRATIFIED SAMPLES

• WEIGHTS AND VARANCES REFLECT THE COMPLEX SAMPLES

13

TURNING ATTENTION

TO FEATURES COMMON TO

ALL SURVEYS

(BOTH POPULATION & ESTABLISHMENT

14

DANGERS OF NOT USING SAMPLE WEIGHTS

UNWEIGHTED ESTIMATES:• OF TOTALS WILL BE TOO SMALL

• OF RATES AND OTHER RATIOS COULD BE DISTORTED.

I.E., UNWEIGHTED SAMPLE PROPORTIONS COULD DIFFER FROM THE CORRESPONDING CENSUS PROPORTIONS

15

VARIABILITY OFSURVEY ESTIMATES

• ESTIMATES BASED ON SAMPLES ARE SUBJECT TO SAMPLING VARIABILITY

• ESTIMATES OF SAMPLING VARIANCES MUST ACCOUNT FOR SAMPLE DESIGNS FOR VALIDITY

16

COMPLEX SURVEY FEATURES AFFECTING VARIANCE ESTIMATION

• CLUSTERING ANALYTIC UNITS WITHIN PRIMARY SAMPLING UNITS (PSUs)

• STRATIFICATION OF PSUs

17

DANGER OF USING SAMPLE SUBSETS TO ESTIMATE VARIANCES

• VARIANCE ESTIMATES BASED ONLY ON SUBSETS OF SAMPLE MAY NOT CORRECTLY REFLECT SAMPLE DESIGN

• COULD UNDERSTATE SAMPLING VARIANCE

18

DANGERS OF IGNORING SAMPLE DESIGN IN VARIANCE ESTIMATION

• VARIANCE ESTIMATES PROBABLY TOO SMALL

• “DEGREES OF FREEDOM” WOULD BE TOO LARGE

19

GENERAL REFERENCE FOR SURVEY ANALYSIS SOFTWARE

http://www.hcp.med.harvard.edu/statistics/survey-soft/

Provides descriptions and links to software packages that do variance estimation with complex sample data.

20

SUMMARY

FOR ALL COMPLEX SAMPLE SURVEYS• SAMPLING WEIGHTS SHOULD BE USED

• SHOULD USE COMPLEX SAMPLE ANALYSIS SOFTWARE TO ESTIMATE VARIANCES

• SHOULD USE WHOLE SAMPLE TO ESTIMATE VARIANCES

21