Anonymity and Sensitivity
Fritz Scheuren, SpeakerNORC University of Chicago
February 2013
Scope of USA Migration Studies
• Emigration from USA to World• Immigration from World to USA
Direct Indirect
Outline of Talk
• Problem• Science of Approach• Engineering of Its Implementation• Results So Far!• Next Steps?
Immigrants Sans Paper
• Physical Border Protection Worldwide?•Mostly “Thought” to be South to North?• Only An Indirect Estimate Available
Legal Immigrants First?
• Over a Million Come Annually, still Largest Legal Flow Of Any Country• About 150 Million Made citizens since 1940
Undocumented May Be Much Smaller?
• Flow is believed to average Less than Legal • Stock Estimated, perhaps, at 12 million• Flow is Business Cycle Driven, but usually less
Physical Border Protection
• Extremely long Borders with Canada and Mexico• Mexico Border Not Well Guarded • Drug Trafficking a Major Problem for USA/Mexico
Mainly South to North?
• Mexican Border Crossers but This Model is too Simple• Student and Visitor Visa Overstays • Still Often Tied to US Jobs
Who Comes Legally Now?
• Traditional Americas and Western Europeans but Less legally than earlier• Eastern Europeans Still in Large Numbers• Koreans and Chinese up, Africa coming up
Science Approach
• Two Card Method (Handout)• Two Independent Replicates• Difference Estimate of Number of Undocumented
Two Card Example(Green Card)
First Card
• Two options Given --• First option: US Citizenship or Undocumented• Second Option: Green Card or Other
Two Card Example(Other than Green)
Second Card
• Two Options Given –• First Option: US Citizenship• Second Option: Green Card or Other• Both Cards: Safe Answer
Difference Undocumented
• Estimate Unbiased, if Question Understood and Answered Honestly• But Variance Twice as Large as Direct Question• However Bias Eliminated
Engineering Solution
• Human Factors Design Study• To test Factor Importance• Protecting Anonymity to Gain Unbiased Answers
A Credible Balancing Bar?
Balance of Trust?• Key is Creating High Anonymity to Overcome PERCEIVED level of risk.• Given the Sensitivity of the Subject• Need Data so we tested.
Sensitivity Loss
• The Two Card method has a Variance Cost and is a challenge in moving Univariate analysis to a Multivariate form• The Newness remains for Anonymity form
Anonymity Loss
• Credibility of Promise is a real concern?• So Move to a Promise that CONSTRUCTS Established Trust? • Key Improvement!
Results So Far• About 350 tests of the Two-Card method and its variations have been undertaken.• The tests show that the results are robust to minor wording changes
More Results So Far
• The Results are not robust to being wordy. • Potential respondent are not paid, probably a mistake. • If version was lengthy, more break-offs likely.
Still More Highlights
• Location changes, also are not robust. • Intercepts near the Mexican Border (near San Diego) were seen as more risky, so stopped.
Other Changes?
• If scaled up, more characteristics and more intercept locations are definitely needed.• Respondent based area probabilities needed to adjust for self selection.
Interviewer Training
• The extensive paradata calculated needs to be turned into a model for training.• Intercept Interviews are changing NORC Team. • How Measure/Control?
Intercept Scalability
• How replicable are the Interceptors we chose?• How replicable and scalable is this process?•What controls are there and how scientific are we?
Selected Respondent Comments
• [Interview] was easy, because none of the questions are hard and explained it very well.• [Glad you] understand the immigrants and their education level.
More Respondent Comments
• He was hesitant at first and I asked him why. • He said because people [might identify him].I asked him how I was going to find out. I told him [many] are similar].
Next Steps Coming?
•Write up Results of this small NSF grant?• Expand to another Undocumented group: Asians, Russian Jews?• Take Sting off Hispanics
Portable to Europe?Your Question!
• In a register based society is this needed?• Some evidence says Yes but little leakage?• More references needed? Short list follows?
References• Cochran, William G. Sampling Techniques.
New York: Wiley, 1977. Print.• Dyson, Freeman J. "Is Science Mostly
Driven by Ideas or by Tools?" Science 338.6113 (2012): 1426-427. Print. • Galison, Peter. Image and Logic: A Material
Culture of Microphysics. Chicago: University of Chicago, 1997.
References
• GAO. A "Grouped Answers" Approach to Surveying Foreign-Born Respondents. Rep. no. GAO-06-775. N.p.: GAO, 2006. Print.• Kuhn, Thomas S. The Structure of
Scientific Revolution. Chicago: University of Chicago, 1996. Print.
References• Lehoux, Daryn, and Jay Foster. "History
and Philosophy of Science: A Revolution of Its Own." Science 338.6109 (2012): 885-86. Print.
• Pasic, Peter.” What the Romans Really Knew,” Science 338:273-274.
• Reynolds, Gregg. Immigrant Experience Survey Interim Report. , 2013.