MONDAY 13T H MAY 2013Research Methods

Finals Exam Review

The Exam

Is worth 15% of your gradeWill include:

- 40 multiple choice questions - 3 short answer questions

How to ask Questions

Open-ended Respondent is asked to provide his or her own

answer Produces narrative data

Closed-ended Respondent selects an answer from a list Choices should be exhaustive and mutually

exclusive Produces data that can be readily processed

Guidelines for Asking Questions

Make items clear – avoid ambiguous questions; do not ask “double-barreled” questions

Short items are best – respondents like to read and answer a question quickly

Avoid negative items – leads to misinterpretation

Avoid biased items and terms – do not ask questions that encourage a certain answer

Avoid terms that have bad associations

Questionnaire Construction

General questionnaire format – critical, must be laid out properly

Matrix questions – same set of answer categories used by multiple questions

Contingency questions – relevant only to some respondents – answered only based on their previous response

Ordering Questions

Ordering may affect the answers givenEstimate the effect of question orderPerhaps devise more than one versionDepending on whether it is in-person, or via

mail, internet etc. Begin with most interesting questions End with less interesting, demographic

data

Self-Administered Questionnaires

Can be home-delivered Researcher delivers questionnaire to home of

respondent, explains the study, and then comes back later

Mailed (sent and returned) survey is most common Researchers must reduce the trouble it takes to

return a questionnaire

Warning Mailings and Cover Letters

Used to increase response ratesWarning mailings – “address correction

requested” card sent out to determine incorrect addresses and to “warn” residents to expect questionnaire in mail

Cover letters – detail why survey is being conducted, why respondent was selected, why is it important to complete questionnaire

Follow-ups to mail surveys increase response rates

Survey Research

Surveys are best suited for studies that have individuals as their units of analysis

Strengths: Surveys tend to be high on reliability and

generalizability, but validity can often be a weak point Surveys useful in describing characteristics of large

population

Survey Research

Weaknesses Standardized questionnaire items often

represent the least common denominator in assessing people’s attitudes, orientations, circumstances, and experiences

Surveys often appear superficial in their coverage of complex topics

Difficult populations are hard to contact through customary sampling methods

Generally weaker on validity and stronger on reliability

Subject to recall error Social desirability may be a problem

What is Field Research?

Field research is usually associated with qualitative data

It encompasses two different methods of obtaining data: Direct observation Asking questions

Often no precisely defined hypotheses to be tested

Used to make sense out of an ongoing process

Questions related to field research tend to be ‘How?’ and ‘Why?’

Characteristics of Qualitative Research

Context is criticalIn-depth and detailedResearcher immerses self in data Bias is recognizedInductive rather than deductive (bottom up)Data are interpreted rather than analyzed

Roles of the Observer in Field Research

Full participant (e.g., a participant in a demonstration against stop and frisk)

Participant as observer (e.g., Perrone’s research on drug use in New York dance clubs)

Observer as participant (e.g., observational studies of the police)

Complete observer (e.g., research in a courtroom setting, or setting that is open to the public)

Recording Observations

Note-taking, tape recording when interviewing and when making observations (dictation device)

Videotaping or photographs can make records of “before” and “after” some physical design change

Field notes – observations are recorded as written notes, often in a field journal; first take sketchy notes and then rewrite your notes in detail

Structured observations – observers mark closed-ended forms, which produce numeric measures

Strengths and Weaknesses of Field Research

Strengths:Provides great depth of understandingFlexibility (no need to prepare much in

advance)More appropriate to measure behavior than

surveysHigh on validityWeaknesses:Low on reliability – often very personalGeneralizability – personal nature may

produce findings that may not be replicated by another

Precise probability samples can’t normally be drawn

Agency Records

Published Statistics – government organizations routinely collect and publish compilations of data (e.g., NCVS, Census Bureau, BJS; often available in libraries and online

Nonpublic Agency Records – agencies produce data not routinely released (e.g., police departments, courthouses, correctional facilities)

New Data Collected by Agency Staff – collected for specific research purposes; less costly than collecting the data yourself and more control

Agency records are often used for descriptive studies

Problems with Reliability and Validity

Virtually all criminal justice record keeping is a social process – “social production of data” Records reflect decisions made by CJ personnel as well as

actual behavior by juveniles and adults Discretion factors in keeping records

Criminal justice organizations are often more interested in keeping track of individual cases than in examining patterns

Potential for clerical errors due to volume of dataUsers of data series collected over time must be

especially carefulImperative that you understand how the data were

collected

Secondary Data Analysis

Sources – websites (BJS, NCVS, ICPSR, NACJD), libraries

Advantages Cheaper Faster Benefit from work of skilled researchers

Disadvantages Data may not be appropriate to your research

question Not useful for evaluation studies (which are designed

to answer specific questions about specific programs) Threats to validity

Content Analysis

Systematic study of messages – can be applied to virtually any form of communication Decide on operational definitions of key variables Decide what to watch, read, listen to & time frame of

documents Analyze collected data

Well-suited to answer “Who says what, to whom, why, how, and with what effect?”

Content Analysis

Systematic study of messages – can be applied to virtually any form of communication Decide on operational definitions of key variables Decide what to watch, read, listen to & time

frame of documents Analyze collected data

Well-suited to answer “Who says what, to whom, why, how, and with what effect?”

Examines content and meaning

Content Analysis

Essentially a coding operation Communications need to be coded according

to some conceptual frameworkChoice between depth & specificity of

understanding: Manifest content – visible, surface content –

similar to using closed-ended survey questions Latent content – underlying meaning

Evaluation Research

It is gaining in popularity among researchers that really want to make a difference

Federal requirements for program evaluations often accompany the implementation of new programs

Funding is available for program evaluationsCan utilize a number of different research

designsLinks the intended actions and goals of

criminal justice policy to empirical evidence that supports them having the desired effects

Conditions Requisite for Randomized Experiments

Staff must accept random assignment and agree to minimize exceptions to randomization

Case flow must produce enough subjects in experimental and control groups for statistical tests

Experimental interventions must be consistently applied to experimental and withheld from control group

Need equivalence prior to intervention, and ability to detect differences in outcome measures after intervention

The Policy Process

Begins with a demand for a new course of action or opposition to an existing policy

Policy makers consider their ultimate goals and means of achieving those goals

Resources are allocated (considerations include; personnel, equipment, supplies etc.)

What are the policy outputs (i.e. what is actually produced?)

What is the impact of the policy output?

Interpreting Data

Empirical research is a logical rather than a mathematical operation

Statistics – branch of math appropriate to research

Descriptive statistics – used to summarize and describe data in manageable forms

Inferential statistics – assist in forming conclusions from our observations; usually about a population based on studying a sample

Types of Analysis

Univariate analysis – describing single variables (e.g. # of males; average age; place of birth etc.)

Bivariate analysis – describes the associations that connect one variable with another

Multivariate analysis – examines relationships among three or more variables

Data Analysis

Central tendency - mean, median, modeMeans are susceptible to extreme values. A few very

large, or a few very small numbers can change the mean dramatically

Because of this, it is important to examine measures of dispersion

Simplest measure of dispersion is the range – the distance from the highest to the lowest value (e.g. 13 to 19 years)

Standard deviation – the average amount of variability in a set of scores (i.e. the average amount each individual observation varies from the mean)

The larger the standard deviation, the larger the average distance each data point is from the mean of the distribution

Rates

Fundamental descriptive statistics in criminal justice research

Used to standardize some measure for comparative purposes

Total Murders in Four States, 2004 Total Murders Total Population

California 2,407 35,894,000 Florida 946 17,397,000

Louisiana 574 4,516,000 Pennsylvania 650 12,406,000

Short Answer Questions

1. Compare and contrast the strengths and weaknesses of field research and survey research. Give specific attention to the topics of validity, reliability and generalizability2. Discuss the advantages and disadvantages of secondary data and give an example of each3. Identify the difference between the various roles of the observer in field research. Give an example of a research scenario that would be appropriate for each role

Final Class

Final Exam Article critiques due