7/28/2019 Owen Perkins - CIPP
1/32
June 25, 2013Owen Perkins
The CIPP Approach toEvaluation
7/28/2019 Owen Perkins - CIPP
2/32
2
Meant to serve decision makers at all levels
Good decision making is dependent on evaluative
information A systems approach to evaluation
Decisions are made about inputs, processes, andoutputs
Approach was a shift from program objectives to the
decision(s) of program managers Evaluation of the decision(s) the administrator(s) mustmake
Management-Oriented Evaluation Approach
7/28/2019 Owen Perkins - CIPP
3/32
3
Daniel Stufflebeam, supported a decision-oriented evaluationapproach to assist administrators make good decisions
His approach was developed in the 1960s
His Evaluation method proposed that administrators face fourkinds of decisions
Context evaluation
Input Evaluation
Process evaluation
Product evaluation Emphasis is on decision making by program administrators
The CIPP Evaluation Model
7/28/2019 Owen Perkins - CIPP
4/32
4
Determine needs to define objectives of theprogram
Asks what should we do? Collect data to determine goals of program
Example: A standardized test is given toevery student in 6th grade across a state.
What are the objectives of this program?Raise level of education, standard to evaluateteacher performance, etc.
Context Evaluation
7/28/2019 Owen Perkins - CIPP
5/32
5
Determine resources, alternative strategies,
and determine plan that has best potential to
meet the needs of the program
Asks how should we do it?
Involves determining the steps and resources
needed to accomplish goal or objective
Input Evaluation
7/28/2019 Owen Perkins - CIPP
6/32
6
To evaluate how well the plan has been
implemented Asks are we doing it as planned?
Continuously monitor the program, allows
decision maker to see conflicts, strengths and
weaknesses of program and materials, howwell it is following the established plan, etc.
Process Evaluation
7/28/2019 Owen Perkins - CIPP
7/32
7
Determine program attainments
What results were obtained? How well were needs reduced?
How did the program work?
Compare actual outcomes to anticipated outcomes
Should the program being evaluated be continued,
modified or dropped?
Product Evaluation
7/28/2019 Owen Perkins - CIPP
8/32
8
1. Focus the evaluation
2. Collection of Information
3. Organization of Information
4. Analysis of Information
5. Reporting of Information
6. Administration of the Evaluation
Stufflebeam suggested evaluators followthese general steps:
7/28/2019 Owen Perkins - CIPP
9/32
9
Identify the level of decision making being addressed
For each level of decision making analyze potential
decisions Define criteria for each decision situation by
specifying variables for measurement and standardsfor use in the judgment of alternatives
Define policies within which the evaluator mustoperate
Focusing the Evaluation
7/28/2019 Owen Perkins - CIPP
10/32
10
Specify the source of information to be
collected
Specify the instruments and methods for
collecting the needed information
Specify the sampling procedure to be used
Specify the conditions and schedule forinformation to be collected
Collecting Information
7/28/2019 Owen Perkins - CIPP
11/32
11
Provide a format for the information that
is to be collected Designate a means for performing the
analysis
Organization of Information
7/28/2019 Owen Perkins - CIPP
12/32
12
Select the analytical procedures to be
used Designate a means for performing the
analysis
Analysis of Information
7/28/2019 Owen Perkins - CIPP
13/32
13
Define the audiences for the evaluation
reports Specify a way to present information to
audiences
Develop format for evaluation reports Schedule the reporting of information
Reporting of Information
7/28/2019 Owen Perkins - CIPP
14/32
14
Summarize the evaluation schedule
Evaluate the potential of the evaluation design forproviding information that is valid, reliable,
credible, timely, and pervasive
Specify and schedule means for periodic
updating of the evaluation design Provide a budget for the total evaluation program
Administration of the Evaluation
7/28/2019 Owen Perkins - CIPP
15/32
15
Holistic approach:
Broad, vague, and summative Good or bad?
Idealized notion of process and program:
What the process should be rather than itsactual form
Too top-down and managerial in approach
Strengths and Weaknesses of CIPP
7/28/2019 Owen Perkins - CIPP
16/32
16
Imprisonment and Career Development:
An Evaluation of a Guidance Program for Job Finding
By: Gemma Filella-Guiu and
Angel Blanch-Plana
7/28/2019 Owen Perkins - CIPP
17/32
17
This study describes the evaluation results of PORO;a three level occupational guidance program
Designed to improve offenders job findingpossibilities and to reduce reoffending afterimprisonment
Used Stufflebeams CIPP model, considering the
efficiency, effectiveness and impact of the program Evaluation outcomes and practical suggestions for
career development applications addressed to prisonpopulations are reported
Article Abstract
7/28/2019 Owen Perkins - CIPP
18/32
18
The Guidance Program for Job Search
(PORO) Go figure?
Wikipedia: Poro is a fraternal secret
society of Sierra Leone and Liberia
PORO Definition
7/28/2019 Owen Perkins - CIPP
19/32
19
All internees (prisoners) are categorized bytheir individual judicial situation
PORO comprises three levels of guidance tointernees:1. Internees given professional and occupational
training in prison
2. Internees given ordinary parole permits
3. Has two distinct sections: Internees go into community on daily basis
Probation and freed internees in community
Program Description
7/28/2019 Owen Perkins - CIPP
20/32
20
Context Evaluationis the delineation and specification of the programsenvironment, the population and sample of individuals to be used, andthe program objectives
Input Evaluation identifies and assesses system capabilities,determines methods that meet objectives and needs, and studies therelations between cost and potential benefits of each proposed method
Process Evaluation helps to provide feedback to the teachers on
progress of the program
Product Evaluation allows decisions to be taken regarding thecontinuity, termination or modification of the program
CIPPs Use in Study
7/28/2019 Owen Perkins - CIPP
21/32
21
What should we do?
Collect data to determine goals ofprogram
Context Evaluation
7/28/2019 Owen Perkins - CIPP
22/32
22
145 people in the experimental group
51 in the control group
(N = 196) 11% females and 89% males
Between 25 and 35 year old
Serving sentences of less than 4 years.
Specialties taken in the occupational courseswere car mechanics, graphic arts, industrialdressmaking, building, electricity, binding,carpentry and computing
Context Evaluation: Collect Data
7/28/2019 Owen Perkins - CIPP
23/32
23
The main objectives of this intervention were:a) To improve individual competence in job searching
b) To facilitate the acquisition of useful job searchknowledge and skills
c) To increase professional self-esteem
The accomplishment of these specific aims is as
fundamental for enabling ex-convicts to adjust tothe job market and the community on a long-termbasis
Context Evaluation: Determine Goals
7/28/2019 Owen Perkins - CIPP
24/32
24
How should we do it?
Determine resources, alternativestrategies, and determine plan that has
best potential to meet the needs of the
program
Input Evaluation
7/28/2019 Owen Perkins - CIPP
25/32
25
The contribution of an expert group was requested toassess the design and methodological validity of PORO inorder to achieve the best possible adaptation to aconfinement context
The potential benefits that might be obtained were
rewarding enough considering the assignment ofoperational, material and human resources = Cheap Program
Input Evaluation: Determine plan that has bestpotential to meet the needs of the program
7/28/2019 Owen Perkins - CIPP
26/32
7/28/2019 Owen Perkins - CIPP
27/32
27
Feedback from all course teachers and 52 randomly
selected participants on the program were used toevaluate the process dimension
No serious difficulties arose during the teachingprogram on the majority of occupational courses
Internees were not very enthusiastic about the programin its initial stages. However, in the end, this lack ofmotivation was gradually overcome during the programdevelopment, final opinions being highly positive
Process Evaluation: How well has theplan been implemented?
7/28/2019 Owen Perkins - CIPP
28/32
7/28/2019 Owen Perkins - CIPP
29/32
29
Increase in job knowledge and skills scores
Increase in professional self-esteem scores
Based on the findings it should be expected that other
internees would reach these same employment rates whentheir sentences have been served
The program had a high level of efficiency: The material, human and operational resources used in the
[program] development did not pose any additional expense
The program did not have any adverse effects: Undesirable contingencies were not noted once the investigation
finished, nor were unforeseen relevant consequences due to theoccupational guidance [program] detected
Product Evaluation: Determine Program Attainments
7/28/2019 Owen Perkins - CIPP
30/32
30
The resultsgive further support to theidea that career intervention addressed atimprisoned people in adverse social andeconomic circumstancesis of paramountimportance for improving the employment
possibilities of people whose [behavior] andlack of skills are troublesome to them and tosociety
Conclusion
7/28/2019 Owen Perkins - CIPP
31/32
31
This case study is an excellent example
of the CIPP evaluation approach Each step of the approach was carefully
considered
Conclusion, contd
7/28/2019 Owen Perkins - CIPP
32/32
June 25, 2013GO HOKIES!
The End