Post on 27-Jul-2018
transcript
IRRIInternational Rice Research Institute
Guidelinesfor the Disseminationand Impactof IRRI Information(GN1-05)
etting he essage utT M OG
March 2000IRRI
International Rice Research Institute
Guidelinesfor the Disseminationand Impactof IRRI Information(GN1-05)
etting he essage utT M OG
ACKNOWLEDGEMENTS
AUTHORS:
Revised version:
Paul Marcotte, IRRI Training Guidelines, March, 2000
Original Publication:
Luka 0. Abe, Paul Marcotte, Robert Raab: ISNAR Training Guidelines 1990
IRRI PRODUCTION STAFF
Editor: Madeline B. QuiamcoLayout and Graphic Artist: Irvin PanganibanProduction: Gina E. Zarsadias and Rina P. Coloquio
International Rice Research InstituteIRRI Training Guidelines: A Handbook for the TrainerDAPO Box 7777Metro ManilaLos Baños, Laguna, Philippines
TABLE OF CONTENTS
Introduction
Section 1: The Design and Delivery of a Training Event ........................1
Step 1: Assessment of Needs ..........................................2
Step 2: Prospectus/Agenda Development ........................3
Step 3: Module Construction ..........................................5
Step 4: Testing and Validation .........................................5
Step 5: Redesigning ........................................................6
Step 6: Production of Materials .......................................6
Step 7: Delivery ..............................................................7
a. The Creation of the Learning Environment .....8
b. Facilitation Skills ..........................................12
c. Presentation Skills .......................................15
Step 8. Evaluation .........................................................18
Section 2: Attachments
1. Prospectus/Agenda Example .......................................23
2. Event Evaluation Form ................................................31
INTRODUCTION:
Training is a process that enables individuals to acquire knowledge, skills/tools, andabilities that will allow them to fulfill the requirements of their job, achieve theircareer aspirations, and attain the goals of their organizations. For the organization,training is one of the processes by which it disseminates the products of its researchprograms, receives feedback from its partners and clients, and ultimately providesthe baseline to track the information and conduct evaluation. Thus, training ismultifunctional in that it plays a vital role in staff development and at the same timeis a conduit for information. Both functions are essential to the success and longevityof the organization.
IRRI has always considered training to be one of the key mechanisms in disseminatingits accumulated knowledge and techniques to a broad audience. Through its degreeand non-degree programs, it has developed human resource capacity in the riceresearch sector. Its distance learning program and materials are models for adult/distance learning. Its on-campus and regional training have literally changed theapproach and method of rice production in the world. Training has played a majorpart in the process historically, and remains an essential technique for multiplyingIRRI’s impact.
In recognition that training is one of the critical functions of an organization, wehave prepared this handbook for IRRI scientists, trainers, and our partners. Werecognize that our own science staff and NARS researchers change dramaticallyover time, that we now have new deliverers of information that include both privateand NGO/PO type organizations, that our information is accelerating rapidly withscientific and technological advancements. Given these developments, training servesan essential function in the development and continuation of the invisible college ofrice scholars and farmers.
This handbook contains:
Section 1: A generalized model of the design and delivery of a training event and
Section 2: Examples and forms
At the outset, it should also be noted that there are 3 overarching concepts thatapply to this handbook. The first is that this document has been prepared for all IRRIstaff at headquarters and in the field, not just the Training Center. It is recognized
that multiple programs, projects and individuals participate in and organize “training”events. These guidelines are intended to assist in the preparation and delivery ofthose events.
Secondly, “training” occurs in many forms such as consultations, conferences,presentations, seminars and workshops, and field demonstrations. In this handbookwhen the term “Training” is used, it is meant to be in the larger sense, ie. thedissemination of information where learning and application is the goal. The guidelinescontained in this handbook have been prepared to assist in any or all of those events,not just the Training Center.
Third, beginning in the year 2000, the Training Center is part of the IRRI wide effortto become proactive in its dissemination and evaluation of information. This includesall of the previous delivery mechanisms such as IPMO, TC, and CREMNET inconjunction with the Research Divisions. The goal is to cover the entire continuumfrom needs assessment including PRA, priority setting of the research agenda, researchexperimentation, to delivery, monitoring and impact evaluation. This also necessitatesthe identification of “new” delivery partners to efficiently reach the target group offield-based extension organizations and the farm communities. It is intended thatthe contents of this handbook become the standard operating procedure for allparticipants on the continuum when they are preparing and offering a “training”event.
IRRI Training Guidelines 1
Evaluation Assessment of Needs
Prospectus/Agenda
Testing and Validation
Redesign/Modification
MaterialProduction
ModuleConstruction
Delivery
Section 1: The Design and Delivery of a Training Event
The objective of this section is to describe a generalized model for training eventdesign and delivery. The model for training design and delivery of a training event, asdepicted in Figure 1, has at least 8 essential elements. These are: assessment ofneeds, prospectus/agenda development, module construction, testing and validation,redesigning/modification, material production, delivery/presentation, and evaluation.These elements are sequential and interdependent, beginning with the assessmentof needs. The model is also an interactive process, with evaluation validating individualelements and circumscribing the entire process. Each of these elements will bedescribed and illustrated in their development sequence.
Figure 1. The design and delivery of a training event
Validation/Evaluation Feedback
Development Sequence
k
IRRI Training Guidelines 2
STEP 1: Assessment of Needs
IRRI exists to achieve its goal, “… to improve the well being of present and futuregenerations of rice farmers and consumers.” In order to accomplish this, step one ofthe training event process is to identify relevant needs and wants of partners/targets/stakeholders; i.e., any party that has a substantial interest in the group’s work. Thisrequires a data-gathering process. Appropriate data collection instruments includefocus groups, participant observation, surveys, interviews, and maybe even intuitionbased on experience. The following are the recommended steps of the process.
1. Establishing Clear Goals: Prior to the development of a data collectioninstrument, tool or protocol, it is necessary to establish what is to beaccomplished by the data gathering process. Ask questions such as: why isthe data being collected and how will it be used?
2. Appropriate Data Gathering Instruments: There are a number of instrumentsfrom which to choose depending on the goal of the research. The followingare examples of instruments:
A. Focus group -- “… a facilitated group discussion in which open-endedquestions are asked in a way to trigger discussion amongst a panel ofparticipants.” (bdick@scu.edu.au, 2000).
B. Participatory Learning/Action -- There are a number of approaches thatfall into this category. Some are: activist participatory research; agro-ecosystem analysis; applied anthropology; farming systems; and rapidrural appraisal. The common characteristics of these approaches are:multiple perspectives, group learning, context specificity, facilitativeprocess, stakeholder involvement they lead to change. Pretty et al (IIEDTrainer’s Guide, 1995).
C. Observation Method -- This is a primary technique for collecting dataon nonverbal behavior: it may be participatory, in which the researcheris a participant in the activities being observed, or non-participatory, inwhich the observer does not participate in the activity being observed.(Bailey, 1994)
IRRI Training Guidelines 3
D. Survey Research -- This includes questionnaire, mailed questionnaireand interview, each of which have specific purposes, advantages anddisadvantages, and the selection of which to use requires clarity ofpurpose and use (Bailey, 1994).
3. Clear/Relevant Questions: Once the method has been selected, then clear andrelevant questions must be constructed. Proper questions collect data thatwill answer the research question, and at the same time serve reliability andvalidity concerns.
4. Adequate Sampling: While it would be ideal to study the entire population ofthe universe of the stakeholders, a correctly selected sample or subset of thepopulation that most accurately represents the population may be chosen.Rules are dependent on the degree of accuracy of representation desired.
5. Meaningful Data Analysis: The data collected must be turned into informationthat can be related to decisions that need to be made.
6. Results Presented: Clear, precise, targeted to the audience. See the section onPresentation that follows for more detail.
7. Informed Decisions: The ultimate goal of the needs analysis is to make aninformed decision based on reliable and valid information.
Once the needs analysis for training has been accomplished in a professional, reliableand viable way, the “trainer” can proceed to match organizational needs to personalcapabilities, and begin the development of a prospectus.
STEP 2: Prospectus/Agenda Development
Once an informed decision has been made based on the “needs analysis,” step twoof the design and delivery process is to develop a prospectus and agenda for thetraining event. The purpose of the prospectus/agenda is multifold. For the trainer,the prospectus provides a framework to accomplish the training objectives bydocumenting step-by-step the items that need to be accomplished for a successfulevent. For the trainee, the prospectus provides information, builds expectations,and allows for preparation for the event. There are at least 11 items that need to be
IRRI Training Guidelines 4
systematically addressed in the prospectus and agenda. The items and a briefexplanation are as follows:
Introduction/Background -- this section provides information to the participant onwhy the workshop is being held, its importance, and where it fits into the generalframework of institutional or individual development.
Purpose -- a purpose statement provides information on the specific reason why theworkshop is being held.
Content -- a description of the general content of the workshop provides informationon topical areas to be covered, builds expectations, and allows individuals to beginpreparation or stimulates thinking about the topics.
Objects/Expected Outcomes/Outputs -- the objectives, expected outcomes, and outputsmust be stated prior to the workshop and must be verifiable so that evaluations canbe conducted subsequent to the event.
Target group -- the target group must be carefully identified so that materials andworkshop dynamics can be prepared specifically to suit the situation.
Resource Team and Materials -- the team should be identified for informationalpurposes for the participants. Identifying materials and distributing them prior tothe workshop if possible will facilitate and accelerate the learning process.
Methods/Dynamics -- Pedagogical methods should be specifically defined, particularlyfor interactive and participatory events.
Logistics -- these must be stated prior to the workshop and should include venue,accommodations, transportation, and expense information.
Date -- advance notice will allow people to block out the time required.
Contact -- a contact person to clarify information should be identified.
Agenda -- a daily agenda should be set. This includes time, materials, trainerresponsibilities, and dynamics such as small groups, plenary presentations, etc.
A sample prospectus/agenda is attached for your information (See Attachment 1).
IRRI Training Guidelines 5
Proper Materials
Print MaterialsLecturesCases/ExercisesAudio-visuals- Slide-tape- Video- Web-based
Goals/Objectives
Skill developmentBehaviorManagement
Proper Implementation
DynamicsEnvironmentStyle/PedogogyFitFacilitation
Knowledge enhancementt
n s
STEP 3: Module Construction
The third step in the process of developing a training event is the construction of thetraining module. As is the case for the entire process, a goal/objective must be setfor each module. Depending on the target audience that has been chosen, the goalsmay be general knowledge, awareness, skill development, attitudinal change, or bettermanagement. Once the goal has been established, then it may be attained with theproper combination of materials and implementation strategies.
Considerations for proper materials depend on the goal and the target group of thetraining. Thus print materials, lectures, cases, practicum, and audio-visuals areselected to suit the situation. Likewise the implementation mode, i.e., dynamics,delivery style, language, and techniques such as brainstorming and exercise, dependsupon the goal and materials. This process is illustrated in the following diagram.
Figure 2. Module construction
STEP 4: Testing and Validation
All materials should be tested prior to being put into their final form. There are anumber of ways to accomplish this objective:
1. Peer Review. This method can take various forms, such as a review or critiqueby professional peers. This may be formal as in a publication review procedure,or informal such as a working group review, brainstorming, and discussion.
2. Expert Consultation. An expert can be hired to review the materials. This personmust be knowledgeable about both the content and the process of the trainingbeing prepared.
IRRI Training Guidelines 6
3. Field testing of the materials in a workshop setting. Materials may be introducedinto a workshop setting, and then evaluated by the participants.
4. Review by either a project manager or regional specialist who may have fieldexperience or an intuitive feeling for a match with their local experience andpersonal needs assessment orientation.
STEP 5: Redesigning
The redesign phase is an assessment of the original analysis -- the “what is” of thenature of the given knowledge base according to our original analysis of needscompared to the design phase and “what ought to be” the body of knowledge.
The redesign itself focuses on the intervention, that is, whether or not what hasbeen designed and field tested enables the participant to move from the “what is”(i.e., existing knowledge) to the “what ought to be” (i.e., the required knowledge,skills, and attitudes). In other words, has what you have designed moved theparticipant from where s/he was to where s/he wants to be. Analysis of theinformation collected in the testing and validation phase will guide the redesign.
STEP 6: Production of Materials
For the production of materials, a number of items need to be considered.
Comprehension -- the materials should cover all aspects of the topic that is to bepresented. For short workshops, when all areas cannot be presented, counterpoints,different methodologies, and theory papers should be included in the training packageas informational, supplementary, and background reading.
Coverage -- each component must be carefully defined and broken down into itssmallest element so that there is a perspective developed and it is understood howpieces of the puzzle fit together.
Format -- it is preferable to have a notebook or a binder for a workshop. Miscellaneouspapers and sheets of paper will be lost, stacked on a shelf, or mixed with otheritems.
IRRI Training Guidelines 7
Accessibility -- a file must be accessible, understandable, and easy to reference. Thus,it should be tabbed and have an index to the context.
STEP 7: Delivery
The professional delivery of the event is essential to its success. When most peoplethink of a workshop and delivery, they automatically begin to think about preparationand presentation of lectures. While it is true that the proper preparation and practiceare essential to the success of a training event, presentation skills are not the onlyelement that needs to be considered. Other elements enable the success by allowingthe participants to actively learn, and thus make the information delivered in thetraining event part of their everyday work life. It must be remembered that trainersteach people not materials, and therefore the participant and the process by whichthey are enabled to learn is the most important aspect of training. Major componentsto consider in actualizing the trainee are:
- Creation of the learning environment
- Facilitation techniques
- Presentation skills
If any of these elements are missing, the training event and desired transfer ofinformation will be marginalized.
IRRI Training Guidelines 8
Figure 3. Delivery elements
Delivery Element 1: The creation of a learning environment
1. The interactive nature of the event/ownership
Participation may be enhanced with interactive techniques such as exercises,cases, problem solving, group and individual tasks. The trainer will have todecide, based on the technical nature of the materials and the level of thetrainees, how much of the event time will be dedicated to lecture, interactiveexercise, and hands-on participation. But it must not be forgotten that active,participatory process, self-directed, experiential learning is the most efficientand effective type of learning. Active participation will create a vested interestor ownership within the participant, thus enhancing the potential for successof the training.
In addition to the event specific techniques, there are generic meeting rolesthat also enhance the dynamics of the events. These are:
Chair/Convenor: The group leader who sets the meeting agenda, decides whenand how things will be done. Duties include keeping the group on task and on
PresentationLearning
Environment
Facilitation
PreparationExecution
- organization- - - -
- Interaction- Configuration- Ownership
Skills
group memoryproblem analysisconflict managementevaluation
nn
st
y
IRRI Training Guidelines 9
time, proposing and eliciting ideas from the group, creating new ideas whennecessary, and ensuring that the group members have access to informationgenerated.
Facilitator: The neutral role of making the group process flow as efficiently aspossible by providing just enough structure so that the group dynamics don’tinterfere with the agenda. S/he should not be a key participant in the contentdiscussion on agenda issues, but monitors and prompts the discussion onagenda issues, ensuring that some conclusion or consensus is reached oneach issue and making sure that all group members feel included in the process.
The Timekeeper: Pays attention to the passage of time and notifies the groupwhen to move on to the next task.
The Vibes master: S/he is given power by the members to moderate (lessen theintensity of) dysfunctional behavior within the group. S/he steers the flow ofemotion away from dangerous waters. When inappropriate behavior or seriousattacks surface, it is the vibes master who is responsible for cutting it off bycalling attention to it. Since it is the “role” that calls attention to the behaviorrather than a member of the group, the notice is depersonalized and decisioncan be accepted more readily.
Rapporteur: This is potentially the most powerful position in the group sincethe person in this role controls information (and therefore decisions madeupon it) by recording and interpreting what takes place in the group. Duties ofthe rapporteur are to produce a written summary of the conclusions, actions,and recommendations of the group and to report to the plenary sessions whenspecified.
2. The setting
The setting and room configuration are important factors in the successfulcreation of the learning environment. With respect to the room configuration,there are a number of possible designs depending on what is to be accomplished.Some of the standard configurations are as follows:
Small group: Semi-circle, no table vs. semi-circle with table. Generally peoplewill talk to those they are facing, so if you want two people to engage each
IRRI Training Guidelines 10
other in a conversation, have them face one another. In a small group, it isdifficult to identify the leadership position if there is no table or if there is around table. If a rectangular table is placed in the same group, then theleadership position will be associated with the end of the table. In either ofthese 2 configurations, participants will associate the end that has audio-visual aids as the business end of the design and those wishing to be activelyinvolved will gravitate to that end.
With respect to classroom/training room designs, there are some fairly commonones that are effective depending on what the trainer wants to accomplish.(Training: Special Report: Off-site Training Meetings/November, 1990). Severalare as follow:
Traditional Classroom: The classic arrangement for dissemination of informationis the classroom. This arrangement is appropriate for information downloadin the form of lecture and computer-based training. This is very effective forcontrol of the class and information, but allows for little interaction with thetrainer or fellow participants. There may also be a psychological barrier createdas it may remind participants of power relationships from school days.
Conference Style/Rectangle: The conference style configuration enables groupdiscussions with relatively limited presentation input. Lecturing is disenabledas people will have to move chairs or sit sideways, thereby losing their writing
IRRI Training Guidelines 11
surface. For effective small group discussions, the optimum number is 6-8, upto 12, but the larger the group the less effective the discussion.
Horseshoe: The horseshoe configuration is suited to small group interactionand at the same time allows for presentation. It is less formal, allows thetrainer to move around, and is suited for cross-table and side-to-sidediscussions. Control remains high in this configuration and sight andparticipation is medium.
Fan-type/Half-round/Chevron: The final common configuration is the Fan type/Half-round/Half-Chevron. These configurations have 4 primary benefits:
All participants have a clear view of the trainer, visual aids, and the participants.It is easy to switch from listening, to working, to discussions. Cross-roomdiscussion is facilitated as there is little visual obstruction. Trainers can roamto assist individuals and groups working on exercises.
IRRI Training Guidelines 12
A summary of the various designs and their effectiveness follows:
Room Configuration Effectiveness
Semi-circle Classroom Rectangle Horseshoe Half-round
Control Med High Med High Med
Sightlines High Low Med Med High
Participation High Low Low Med High
(Adapted from Training: Special Report Nov. 1990)
Delivery Element 2: The selection and the proper useof facilitation skills
Facilitation quite simply is ”… enabling groups to succeed. More specifically,facilitation is the design and management of structures and processes that help agroup do its work and minimize the common problems people have working together.”(Justice and Jamieson, 1999:5). According to these authors/practitioners, facilitationis a neutral process that focuses on accomplishment involvement, flow,communication, participation, energy, and environment. Those of particular interestfor the IRRI trainer are described here:
1. Organization
In the delivery of a training event, the primary considerations for organizationare to ensure that the best approach/fit has been selected for the needs of theparticipants and that the design of the event will enable the group to succeed.Depending on the needs and structure, the event and activities may take oneor several forms. Examples are: workshop, conference, seminar, colloquium,and/or consultation.
IRRI Training Guidelines 13
2. Creating and using group memory
Capturing the record of the group is an essential task for success of an event.This capturing of information serves as a record for the event, keeps theindividuals and group on track, and encourages participation. Having a recordkeeper or rapporteur creates the story that is told later.
3. Building the database of ideas
Building a database of ideas for training events may come from multiple sources.One source is the participants themselves and there are a number of techniquesto get the ideas out on the table for discussion. Some of these are: (Summarizedand/or adapted from Justice and Jamieson, 1999):
Leading Questions: One of the necessary tasks of a facilitator is to get everyonein the training event involved and learning. One of the facilitation skills is toask questions that lead participants to develop solutions of their own. Thisrequires the facilitator to seek out ideas by asking open-ended questions,encouraging discussions and dialogue, being an active listener, playing backinformation, openly accepting ideas, and recognizing the audience feeling/acceptance. (Training and Development Journal, 1991).
One type of leading question is the “problem probe”. The purpose of a problemprobe is to identify and agree upon the problem so that the solution constructedis the correct one for the problem. In order to understand the dimensions,boundaries, and nature of the problem, the group is led through a series ofcues or question that help to clarify and define the problem. There are multiplesteps to the method:
Step 1: Each participant writes what s/he thinks the problem is. Participantsare led through a series of probes such as: what happens; wheredoes the problem occur; when does the problem occur; how longhas it been going on; what resources are involved, etc. (The probescan be adapted to the particular problem based on the knowledgeand experience of the group and trainer.)
Step 2: Prepare flip charts with each question on the top. Lead group throughquestions and record data without comment or disagreement. Gothrough the entire series again.
IRRI Training Guidelines 14
Step 3: Discussion
Step 4: Group forms problem statement ending with “The primary cause(s)of the problem appear(s) to be… ”
Step 5: Check/Feedback
A second tried and true method of soliciting ideas is brainstorming. The purposeof brainstorming is to create a quantity of ideas. While a great number ofarticles and how-to advice has been written about brainstorming, thefundamentals have changed little since Brainstorming, by Charles Clark, 1953.His simple rules were:
Criticism is ruled out.
The wilder the ideas, the better.
Quantity is wanted.
Combination and improvement are sought.
These rules have been refined to include the advice that each idea should berecorded verbatim to ensure that the idea is not altered or interpreted by thefacilitator. There are a number of options that may be used such as: suspendingassumptions, segmenting the process, round robin, anonymous alternatives,being someone else or changing hats (De Bono), idea derby, and guided imagery.
4. Conflict management
While conflict management/resolution is a primary management tool, it isalso a facilitator skill. The first step in conflict management is to recognizethat conflict exists and is part of human communicative behavior. Once it isrecognized that it exists, it should not be repressed. Rather it should bechanneled into positive behavior. When seen as a normal social phenomenon,conflict is positively functional in the following ways:
- It can prevent stagnation.
- It can stimulate interest and curiosity.
IRRI Training Guidelines 15
- It can be the medium through which problems can arise and be discussed.
- It can be the root of personal and organizational change.
- It can foster cohesiveness.
According to Justice and Jamieson, there are several simple strategies torecognize and manage conflict in a functional manner:
Pinpoint conflict
Identify conflict
Identify resolution options
Point out positive aspects
Differentiate position from interest
Link 2 positions with “and” to determine compatibility/incompatibility
Point out negative consequences of failure to resolve
Surface value of differences
Evaluate (Workshop, Follow-up, Impact) See Step 8
Delivery Element 3: Presentation
While we often hear that some speakers are naturally gifted or that a competentprofessional is a good speaker, neither of these statements is completely true.Communication requires not only a message, but also a skilled sender and a perceptive/engaged receiver.
The skills necessary to accomplish this are acquired through practice and hard work.A separate handbook of guidelines for presentation skills is currently being producedby the Training Center. The following highlights some of the materials from train-the-trainer workshops.
IRRI Training Guidelines 16
There are two phases to presentation: preparation and execution. In the preparationphase, a number of items need to be considered:
Audience -- Make sure that materials are appropriate for the target audience.
Relevance -- Continually question relevance of material, level and complexity, necessity,importance, etc.
Clarity -- Ask: Are statements clear, precise, understandable?
Simplicity -- Start with simplicity rather than complexity. Materials can always be mademore complex but audiences will be lost if material is too complex for them at thebeginning.
Planning/Preparation -- Speaking, teaching, presenting, are acquired skills. It is essentialthat the presentation be thoroughly planned, prepared, and practiced. Know exactlywhat you are going to say, when to say it, and why. PRACTICE, PRACTICE, PRACTICE.
If phase 1 (the preparation phase) is carried out properly, phase 2, the executionphase, will take care of itself.
Two areas should be considered:
Interest - keep the audience interested by voice modulation, mannerisms, appropriatevisual aids that speak for themselves, and eye contact.
Sensitivity - clarify if the audience does not understand, answer questions, be flexibleand good humored.
1. Personal Effectiveness
For personal effectiveness, there are several items to consider:
Don’t distance yourself -- beware of physical and psychological barriers.
Make the audience comfortable.
Maintain coherence -- be simple, structured, concrete, and visual.
Maintain connection -- clarify, change direction as needed.
IRRI Training Guidelines 17
2. Visual Aids
If they are properly done, visual aids will assist the education process. It isestimated by adult learning specialists that 40% more of the materials will beretained if proper visual aids are used in the training process. If they areimproper, they are distracting, confusing, and disruptive. Some key things tonote about the content of visual aids:
Simplify -- one topic, key words, phrases
Clarity -- visible, no more than 2 types of lettering: can be multiple colors butyellow, brown, and purple are not good; provide a key if necessary; highlightsignificant figure, words or numbers;
Visibility -- use symbols, diagrams or key words -- no text.
With respect to the use of visual aids, the following are some do’s and don’ts.
Do’s Don’ts
Remove them when they are not needed Speak to the machine
Stand to the side Stand in front of the light oraudience,
Let them speak for themselves Speak while you are changing visualaids
Point to the screen, not the projector
Have one key message per visual aid
Distribute handouts at the end
Questions
The following is a list of tips on how to handle questions: be alert, listen carefully,ask for clarification; don’t let yourself be interrupted; correct false options ormisinformation; speak to one of your friends; avoid “um” and “em”, don’t let yourself
IRRI Training Guidelines 18
Event/Intervention
Training Event Evaluation
Assess Satisfaction of Trainees
Event AssessmentQuestionaire
Completion of Training
Personal Acquisition
Skills/KnowledgeAcquisition
Assess Change inKnowledge, Skills,Attitudes
Pre-Test/Post Test
Completion of Training
Follow-up
Skill/KnowledgeTransfer
Assess Extentof Applicationof Skill/Knowledgeto Job Related Activities
Survey: Interviewand/or Questionaire
3 months to 6 months
Impact
Organizational Performance Change
Assess OrganizationalChange as a Resultof Skill/KnowledgeTransfer and Incorporation
Baseline Comparison
1-3 years
be taken out of your topic; know your subject but admit it if you don’t havean answer; stay calm; pay attention to messages from the person asking thequestion--overt or hidden.
STEP 8: Evaluation
Since evaluation is the final step in the process and confirms (or denies) your approach,the following section is covered in more detail.
“Millions for training, but not one cent for evaluation. By design or by happenstance,this is an all-too-common occurrence. In many instances it is assumed that trainingprograms have been effective because participants enjoyed the presentations” (Cascioand Awad, 1981:307).
In our recent past, it was said that our clients were smiling, and therefore we aredoing an excellent job. That, of course, was said in jest; however, there are alarminggrains of truth in that, in the past, training events have not been documented in astandard fashion, assessment often has been oriented towards acceptance orappreciation of the event, and little follow-up has been done to date. Given thattraining is one of IRRI’s primary programs, and that it is one of the vehicles orconduits for working group and research materials, it is essential that we instituteand implement a continuous self-evaluation of our program.
To accomplish this task, a conceptual model that measures training outcomes bylevels, is proposed.
Figure 4. Evaluation type/sequence
IRRI Training Guidelines 19
The conceptual model for evaluating training begins with the need to improveorganizational performance and then sequentially progresses through the provisionof training (level 1), the completion of training (level 2), the resumption of job-related activities (level 3), and the changed organizational performance (level 4).
Improved Organization Performance
The need to improve organizational performance is a given with many organizations.As discussed in “Step 1: Needs Analysis,” the type of improvement required and thespecific intervention prepared is based on a training diagnosis or ex-ante evaluationperformed to identify training gaps or needs for information in the organization.This diagnosis or identification of gaps or needs comes from a variety of sources andinteractions with NARS and their partners, including extension organizations andfarmers.
In our analysis, assessment, and prioritization of training needs we must make thatour planned events are consistent with our strategy, in line with the levels of ourmaterials, planned for appropriately targeted audiences, and within the capability,time, and professional constraints of our staff.
Provision of Training (Level 1)
The evaluation of level 1 focuses on the training event itself as it assesses thesatisfaction of trainees with the trainers, the facilities, and the content of the program.These are simply reaction criteria that measure the trainees’ impressions and/orfeelings about the program. This is commonly accomplished with an evaluation formadministered at the end of a training event.
While we have used a variety of formats for this in the past, we have been workingon a standard data collection format so that the information collected on events iscomprehensive and comparable (See Attachment II)
Data collection is a necessary step in assessing the success of the event and, evenmore important, it is absolutely essential ultimately for the training programevaluation. The recommended reporting format for the level 1 is as follows:
IRRI Training Guidelines 20
Part 1: Review of the workshop
A. Background
B. General Assessment
C. Preparation
D. Organization
E. Content
i. Agenda
ii. Materials
iii. Presentations
iv. Discussions
v. Synthesis
vi. Report
F. Participants
G. Facilities
H. Interpretation
I. Administrative support
J. Lodging/Entertainment/Transport
K. Resources
Part 2. Summary of Participants’ Evaluation (Recommended Format Attached)
Part 3. Conclusions/Recommendations
Part 4. Annexes
IRRI Training Guidelines 21
Completion of Training (Level 2)
Level 2 evaluation, done at completion of training, evaluates the output of training interms of increased knowledge, change skills, and attitudes of trainees. This refers to thelearning criteria, which are more rigorous measures of training outputs such as examsor performance tests. The assessment of this level will not necessarily be required orappropriate for all the events. However, for certain types of targeted skill training, pre-tests and post-tests on the skills offered in the courses would provide an assessment ofchanges that occurred as a result of the course. Since our courses are normally ofshort duration, one to two weeks, factors such as history, maturation, testing, andinstrumentation which often contaminate measurement would not be relevant. Forthose events which level two evaluations are appropriate, we should begin to developpre-event or pre-test questionnaires for use as soon as possible.
Resumption of Job-Related Activities (Level 3)
Level 3 evaluation, done at resumption of job-related activities, evaluates the applicationof the newly acquired skills, knowledge, and attitudes as they are related to job activities.These are behavioral criteria that indicate a positive transfer from training to the jobhas occurred. This is an ex-post evaluation that should be conducted three to sixmonths after the training event. This evaluation requires a systematic observation ofbehavior that should be accomplished by a supervisor and supplemented by self-reports. Reports on behavioral changes should be submitted to the trainers, the trainingunits, appropriate National Agricultural Research and Extension Systems (NARES)leaders, and funding and coordinating organizations.
Changed Organizational Performance (Level 4)
Level 4 evaluation, that of changed organizational performance, evaluates change inthe organization that is the result of the training received. While the extent towhich training has improved organizational performance is the ultimate measure oftraining success, it is also the most difficult to access because it is difficult to attributechange. It is essential at the outset that the expected result is clearly defined, and atangible product is possible, so that empirical evidence collected can be measuredagainst some anticipated or expected outcome.
IRRI Training Guidelines 22
BIBLIOGRAPHYLuka A, Marcotte P. 1989. Proceedings of the SACCAR/ISNAR regional workshop on
human resource management in National Agricultural Research Systems. TheHague: ISNAR.
Bailey K D. 1994. Methods of Social Research, The Free Press.
Cascio W F, Awad. E M. 1981. Human resources management. Reston, Virginia:Reston Publishing Company Inc.
Casse P, Declerck R, Deol S, Rena O.1984. Management training: Guidelines fortrainers. EDI Training Materials 670/022 International Bank for Reconstructionand Development.
Charles C. 1953. Brainstorming.
ISNAR. 1987. Communications techniques trainers manual. The Hague: ISNAR.
Justice T, Jamieson DW. 1999. The facilitator’s fieldbook, HRD Press.
Kerrigan J E, Luke JS. 1987. Management training strategies for developingcountries. Boulder, Colorado: Lynne Rennier Publishers.
Samuel P. 1983. Training for public administration and management in developingcountries: A review. World Bank Staff Working Paper Number 584. WashingtonD.C.: The World Bank.
Pretty J N, Guijt I, Scoones I, Thompson J. 1995. A trainer’s guide for participatoryleaming and action, IIED.
Sachdeva P. 1989. Implementing ISNAR’s training strategy and program. TheHague: ISNAR.
Stanley L A. 1987. Guide to training needs assessment. Ljubljana, Yugoslavia:International Center for Public Enterprises in Developing Countries.
Training: Special report, November 1990.
Training and development journal, 1991.
IRRI Training Guidelines 23
Section 2: Attachment 1. Prospectus/Agenda Sample
Problem-based Technology Generation for Rice Environments
(an on-the-job training for Indonesian scientists)Training Center, IRRI, Los Banos, Laguna, Philippines
21 February-14 March 2000
COURSE DESCRIPTION
INTRODUCTION
Despite technology advances in agriculture in the last three decades, productivity amongthe greater proportion of the world’s farmers have remained low. It is for this reasonthat equity issues have been raised against agricultural technology development andtransfer; the benefits of technology advances do seem to exclude resource-poor farmers,and attempts at correcting this inequitable situation have met with limited success.Forerunners of a recent trend in agricultural development advocate the need to considerthe farmer’s problems and conditions even at research planning stage, especially theproblems of the majority who farm in resource-poor areas where resource-intensivetechnologies have failed to make a beneficial impact.
Through collaborative research and human resource development efforts, NationalAgricultural Research Systems (NARS) have expanded and strengthened their researchbase to systematically tackle emerging problems in the various rice ecosystems.Nevertheless, there is a need for rice scientists in the NARS to recognize the continuousprocess from problem identification to research planning and implementation to researchresults utilization, and acquire the skills to get involved actively in that process.
This 1-month on-the-job training course intends to develop such understanding andskills in Indonesian rice scientists and R & D professionals to enable them to designresearch that responds directly to farmers’ field problems. It will help them improvetheir skills in problem identification/analysis, research planning, and researchinformation/technology utilization, and enhance their understanding of their role inmaking agricultural technology useful for the greater proportion of Indonesian farmers,including those in the marginal areas.
IRRI Training Guideline s 24
OBJECTIVES
At the end of the course, participants will have enhanced their capability to
• understand the problem identification-research-technology/information utilizationprocess;
• identify and analyze the effects of bio-physical and socioeconomic factors onproduction systems and production problems;
• use various diagnostic tools to identify key production constraints;
• translate diagnostic results to strategic research problems and prioritize theseproblems; and
• design and evaluate technologies to address key production constraints.
METHODOLOGY
Mini-lecturesBrainstorming sessionsGroup discussion/Sharing of experiencesHands-on activitiesField practicumGroup projectPresentation and critique
COURSE REQUIREMENT
Participants are expected to use available data and information on rice production inthe area to be studied that may be useful in cross-checking their own findings. Thesecould be existing base maps, climate data (on rainfall, temperature, wind velocity, etc.),soils data, current crop calendar, and current agronomic data (on yield, seeding rate,crop establishment practices, etc.) documented in reports about the locale.
IRRI Training Guideline s 25
PROGRAM SCHEDULE
Venue: Breakup Room Time: 8:30 a.m.
Week 1
Day 1
Opening Session
• Welcome remarks and P L Marcotte (Paul)introduction of training team Head, Training Center
• Course overview and M B Quiamco (Madz)leveling of expectations Senior Associate Scientist,
Training Center
COFFEE BREAK
• Filling the world’s rice bowl (a video presentation)
• Tour of TC facilities and introduction to TC staff
Session II: Methods, Tools, and Techniquesfor Problem Diagnosis and Prioritization
• Problem-based research – an overview
• Activity 1: Brainstorming sessionWho are the Indonesian farmers?
Day 2
• Debriefing on Activity 1
• Agricultural development approaches- Evolution of approaches- Farmer participatory approach
IRRI Training Guideline s 26
• Concepts and approaches to problem diagnosis
• Participatory methods and skillsA. Agroeconomic characterization
− Observation skills− Inventory skills− Mapping skills− Measurement skills
• Activity 2: Developing observation, inventory,mapping, and measurement skills
Day 3
B. Problem identification and diagnosis− Selecting key informants− Formulating guide/key questions− Using different types of questions− Facilitating a brainstorming session− Group management− Listening− Appreciating farmers’ opinion− Handling neutrality− Using appropriate language− Probing techniques
• Activity 3: Preparing for and conducting aninterview
Day 4
• Visits to− the Philippine Council for Agriculture and Natural
Resources Research and Development (PCARRD) BPI Economic Garden, Los Banos, Laguna
− Masajo Farm
IRRI Training Guideline s 27
Day 5
• Activity 3 continued
• (3 p.m.) Presentation of results to training team
Week 2
Days 1-3
• An overview of the problem V P Singh (Pal)identification-research planning Coordinator,technology/information South Asiadissemination continuum Project and Systems
Agronomist, IRRI
• Participatory problem identificationand diagnosis: the process
• Diagnosis and design interface: Translatingfield problems into research questions
• Activity 4: Group exercises in problemidentification and research question formulation
Days 4-5
Session III: Methods, Tools, and Techniques forExperimentation and Technology Evaluation
• Types of experiments and their features(on-station, on-farm, participatory experiments)
• Experimentation and evaluation- Evaluation of long-term trials- Criteria for assessing technology
• technical feasibility
IRRI Training Guideline s 28
• biological feasibility• socioeconomic feasibility• sustainability• criteria fo farmers’ evaluation
Week 3
Day 1
• PRA (participatory resource appraisal) tools M B Quiamcoand outputs- maps- resource flows- charts and calendars- diagrams- timelines and trends
• Planning, briefing on the fieldpracticum
Day 2
• Travel to Nueva Ecija
Days 3-5
• Activity 5: Field practicum— Participatory Training Teamagro-ecosystems characterization
1. Agroeconomic characterization andproblem identification with farmers
2. Brainstorming session and group discussion:Translating diagnostic data to strategicresearch problems
3. Feasibility assessment and evaluation4. Preparation of visual presentation tools for
verification meeting with farmers5. Verification meeting6. Revision of visuals to incorporate farmers’ inputs
IRRI Training Guideline s 29
Days 6-7 (Weekend trip to Pangasinanor Baguio City)
Week 4
Day 1
• Travel back to Los Banos
Days 2-4
Session IV: Beyond Problem Diagnosis:Problem-based Research Planning
• Techniques and guidelines for V P Singhresearch planning
• Activity 6: Formulating a researchplan for practicum site
Day 5
• Presentation of research plan
• Debriefing with training team
• Closing program
IRRI Training Guidelines 31
Course TitleDate
Venue
Please list what you consider to be 3 strengths of the course:
Please list what you consider to be weaknesses of the course:
Your cooperation in completing this questionnaire will be greatly appreciated. The informationyou provide will be useful in planning future events and will help resource persons and courseorganizers to improve their materials and presentations
In general, I would rate the course as:
On balance, would you say that the course objectives were achieved?
ExcellentGoodAverageFairPoor
YesPartiallyNo
A.General
Reactions
1.
2.
3.
1.
2.
3.
Section 2: Attachment 2. Event Evaluation Form
IRRI Training Guidelines 32
In th
is se
ction
we w
ould
like f
or yo
u to
rate
the c
onte
nt, t
he u
sefu
lness
, the
pres
enta
tion,
and
the t
ime a
llotm
ent
for ea
ch o
f the
topic
s inc
luded
in th
e cou
rse. W
hen
ratin
g , y
ou sh
ould
cons
ider s
uch
facto
rs as
rigo
r of
mat
erial
(theo
ry, so
undn
ess,
met
hodo
logy)
. In
rega
rd to
, r
ate t
he to
pic i
n te
rms o
f its a
pplic
abilit
y/rel
evan
ce in
your
day-
to-d
ay w
ork a
ctivi
ties.
Fact
ors t
o co
nside
r in
asse
ssing
in
clude
clea
rnes
s, log
ical s
truct
ure,
good
use
of v
isuals
, etc
. Plea
se pl
ace a
chec
k in
the b
ox th
at m
ost a
ccur
ately
repr
esen
ts yo
ur
opini
ons o
f the
se fa
ctors.
cont
ent
usef
ulne
sspr
esen
tatio
n
B.To
pics
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
11.
12.
13.
Exce
l- len
t
Good
Ave
r- Fa
ir Po
or
age
Cont
ent
U
sefu
lnes
s to
Wor
k
Pr
esen
tatio
n
T
ime
Allo
tmen
t
Exce
l- len
t
Good
Ave
r- Fa
ir Po
or
age
Exce
l- len
t
Good
Ave
r- Fa
ir Po
or
age
Too
Ju
st T
ooSh
ort R
ight L
ong
IRRI Training Guidelines 33
Very Good Good Fair Poor
Please use the space below to write down any additional comments or suggestions you might have.
Accommodation.........................................Meals..........................................................Stipend.......................................................Lectures......................................................Discussions................................................Papers/Handouts........................................Organization and Management...................Quality of visual aids..................................Quantity of visual aids................................
C.Features
F.Comments
D.What additional topics would you
have likedincluding in this
training activity?
C.On what othermanagement
topics would youlike to receive
training infuture events?