Chapter Eight: Chapter Eight: EvaluationEvaluation

OverviewOverview

The Purpose of evaluationThe Purpose of evaluation

ObjectivesObjectives

Status of evaluation and measurementStatus of evaluation and measurement

Measurement Measurement ProductionProduction ExposureExposure AwarenessAwareness AttitudesAttitudes ActionAction Supplemental activitiesSupplemental activities

The Purpose of EvaluationThe Purpose of Evaluation

The fourth step of the The fourth step of the public relations public relations process is evaluation – process is evaluation – the measurement of the measurement of results against established results against established objectives (194).objectives (194).

ContinuedContinued--- ---

_ Evaluation is the measurement of results _ Evaluation is the measurement of results against objectives , this can enhance against objectives , this can enhance future performance and also establish future performance and also establish whether the goals of management by whether the goals of management by objective have been met .objective have been met .

_ Evaluation is well described as ( the _ Evaluation is well described as ( the systematic assessment of a program and systematic assessment of a program and it’s result , it is a mean for practitioners to it’s result , it is a mean for practitioners to offer accountability to Clients and to offer accountability to Clients and to themselves .themselves .

Objectives:Objectives:A Prerequisite for EvaluationA Prerequisite for Evaluation

• Before any public relations program can Before any public relations program can be properly evaluated , it’s important to be properly evaluated , it’s important to have a clearly established set of have a clearly established set of measurable objectives , these should be measurable objectives , these should be part of the program plan but first some part of the program plan but first some points need to be reviewed .points need to be reviewed .

1. Public relations personnel and 1. Public relations personnel and management should agree on the criteria management should agree on the criteria that will be used to evaluate success in that will be used to evaluate success in attaining objectives .attaining objectives .

ContinuedContinued… …

2. Don’t wait until the end of public relations 2. Don’t wait until the end of public relations programs to determine how it will be programs to determine how it will be evaluated . ( Evaluating impact starts in the evaluated . ( Evaluating impact starts in the planning stage , you break down the planning stage , you break down the problem into measurable goals and problem into measurable goals and objectives then after implementing the objectives then after implementing the program , you measure the results against program , you measure the results against goals )goals )

_ If the objective is informational , _ If the objective is informational , measurement techniques must show how measurement techniques must show how successfully information was successfully information was communicated to target audiences .communicated to target audiences .

ContinuedContinued……

_ _Motivational objectives are more difficult Motivational objectives are more difficult to accomplish , if the objective is to to accomplish , if the objective is to increase sales or market share , it’s increase sales or market share , it’s important to show that public relations important to show that public relations efforts caused the increase rather than efforts caused the increase rather than advertising or other marketing strategies advertising or other marketing strategies , But if the objective is to change , But if the objective is to change attitude or opinion , research should be attitude or opinion , research should be done Before and after public relations done Before and after public relations activity to measure the percentage of activity to measure the percentage of ChangeChange

Objectives:Objectives:A Prerequisite for EvaluationA Prerequisite for Evaluation

Evaluation QuestionsEvaluation Questions

Adequately planned?Adequately planned? Message understood?Message understood? How more effective?How more effective? All primary and secondary audiences reached?All primary and secondary audiences reached? Organizational objective achieved?Organizational objective achieved? Unforeseen circumstances?Unforeseen circumstances? Within budget?Within budget? Steps to improve?Steps to improve?

Current Status of Measurement Current Status of Measurement and Evaluationand Evaluation

_ _The proportion of public relations Budgets The proportion of public relations Budgets devoted to measurement and evaluation devoted to measurement and evaluation grew over the 1990s to about 5 percent . On grew over the 1990s to about 5 percent . On the most basic level , practitioners can the most basic level , practitioners can measure message distribution and media measure message distribution and media placements . The second level would be placements . The second level would be measurement of audience awareness , measurement of audience awareness , comprehension and Retention . The most comprehension and Retention . The most advanced level is the measurement of advanced level is the measurement of change in attitudes , opinions and behaviorschange in attitudes , opinions and behaviors

Current Status of Measurement Current Status of Measurement and Evaluationand Evaluation

Sophisticated TechniquesSophisticated Techniques

Computerization Computerization

Clips as evaluation supplanted by many Clips as evaluation supplanted by many methodsmethods

Management demands more Management demands more accountability to the bottom lineaccountability to the bottom line

Measurement:Measurement:ProductionProduction

_ One elementary form of evaluation is simply to _ One elementary form of evaluation is simply to count how many news releases , feature stories , count how many news releases , feature stories , photos , letters and the like are produced in a photos , letters and the like are produced in a given period of time given period of time

_ This kind of evaluation is supposed to give _ This kind of evaluation is supposed to give management and idea of staff’s productivity and management and idea of staff’s productivity and output .output .

_ P.R professionals , don’t believe that this _ P.R professionals , don’t believe that this evaluation is very meaningful , because it evaluation is very meaningful , because it emphasizes quantity instead of quality . It may be emphasizes quantity instead of quality . It may be more cost effective to write fewer news releases more cost effective to write fewer news releases and spend more time on the few that are and spend more time on the few that are newsworthy .newsworthy .

Measurement:Measurement:ProductionProduction

_ Another side of production approach is to _ Another side of production approach is to specify what the public relations person should specify what the public relations person should accomplish in obtaining media coverage .accomplish in obtaining media coverage .

_such evaluation Criteria not only are unrealistic , _such evaluation Criteria not only are unrealistic , they are also impossible to Guarantee because they are also impossible to Guarantee because public Gatekeepers not the public relation make public Gatekeepers not the public relation make such decisions. Management may argue that such decisions. Management may argue that such placement goals provide incentives to such placement goals provide incentives to public relation staff and are tangible criteria in public relation staff and are tangible criteria in employee performance Evaluation employee performance Evaluation

Measurement:Measurement:ExposureExposure

The most widely practiced form of evaluating The most widely practiced form of evaluating public relation programs is the compilation of public relation programs is the compilation of press clippings and radio – television press clippings and radio – television mentions . Public relation firms and company mentions . Public relation firms and company departments working primarily on a local departments working primarily on a local basis often have a staff member scan and basis often have a staff member scan and clip the area newspaper . Large companies clip the area newspaper . Large companies with national and international outreach hire with national and international outreach hire clipping services to scan large numbers of clipping services to scan large numbers of publications , it’s also possible to have publications , it’s also possible to have electronic clipping services monitor .electronic clipping services monitor .

Measurement:Measurement:ExposureExposure

Measurement of exposure relies on:Measurement of exposure relies on:

Media impressionsMedia impressions

Hits on the internetHits on the internet

Advertising equivalency Advertising equivalency

Systematic trackingSystematic tracking

Requests for 800 numbersRequests for 800 numbers

Cost per person Cost per person

Audience attendanceAudience attendance

Media ImpressionsMedia Impressions

_ In addition to the number of media _ In addition to the number of media placements , P.R departments and firms placements , P.R departments and firms report how many people have been exposed report how many people have been exposed to the message, these numbers are to the message, these numbers are Described as Media impression , the Described as Media impression , the potential audience reached by a periodical potential audience reached by a periodical or a broadcast program .or a broadcast program .

_ National news story can generate millions of _ National news story can generate millions of impressions by simple multiplication of each impressions by simple multiplication of each placement by the audience of each medium placement by the audience of each medium

ContinuedContinued

_ Media impressions are commonly _ Media impressions are commonly used in advertising to document , used in advertising to document , this figure give a rough estimate of this figure give a rough estimate of how many people are exposed to a how many people are exposed to a message , they disclose how many message , they disclose how many people read or heard the stories people read or heard the stories and more important how many and more important how many absorbed or acted on the absorbed or acted on the information .information .

Hits on the InternetHits on the Internet

_ A cyberspace version of media _ A cyberspace version of media impression is the number of people impression is the number of people reached Via an organization’s world reached Via an organization’s world wide web site or home page . Each wide web site or home page . Each instance of a person accessing a site instance of a person accessing a site is called a is called a Hit Hit or a or a Visit Visit ..

Advertising EquivalencyAdvertising Equivalency

_ Another approach is to calculate the value of _ Another approach is to calculate the value of message exposure , this is done by converting message exposure , this is done by converting stories in the regular news columns or on the stories in the regular news columns or on the air into equivalent advertising Costs .air into equivalent advertising Costs .

_ some practitioners even take the approach of _ some practitioners even take the approach of calculating the cost of advertising for the calculating the cost of advertising for the same amount of space and then multiplying same amount of space and then multiplying that total three to six times to reflect a that total three to six times to reflect a number of research studies that show that a number of research studies that show that a new story has Greater Credibility than an new story has Greater Credibility than an advertisement .advertisement .

Systematic TrackingSystematic Tracking

As noted earlier , message exposure As noted earlier , message exposure traditionally has been measured By traditionally has been measured By Sheer Bulk , new advances in computer Sheer Bulk , new advances in computer software and database , now make it software and database , now make it possible to track media placements in a possible to track media placements in a more sophisticated way .more sophisticated way .

_ Computer databases can be to analyze _ Computer databases can be to analyze the content of media placements by the content of media placements by such variable as market penetration …such variable as market penetration …

Systematic TrackingSystematic Tracking

_ The value of systematic tracking is _ The value of systematic tracking is manifested in several ways one is manifested in several ways one is continuing regular feedback during a continuing regular feedback during a campaign to determine if an campaign to determine if an organization’s publicity efforts are organization’s publicity efforts are paying off in terms of placements and paying off in terms of placements and mention of key messages , tracking mention of key messages , tracking coverage and comparing it over a coverage and comparing it over a period of time is called period of time is called BenchmarkingBenchmarking . .

Requests and 800 numbersRequests and 800 numbers

_ Another measure of media exposure is _ Another measure of media exposure is to compile the number of requests for to compile the number of requests for more information , a story in a more information , a story in a newspaper or an appearance of a newspaper or an appearance of a company spokesperson on a broadcast company spokesperson on a broadcast often provides information as to where often provides information as to where people can get more information about people can get more information about a subject .a subject .

_ Requests for materials also show the _ Requests for materials also show the effectiveness of a public relation effectiveness of a public relation programprogram

Cost Per PersonCost Per Person

_ A way to evaluate to the message is to _ A way to evaluate to the message is to determine the cost of reaching each determine the cost of reaching each number of the audience , the technique is number of the audience , the technique is commonly used in advertising to place commonly used in advertising to place costs in perspective .costs in perspective .

_ Cost effectiveness , as this technique is _ Cost effectiveness , as this technique is known, also is used in public relations.known, also is used in public relations.

Cost-per-thousand (CPM) is calculated by Cost-per-thousand (CPM) is calculated by taking the cost of the publicity program taking the cost of the publicity program and dividing it by the total media and dividing it by the total media impressions (discussed earlier) .impressions (discussed earlier) .

Audience AttendanceAudience Attendance

_ Counting attendance at events is a relatively _ Counting attendance at events is a relatively simple way of evaluating the effectiveness of simple way of evaluating the effectiveness of pre-event publicity.pre-event publicity.

_ Poor attendance at a meeting or event can _ Poor attendance at a meeting or event can indicate inadequate publicity and promotion.indicate inadequate publicity and promotion.

_ Another major cause is lack of public interest, _ Another major cause is lack of public interest, even when people are aware that a meeting even when people are aware that a meeting or event is taking place.or event is taking place.

_ Low attendance usually results in _ Low attendance usually results in considerable finger-pointing, thus an considerable finger-pointing, thus an objective evaluation of exactly what objective evaluation of exactly what happened – or didn’t happen – is a good happened – or didn’t happen – is a good policy.policy.

Measurement of Audience Measurement of Audience AwarenessAwareness

_ _Techniques of measuring audience Techniques of measuring audience exposure and accurate dissemination exposure and accurate dissemination have been discussedhave been discussed..

_ _A higher level of evaluation is to A higher level of evaluation is to determine whether the audience determine whether the audience actually became aware of the actually became aware of the message and understood itmessage and understood it..

ContinuedContinued……

_ _The tools of survey research are needed to answer The tools of survey research are needed to answer such questionssuch questions..

_ _Members of the target audience must be asked Members of the target audience must be asked about the message and what they remember about the message and what they remember about itabout it..

_ _Public awareness of what organization sponsors Public awareness of what organization sponsors an event also is importantan event also is important..

_ _Another way of measuring audience awareness Another way of measuring audience awareness and comprehension is day-after recalland comprehension is day-after recall..

__Under this method, participants are asked to view Under this method, participants are asked to view a specific TV programme , the next day they are a specific TV programme , the next day they are interviewed to learn which message they interviewed to learn which message they rememberedremembered. .

Measurement:Measurement:AttitudesAttitudes

_ Closely related to audience _ Closely related to audience awareness and understanding of a awareness and understanding of a message are changes in audiences message are changes in audiences perception and attitudes . A major perception and attitudes . A major technique determine such changes is technique determine such changes is the baseline study .the baseline study .

Measurement:Measurement:AttitudesAttitudes

Measurements of audience Measurements of audience attitudes attitudes can be accomplished through:can be accomplished through:

Baseline StudiesBaseline Studies – a measurement of – a measurement of attitudes before, after, and during a attitudes before, after, and during a campaign. It’s also called benchmark campaign. It’s also called benchmark

studies studies

Measurement:Measurement:Audience ActionAudience Action

Measurement of audienceMeasurement of audienceactionaction is determined by a is determined by a campaign objectives … campaign objectives … ……where they met?where they met?

Measurement:Measurement:Supplemental ActivitiesSupplemental Activities

Measurement of supplemental activities Measurement of supplemental activities includes:includes:

– Communication auditCommunication audit

– Pilot tests and split messagesPilot tests and split messages

– Meeting and event attendanceMeeting and event attendance

– Newsletter readershipNewsletter readership

– Content analysisContent analysis

– Readership interest surveysReadership interest surveys

– Article recallArticle recall

– Advisory boardsAdvisory boards

Communication auditsCommunication audits

_ communication activity of an _ communication activity of an organization should be evaluated at organization should be evaluated at least once a year to make sure that least once a year to make sure that every primary and secondary public every primary and secondary public is receiving appropriate messagesis receiving appropriate messages

_ A communication audit could include _ A communication audit could include the following :the following :

ContinuedContinued--- ---

1.1. Analysis of all the communication Analysis of all the communication activities activities

2.2. Informal interviews with rank and Informal interviews with rank and file employees file employees

3.3. Informal interview with community Informal interview with community leaders .leaders .

Pilot tests and split Pilot tests and split messagesmessages

_ A variation of Pre-testing is the pilot _ A variation of Pre-testing is the pilot test , before going national with the test , before going national with the P.R messages , companies often test P.R messages , companies often test the message and copy points in the message and copy points in selected cities to learn how the selected cities to learn how the media accept the message and how media accept the message and how the public reacts .the public reacts .

_ The split message approach is _ The split message approach is common in direct mail campaigns common in direct mail campaigns

Pilot tests and split Pilot tests and split messagesmessages

_ With perception analyzer systems , _ With perception analyzer systems , some times called some times called theater systemtheater system , , selected subjects are asked to view selected subjects are asked to view speeches .speeches .

_ The think- aloud method can be used _ The think- aloud method can be used to asses with content in a similar to asses with content in a similar fashion to the perception analyzers .fashion to the perception analyzers .

Meeting and event Meeting and event attendanceattendance

_ P.R practitioners use a number of _ P.R practitioners use a number of information methods to evaluate information methods to evaluate success of the meeting , but they success of the meeting , but they also employ more systematic also employ more systematic methods . The most common methods . The most common technique is an evaluation sheet that technique is an evaluation sheet that participants fill out at the end of the participants fill out at the end of the meeting .meeting .

Newsletter readershipNewsletter readership

_ Editors of newsletters should evaluate _ Editors of newsletters should evaluate readership annually , such an readership annually , such an evaluation can help ascertain 1. reader evaluation can help ascertain 1. reader perception 2.the degree to which perception 2.the degree to which stories balanced 3. the kinds of stories stories balanced 3. the kinds of stories 4. topics that should be covered 5. the 4. topics that should be covered 5. the credibility of the publication 6. the credibility of the publication 6. the extent to which the newsletters is extent to which the newsletters is meeting organizational objectives meeting organizational objectives

ContinuedContinued… …

_ A newsletters can be evaluated in a _ A newsletters can be evaluated in a number of ways :number of ways :

1.1. Content analysis Content analysis

2.2. readership interest surveysreadership interest surveys

3.3. Article recallArticle recall

4.4. Advisory board Advisory board

Content analysisContent analysis

_ Based on representative samples of _ Based on representative samples of past issues past issues

_ this systematic analysis will show the _ this systematic analysis will show the percentage of the publication is percentage of the publication is devoted to each category devoted to each category

_ stories may be categorized under _ stories may be categorized under general headings such as New general headings such as New product developments , Job related product developments , Job related information information

readership interest surveysreadership interest surveys

_ The purpose of these surveys is to _ The purpose of these surveys is to get feedback about the types of get feedback about the types of stories employees are most stories employees are most interested in reading , the most interested in reading , the most common survey method is simply to common survey method is simply to provide along list of generic story to provide along list of generic story to picks and have employees rate each picks and have employees rate each as 1.important one 2.to some what as 1.important one 2.to some what important 3.or notimportant 3.or not important important

Article recallArticle recall

_ The best kind of readership survey _ The best kind of readership survey occurs when trained interviews ask a occurs when trained interviews ask a sampling of employees what they sampling of employees what they have read in the latest issue of the have read in the latest issue of the publication .publication .

Advisory boardAdvisory board

_ _Periodic feedback and evaluation can Periodic feedback and evaluation can be provided by organizing an be provided by organizing an employee advisory board meet employee advisory board meet several times a year to discuss the several times a year to discuss the direction of the publication this is a direction of the publication this is a useful technique because it expands useful technique because it expands the editor feedback net work and clear the editor feedback net work and clear comments that employees might be comments that employees might be hesitant to tell the editor face to facehesitant to tell the editor face to face..

Thank you Thank you