U.S. Department of EnergyOffice of Environmental Management
January 1996
Environmental ManagementBenchmarking Guide
TABLE OF CONTENTS
Part 1 - Introduction
Benefits of Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
The Benchmarking Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Benchmarking Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
How to Use This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Part 2 - The Benchmarking Model
Planning Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Data Collection Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Analysis Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Action Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Action Phase (Post-Study) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
In the past decade, benchmarking has become a primary resource for organizations seeking to improve qualityand performance. Benchmarking is a method of process analysis in which superior practices are identified andevaluated as models for improvement. Experience gained in both the public and private sectors has reinforcedthe practical value of benchmarking to environmental management activities. Some important advantages ofprocess improvement achieved through benchmarking are the following:
v cost savings
v productivity improvement
v process streamlining
v justification of resources and budget requests
v "out-of-the-box" thinking
This guide is intended for use by the U.S. Department of Energy (DOE) Office of Environmental Management,its management & operations contractors, and its support contractors to perform benchmarking analyses. Theguide provides a flexible model for improving Environmental Management processes that can be used as atemplate for conducting benchmarking studies both at headquarters and in the field.
The following table presents several common misconceptions about benchmarking that should be clarified beforeundertaking a formal benchmarking effort.
A single benchmarking study can lead to process improvements, but the processshould be periodically recalibrated to maintain the highest level of performance
Benchmarking is a one-time effort
Staff organizations perform functions such as document review, correspondence, andother administrative processes which can be benchmarked
Benchmarking can not address staff organizations
Site visits are intended primarily to verify information obtained from other sources suchas surveys and background research
Benchmarking is simply site visits to other organizations
An organization must first have a clear understanding of its own processes in order toset realistic and specific goals for comparative analyses
Benchmarking is simply a review of other practices
Breakthrough ideas often come from similar processes performed by organizations incompletely different industries
Benchmarking uses only competitive analysis
Although measuring quantitative performance can be extremely valuable,benchmarking is ultimately about how processes work and how to improve them
Benchmarking uses only quantitative metrics
FACTMYTH
Environmental Management Benchmarking Guide - i
Introduction
Benefits of Benchmarking
The model presented in this guide offers a general procedure for Environmental Management benchmarkingstudies. Benchmarking teams embarking on a study may choose to follow the proposed phases and stepsrigorously, or they may omit certain steps depending on the goals of the team and the nature of the particularstudy. In either case, the guide provides a common language for headquarters and field personnel at any level tocommunicate benchmarking ideas across the DOE complex. Additional information and reports from completedbenchmarking studies can be found in the DOE Environmental Management Benchmarking Clearinghouse athttp://www.em.doe.gov/bch/index.html on the Internet.
Benchmarking employs comparative analyses with other organizations, called benchmarking partners, to generatecreative solutions to process inefficiencies. A wide range of partners may be selected for a benchmarking study.It is not necessary to select partners that have identical organizational missions, only similar processes. Onewell-known example of utilizing benchmarking partners from different industries is the case of a shotgun shellmanufacturer who sought help from a cosmetics company. Because the cosmetics company was doing anoutstanding job producing lipstick cases, their manufacturing process was successfully adapted to produce highquality shotgun shell casings.
It is important to remember that the participation and support of management and stakeholders is essential to thesuccess of a benchmarking study. The amount of stakeholder involvement will vary with the process beingstudied and should be decided by the benchmarking team. Without such involvement, however, changessuggested by the benchmarking team may meet stiff resistance.
The benchmarking model offered in this guide contains 23 unique steps divided into four phases, as shown in thetable below. The approximate time commitment for each phase as a percentage of the total time required tocomplete the study is shown in parentheses. The time needed to complete post-study steps is not included in thisestimate. Post-study activities are often the most difficult steps in the benchmarking effort and may requireconsiderable time to complete, but they are essential in achieving process improvements.
Brief Management
Recalibrate the ProcessIdentify Benchmarking Partners
Track ProgressEstablish Process Measures
Implement Process ImprovementsRefine the Flow Chart
Post-StudyDevelop a Flow Chart
Brief ManagementBrief ManagementDefine the Scope
Brief ManagementDetermine the Performance GapConduct Site VisitsChoose the Team
Develop an Implementation PlanAnalyze Qualitative DataDistribute SurveysSelect the Process
Report FindingsAnalyze Quantitative DataConduct Background ResearchClarify the Objective
ACTION(20%)
ANALYSIS(20%)
DATA COLLECTION(20%)
PLANNING(40%)
ii - Environmental Management Benchmarking Guide
Introduction
The Benchmarking Model
Benchmarking must be conducted with the highest ethical standards to maintain trust and confidence between thebenchmarking partners. Because the sharing of information is such an important aspect of benchmarking, thereare several ethical and legal concepts that should always be considered during the course of a study:
v to avoid misrepresentation, federal employees must identify themselves as such when seeking non-publicinformation
v transferred information remains confidential between the organization conducting the benchmarking studyand the individual benchmarking partner
v organizations requesting process information should be willing to provide similar information to theirbenchmarking partners
v information that will be used to support a government position in another forum, such as contractnegotiations, should not be requested
v organizational secrets or proprietary data should not be requested
v information should not be gathered under false pretenses
v benchmarking partners should be guaranteed anonymity if so desired
When contacting potential partners, the benchmarking team should develop a policy statement on ethics incooperation with each partner organization. Contractual obligations that exist between DOE and its contractorsmust be kept in mind when determining what information can be shared permissibly. When seeking information,remember that if a request seems like it may be inappropriate, it probably is.
Environmental Management Benchmarking Guide - iii
Introduction
Benchmarking Ethics
Key Ideas
�� The "key ideas" section contains the primary concepts behind each benchmarking step.
�� Key ideas provide useful guidance for performing each step of the benchmarking model.
�� Key ideas help maintain the focus of the study.
�� Key ideas identify relevant participants in each benchmarking step.
A benchmarking example...
is developed to demonstrate the execution of each step in the benchmarking model. The example is ahypothetical scenario set at a DOE field site seeking improvements in its facilities management program.
Helpful Hints
Many pages provide hints for carrying out the stepsin your own benchmarking study. Helpful hints mayinclude the following:
v checklists
v suggested approaches
v specific criteria
v memory joggers
v useful resources
iv - Environmental Management Benchmarking Guide
Introduction
How to Use This Guide
�� This guide follows a simple, consistent format. The title of a benchmarking step appears in this box at
the top of each page, followed by a concise definition of the step.
Key Ideas
�� There must be a strong belief in benchmarking and the need to conduct a benchmarking study.
�� Management must embrace the benchmarking objectives and be committed to implementing changes.
�� Relevant stakeholders should be considered during the development of benchmarking objectives.
�� Benchmarking may be used to address the following:
{ a specific problem requiring immediate attention
{ new process requirements or changes in existing processes
{ previously unsuccessful process improvement efforts
A DOE field site...
is attempting to improve the efficiency of its facilities management program. The site managers wish to reducethe costs of the program while maintaining the current level of quality. They are, however, uncertain how tomeet this objective. To determine exactly what changes need to be made, they decide to conduct abenchmarking study. The objectives of the study are to determine what process changes can be made to thefacilities management program in order to improve the overall performance of the site.
Helpful Hints
Your organization should consider benchmarking if any ofthe following are true:
v You are seeking quality improvement
v You are trying to reduce costs
v You are changing management structures
v You are starting a new operation
v You are rethinking existing strategies
Environmental Management Benchmarking Guide - 1
The Benchmarking Model - PLANNING PHASE
Clarify the Objective
�� An organization first must understand the factors driving the decision to benchmark and achieve
consensus on the intended outcome of the benchmarking study. The objectives of the study should be
aligned with the overall mission and goals of the organization.
Key Ideas
�� The benchmarking study should target costly processes that hold significant room for improvement.
�� The selected process may be part of a larger system and should contain several distinct activities.
�� The benchmarking study should encourage stakeholder participation.
After considering several processes...
within the facilities management program, management decides to benchmark the painting process. Thisprocess is chosen because of its high annual cost and its perceived value in the eyes of site managers.Management believes that by benchmarking the painting process used for all facilities within the site, it willfind numerous inefficiencies and areas for improvement.
Helpful Hints
When selecting a process to benchmark, consider the following questions:
v Which processes are most crucial to your success?
v How is your performance measured?
v Are there excessive layers, interfaces, hand-offs, or concurrences?
2 - Environmental Management Benchmarking Guide
The Benchmarking Model - PLANNING PHASE
Select the Process
�� A process is a clearly-defined series of related activities producing a product or service. The
benchmarking study should focus on a process that is important to the success of the organization.
System
Process
Activity ActivityActivityActivityActivityActivityActivityActivityActivity
Process Process
Key Ideas
�� The study sponsor funds the study and acts as a primary liaison to groups outside the benchmarking team.
�� The process owner must have the authority to implement change as a result of the benchmarking study.
�� The process manager understands the activities and events being benchmarked.
�� The benchmarking facilitator maintains focus and provides benchmarking expertise within the team.
�� The core benchmarking team should utilize the skills of other members of the organization and clearly explainthe value of their contributions.
�� The suppliers and customers of the process--those persons providing inputs to or receiving outputs from theprocess--should be represented on the team when appropriate.
�� A responsibility matrix, shown below, can be used to define the roles of the benchmarking team members.
Once the objectives are clear...
the DOE field site manager (the study sponsor) commissions a core benchmarking team to perform the study.The core team includes a DOE project manager (the process owner), the site facilities management contractor(the process manager), and a DOE employee at the site who has participated in a previous benchmarking effortand is capable of training other team members in the benchmarking process (the benchmarking facilitator).
xxxxTracks Progress
xxxxImplements Process Improvements
xxxxxCommunicates Study Findings
xxReviews the Progress of the Team
xxxConducts Visits to Other Organizations
xxxUnderstands the Process Problems
xxCommunicates with Stakeholders
xOversees Administration of the Study
xTrains the Team in Benchmarking
xManages the Study Resources
xKicks Off the Benchmarking Effort
Other TeamMembers
BenchmarkingFacilitator
ProcessManager
ProcessOwner
StudySponsor
Responsibility
Environmental Management Benchmarking Guide - 3
The Benchmarking Model - PLANNING PHASE
Choose the Team
�� The core benchmarking team should always include the process owner, the process manager, and a
facilitator knowledgeable about the benchmarking process. Other key members of the organization may
contribute to various tasks depending on the phase of the benchmarking study.
Key Ideas
�� The benchmarking study should be conducted at the process level.
�� Process boundaries are defined by distinct start and finish activities.
�� Studies that are too broad may do the following:
{ prevent effective process analysis
{ consume too many resources
{ lead to impractical recommendations for change
�� Studies that are too narrow may do the following:
{ overlook process improvement opportunities
{ preclude significant change
The core benchmarking team...
must now define a scope of work that addresses the study objectives while staying within their budget andschedule constraints. If the scope is too large, it will be difficult to determine and implement changes in thepainting process. If the scope is too narrow, however, the suggested changes are likely to be insignificant.After weighing these concerns, the team decides to sharpen the focus of the benchmarking study to the paintingprocess used for exterior building surfaces. This process begins with the purchase of materials and ends withthe approval of the final coat of paint.
4 - Environmental Management Benchmarking Guide
The Benchmarking Model - PLANNING PHASE
Define the Scope
�� The scope of the benchmarking study should reflect the objectives and resources of the organization.
The primary consideration in defining scope is avoiding studies that are either too broad or too narrow.
Facilities Management Program
Exterior Painting
PurchaseMaterials
Paint and Inspect
ApprovePaint Job
Key Ideas
�� It is important to understand as much as possible about the details of your process before benchmarking itwith other organizations.
�� The flow chart defines who participates in the process and the relationships among process activities.
�� The benchmarking facilitator should assist the team members during development of the flow chart.
�� Flow charts are typically developed using standard symbols for process activities ( ), decision points ( ),and start/finish activities ( ).
The team develops...
the draft flow chart shown below to begin evaluating the painting process used for exterior building surfaces.This chart contains all the activities and decisions involved in the painting process. The process begins with thepurchase of painting materials and proceeds through a series of applications and inspections. The process iscomplete when the paint job has received final inspection and approval from the DOE project manager.
Helpful Hints
If you are uncertain whether or not to include anactivity in your flow chart, consider these questions:
v Is the activity contained within the definedprocess boundaries?
v Are you depicting your process at theappropriate level of detail?
v What is the consensus of the team?
Environmental Management Benchmarking Guide - 5
The Benchmarking Model - PLANNING PHASE
Develop a Flow Chart
�� Flow charts describe the sequence of discrete activities in a process. Developing flow charts can reveal
inefficient, redundant, and unnecessary process steps.
EvaluateSurface
Condition
ApplyFirst Coat
InspectApplication
ApplySecond Coat
ApplyAdditional Coat
Repair Damage
Prepare SurfaceInspect
Application
Purchase Materials
Good
Bad
Good
Bad
DRAFT
ApprovePaint Job
Key Ideas
�� All process activities should be examined to ensure that they fall within the process boundaries and aredepicted at the appropriate level of detail.
�� Start and finish activities should be clearly defined in the flow chart.
�� Decision points should be evaluated to verify the logic of the process flow.
Upon careful inspection...
of the draft flow chart, the benchmarking team realizes that the first 'Inspect Application' decision point isunnecessary because a second coat of paint is always applied. This decision point is therefore eliminated andthe sequence of the remaining process steps is approved by the team. The resources required to complete eachstep in the process are also identified. In this case, the time, materials, and cost associated with each activityare estimated and recorded. The specific criteria for each decision point are also examined. The team notesthat the 'Evaluate Surface Condition' decision assesses flaking and smoothness, and the 'Inspect Application'decision considers texture and coverage of a newly applied coat of paint. The team is now confident that itclearly understands the process and is ready to continue with the benchmarking study.
6 - Environmental Management Benchmarking Guide
The Benchmarking Model - PLANNING PHASE
Refine the Flow Chart
�� The benchmarking team should agree on the scope and sequence of each step in the process. The
resources needed to complete each process step also should be determined.
EvaluateSurface
Condition
ApplyFirst Coat
InspectApplication
ApplySecond Coat
ApplyAdditional Coat
Repair Damage
Prepare Surface
Good
Bad
Good
Bad
FINAL
Purchase Materials
ApprovePaint Job
Key Ideas
�� A few vital measures are better than many that are irrelevant.
�� Quantitative measures are effective means of comparing resource requirements such as time and cost.
�� Qualitative measures can provide anecdotal evidence in support of quantitative findings.
�� Some examples of process measures are the following:
{ cost to complete the process
{ time to complete the process
{ quality of the completed process
{ number of rework cycles
{ number of review loops
To compare the painting process...
used at the DOE site to those of other organizations, the team must establish a set of process performancemeasures. They are careful to choose measures that reflect the objective of their benchmarking study: toreduce facilities management costs while at least maintaining current quality standards. The team decides thatthe most appropriate process measures are 1) the time to apply a single coat of paint, which currently averages13 minutes per hundred square feet; 2) the unit cost of a single coat, now 21 cents per square foot; and 3) thetime between repainting, presently 28 months.
Helpful Hints
When selecting process performance measures,consider these questions:
v Which measures are critical to performance?
v How many measures are really necessary?
v Will the desired data be available?
v Is the level of detail appropriate?
v If process measures are difficult to establish,is this really a good process to benchmark?
Environmental Management Benchmarking Guide - 7
The Benchmarking Model - PLANNING PHASE
Establish Process Measures
�� Process measures are used to compare the process performance of different organizations. The team
should select process measures that reflect the objectives of the benchmarking study.
0
10
20
30
min/100 sf cents/sf m onths
Key Ideas
�� Internal benchmarking utilizes partners from other areas within your organization.
�� External benchmarking seeks partners outside your organization.
�� Similar processes may be encountered in widely different industries.
�� Partners should have performance levels that can be realistically emulated.
�� An organization must be careful not to confuse a process that is different with one that is better.
�� Potential benchmarking partners may be identified through the following:
{ trade magazines, industry publications, and professional journals
{ other government agencies
{ professional contacts
Before selecting partners...
for the study, the team must decide whether to use internal or external benchmarking partners. In order tomaximize the potential return on their efforts, they choose to use both internal and external partners. The teamknows that most DOE sites perform painting as part of their facilities management program, and therefore theyselect two other sites with potentially more efficient painting processes as internal partners. After speaking totheir painting materials supplier, the team discovers that a nearby apartment leasing company and a largemanufacturing firm use similar and potentially more efficient painting methods, making them excellentcandidates for external partnering. The team contacts the four organizations, requests their participation asbenchmarking partners, and establishes the necessary information sharing agreements.
Helpful Hints
If your organization is uncertain whether to choose internalor external benchmarking partners, consider the following:
v large organizations such as DOE often can benefitsignificantly from internal studies
v small organizations often need to look externally dueto a lack of similar internal processes
8 - Environmental Management Benchmarking Guide
The Benchmarking Model - PLANNING PHASE
Identify Benchmarking Partners
�� Benchmarking partners are perceived as having better performance compared to your organization.
Considerations for selecting partners include whether the process is similar enough to allow comparison,
whether the organization is likely to share information, and whether the information is recent and reliable.
Key Ideas
�� Briefings should emphasize to management the effectiveness of the benchmarking study in achieving themission and goals of the organization.
�� During briefings, the team should revisit the benchmarking objectives and discuss the next phase of the studyin relation to their findings to date.
�� Briefings should be used to maintain management and stakeholder support of the benchmarking study.
Once the team has completed...
the planning phase of the benchmarking study, they brief site management on current progress and results.Because they have clearly defined the scope of the benchmarking study and have established valuable processmeasures, the team receives further direction and authority to continue with data collection.
Helpful Hints
The briefing for the planning phase of the benchmarking study should do the following:
v address each of the eight steps in the planning phase
v present notable results achieved during the planning phase
v discuss the expected outcome of the impending data collection phase
Environmental Management Benchmarking Guide - 9
The Benchmarking Model - PLANNING PHASE
Brief Management
�� Briefings are needed to gain approval from sponsors and to inform stakeholders of progress. This step
should occur at the end of each phase of the benchmarking study.
Key Ideas
�� Gathering as much relevant background information as possible educates the team on the process beingbenchmarked, enabling better communication between the team and its benchmarking partners.
�� To avoid duplication of effort, the team should identify research that already has been done in this area.
�� A literature review should be conducted to obtain the following information:
{ general background on the process
{ new technologies relating to the process
{ previous studies of the process or similar processes
Before utilizing its partners...
for information exchange through surveys and site visits, the team conducts background research using theDOE Environmental Management Benchmarking Clearinghouse, other Internet services, and various librariesand publishing services. One of the most helpful pieces of information they find is a 1993 benchmarking studyof exterior painting processes conducted by PolyCorp. This helps to educate the team and to stimulate ideasfor their data survey. The team is satisfied that it has gathered all the relevant information on paintingprocesses that can be obtained without contacting its benchmarking partners.
Helpful Hints
Some valuable resources for backgroundresearch include the following:
v libraries
v government agencies
v trade associations
v benchmarking clearinghouses (includingthe DOE clearinghouse athttp://www.em.doe.gov/bch/index.htmlon the Internet)
10 - Environmental Management Benchmarking Guide
The Benchmarking Model - DATA COLLECTION PHASE
Conduct Background Research
�� The benchmarking team should collect all relevant data within the scope of the study that can be
obtained without directly contacting the benchmarking partners. This research will allow the team and its
partners to optimize their time during the survey and site visit steps.
BACKGROUNDRESEARCH
ResearchLibraries
DOEClearinghouse
National Painter's Union
FederalAgencies
ConsumerOrganizations
PolyCorpBenchmark ing
Study
Technical Publications
Vendor Information
Key Ideas
�� The survey should be used to augment data gathered during background research.
�� Do not request excessive or unnecessary information.
�� Include a statement on the ethical issues relating to the transfer of information and a statement ensuring theprotection of the partner's information.
�� Follow up the survey with a letter thanking the partner for participating in the study.
The team develops a survey...
that will be mailed out to the four benchmarking partners prior to site visits. It includes statements on theprocess measures that the team will be using and on the release of results to outside parties once the study hasbeen completed. The team explains that they will be looking primarily at the time to apply a single coat ofpaint, the unit cost of a single coat, and the average time between repainting. The survey also asks forbackground information on the partner organization and for a flow chart of the partner's painting process. Thesurvey is pilot tested within the facilities management program before being sent to the partners.
Helpful Hints
The survey should be written to allow the following:
v qualitative responses to supplement quantitative data
v open-ended discussion of relevant issues
v normalization of data for comparisons among partners
v anonymity of respondents
Environmental Management Benchmarking Guide - 11
The Benchmarking Model - DATA COLLECTION PHASE
Distribute Surveys
�� A survey is used to collect process data from the benchmarking partners. It should be designed around
the process measures in order to ensure that a good comparison between processes can be made.
Background Information- Describe your organization- Describe your site
Process Description- Describe your process- Flow chart your process
Process Measures- How long does it take?- How much does it cost?- How often is it done?- How do you measure these?
Other Information- What works well?- What doesn't?- Why?- How could you improve?
BENCHMARKING SURVEY
SAMPLE
Key Ideas
�� Site visits should be planned in advance so the partners can prepare for the benchmarking team.
�� A site visit questionnaire should be developed to supplement the survey and resolve any unclear responses.
As a follow up...
to the survey, the benchmarking team decides to conduct site visits. These help the team clarify some of thesurvey answers as well as obtain other useful information. From the survey, the team has discovered that byusing power spray painters, one external benchmarking partner can apply a coat of paint in half the time oftheir current process which utilizes rollers. The team does not know, however, the exact technique for using thepower spray painter until it visits the site for a demonstration.
Helpful Hints
Teams conducting site visits should do the following:
v consider the cultures of partner organizations
v limit the visiting team to only a few people
v be grateful and courteous
v agree to provide partners with information they maywant in the future
12 - Environmental Management Benchmarking Guide
The Benchmarking Model - DATA COLLECTION PHASE
Conduct Site Visits
�� Site visits to partner organizations are used to collect information that may have been unobtainable
through the survey or background research. Conducting site visits gives the benchmarking team first-hand
knowledge of the processes used by its partners.
Key Ideas
�� It is important to relay to management the type of information gathered during the data collection phase ofthe study and the means by which it will be analyzed.
�� The briefing should also summarize the sharing agreements established with the benchmarking partners.
�� During the briefing, management may offer insight and direction for conducting the analysis.
Having collected data...
from background research, surveys, and site visits, the benchmarking team presents their preliminary findingsto site management. The DOE field site manager is enthusiastic about the study and encourages the team tocontinue with formal quantitative and qualitative analyses.
Helpful Hints
The briefing for the data collection phase of the benchmarking study should do the following:
v address each of the three steps in the data collection phase
v present notable results achieved during the data collection phase
v discuss the expected outcome of the impending analysis phase
Environmental Management Benchmarking Guide - 13
The Benchmarking Model - DATA COLLECTION PHASE
Brief Management
�� Briefings are needed to gain approval from sponsors and to inform stakeholders of progress. This step
should occur at the end of each phase of the benchmarking study.
Key Ideas
�� The quantitative analysis should be conducted in terms of the process measures.
�� Data should be normalized if necessary to allow relevant comparisons to be made among the organizations.
�� Quantitative results should be displayed graphically to help identify patterns.
After receiving the surveys...
back from its benchmarking partners, the team compiles all the quantitative information in order to makecomparisons with its own process data. This helps the team immediately spot differences between its processand those of the other organizations. Some of these differences include the time to apply a single coat of paint,the cost per coat, and the average time between repainting. The team also compares the process flow chartsprovided by the benchmarking partners with the flow chart of their current process.
Helpful Hints
When analyzing quantitative data, keep in mind the following:
v display the data in different ways to help identify patterns or trends
v assess where and when performance differs across organizations
v statistical representation is not required for case study analyses
14 - Environmental Management Benchmarking Guide
The Benchmarking Model - ANALYSIS PHASE
Analyze Quantitative Data
�� A quantitative analysis must be conducted to compare data from the benchmarking organization with
that of its partners. This analysis identifies potential areas for improvement in productivity and quality
based on the established process measures.
0
2
4
6
8
10
12
14
Partner 1 Partner 2 Partner 3 Partner 4 Partner 1 Partner 2 Partner 3 Partner 48
10
12
14
16
35
40
45
50
55
Partner 1 Partner 2 Partner 3 Partner 4
Key Ideas
�� The qualitative analysis should include a description of the processes used by the benchmarking partners.
�� Qualitative analyses become especially important when evaluating steps that are not easily quantifiable.
�� The qualitative analysis should closely examine overall organizational policy, philosophy, and culture.
In addition to quantitative data...
the team has collected qualitative data on the painting process. This information is used to help the teamunderstand the process differences found in the quantitative data. Some of the qualitative differences includesurface repair procedures, type of paint used, and application techniques.
Helpful Hints
When analyzing organizational culture and policy, considerthe following issues:
v degree of process oversight
v degree of process ownership and authority
v degree of external regulation
Environmental Management Benchmarking Guide - 15
The Benchmarking Model - ANALYSIS PHASE
Analyze Qualitative Data
�� A qualitative examination of process differences must supplement the quantitative analysis in order to
give meaning to the benchmarking process. Qualitative information is used to explain the differences in
quantitative measures among the organizations.
Key Ideas
�� The performance gap demonstrates to the core benchmarking team how much improvement is possible.
�� The changes needed to achieve the desired process improvements often become clear when determining theperformance gap.
�� The team may also find areas in which their organization outperforms the benchmarking partners, thusreducing the need for improvement in those aspects of the process.
The quantitative and qualitative data...
are combined in the performance gap analysis. Qualitative information is linked with specific quantitativefindings to provide an in-depth understanding of the process differences that exist among the organizations.The analysis identifies which process is most efficient in each of the three process measures. Specifically, theteam finds that using power spray painters would be much more efficient than using rollers. Painting withpower spray painters requires less time (7 minutes/100 sf) than rollers and therefore costs less ($0.12/sf). Also,because of the more rigorous surface preparation required for spray painting, the durability of paint jobs isincreased significantly. The average time between repainting with spray painters is 48 months, compared to the28-month cycle typical for roller applications.
16 - Environmental Management Benchmarking Guide
The Benchmarking Model - ANALYSIS PHASE
Determine the Performance Gap
�� The performance gap is the measured difference in performance between the current process and the
processes employed by the benchmarking partners. This gap is determined by combining quantitative and
qualitative data to assess relevant performance for the process measures.
FacilitiesManagement
24
42
36
48
30
54
28months
InternalPartners
24
42
36
48
30
54
36months
ExternalPartners
24
42
36
48
30
54
48months
Key Ideas
�� The briefing presents the results of quantitative and qualitative analyses to the study sponsor and other sitepersonnel and stakeholders.
�� The performance gap is presented as a goal for process improvement.
�� The benchmarking team discusses potential strategies for implementing process changes.
Following data analysis...
the benchmarking team presents quantitative and qualitative results summarizing the performance gaps foundfor the three process measures. Site management is eager to receive recommendations for improving thepainting process and consequently requests a formal findings report and implementation plan.
Helpful Hints
The briefing for the analysis phase of the benchmarking study should do the following:
v address each of the three steps in the analysis phase
v present notable results achieved during the analysis phase
v discuss the expected outcome of the impending action phase
Environmental Management Benchmarking Guide - 17
The Benchmarking Model - ANALYSIS PHASE
Brief Management
�� Briefings are needed to gain approval from sponsors and to inform stakeholders of progress. This step
should occur at the end of each phase of the benchmarking study.
Key Ideas
�� The report should first present the most important results of the benchmarking study.
�� Both quantitative and qualitative results should be displayed.
�� The report should establish reasonable goals for improving the process.
�� Recommendations must be supported by data and analysis.
�� The benefits of implementing recommendations should be quantified whenever possible.
�� To maintain the momentum of the study, the report should be completed soon after the analysis phase.
After summarizing its findings...
the team reaches an agreement on the changes to recommend for improving the painting process. The teamincludes the recommendations, as well as the supporting evidence behind the suggested changes, in a report tothe DOE field site manager. The team members also send the report to the benchmarking partners along withother agreed upon information that previously had not been shared.
Helpful Hints
When writing the benchmarking report, consider the following:
v the audience of the report
v the initial goals of the benchmarking study
v the most important results of the study
v the resources needed to carry out the recommendations
v the effects of process changes on personnel
18 - Environmental Management Benchmarking Guide
The Benchmarking Model - ACTION PHASE
Report Findings
�� Once the benchmarking team knows which process improvements should be made, a report is prepared
for management describing those improvements and the reasons they should be implemented. The report
must convey the strength of the analysis in order to gain approval for implementing the proposed changes.
I. Purpose (1 pg.)
II. Methodology (2 pg.)
III. Results (5 pg.)
IV. Recommendations (2 pg.)
Appendix A - Analysis
Appendix B - Literature Review
Appendix C - Survey
Appendix D - Site Vi sits
Appendix E - Other Data
BENCHMARKING REPORT
SAMPLE
Key Ideas
�� The implementation plan must consider the time and resources necessary to achieve process improvements inan efficient and effective manner.
�� The implementation plan must address the likely effects of the recommended process improvements on theorganizational culture.
�� To minimize organizational resistance, the implementation plan should take an incremental approach tointroducing process changes when appropriate.
�� A contingency plan should be developed in the event that a problem occurs during implementation of themodified process.
In addition to the findings report...
an implementation plan is submitted to management explaining how to make the transition from the currentprocess to the modified process. Input from the process owner and the process manager help the teamdetermine the resources available for initiating the changes. The team takes an incremental approach to theplan and recommends changing one painting team per month from rollers to power spray painters.
Helpful Hints
Be sure to consider the following factors whendeveloping an implementation plan:
v the scope of the proposed changes
v anticipated objections to the proposed changes
v anticipated problems with the modified process
Environmental Management Benchmarking Guide - 19
The Benchmarking Model - ACTION PHASE
Develop an Implementation Plan
�� As the final step in the benchmarking study, an implementation plan is necessary to enable the
transition from the current to the modified process. A detailed plan is essential for successfully
implementing process changes.
Key Ideas
�� Study results and a plan to implement process improvements should be presented in the briefing.
�� The briefing should seek to generate management support for the recommended changes.
�� Stakeholders should be informed of the proposed implementation activities.
The team presents...
a summary of its findings report and implementation plan to the DOE field site manager (the study sponsor) aswell as other federal employees, contractors, and stakeholder representatives. The team accepts suggestionsfor modifying the plan and solicits volunteers to lead the post-study implementation actions.
Helpful Hints
The briefing for the action phase of the benchmarking study should do the following:
v summarize both the findings report and the implementation plan
v present notable results achieved during the action phase
v discuss the expected outcome of the implementation plan
v ease the concerns of personnel affected by the proposed changes
20 - Environmental Management Benchmarking Guide
The Benchmarking Model - ACTION PHASE
Brief Management
�� Briefings are needed to gain approval from sponsors and to inform stakeholders of progress. This step
should occur at the end of each phase of the benchmarking study.
Key Ideas
�� The benchmarking effort is not over when the study is complete.
�� Implementation requires a commitment to change--not just meetings, briefings, and action plans.
�� Senior management must support and cooperate with the benchmarking team to implement process changes.
�� Improvements should be directed at processes, not people.
In the implementation step...
the process owner, process manager, and other participants in the process execute the implementation plan.They hold monthly training classes on how to use the new power painters as each painting team receives them.Over a period of several months, many process changes are implemented until the transition is completed.
Helpful Hints
As process improvements are introduced, they must beclosely coordinated with other groups in the organizationincluding the following:
v management
v quality control
v accounting and finance
Environmental Management Benchmarking Guide - 21
The Benchmarking Model - ACTION PHASE (Post-Study)
Implement Process Improvements
�� Process improvements are implemented through the methods described in the implementation plan.
Without implementing the recommended changes, the process will not be improved and the goals of the
benchmarking study will not be realized.
Key Ideas
�� Progress tracking must be consistent and thorough so that potential problems are identified immediately.
�� Evaluate progress on the basis of the process measures.
�� Use a variety of continuous improvement tools (e.g., histograms, run charts) to monitor performance.
�� Verify that change is actually occurring; do not measure simply for the sake of measuring.
�� Maintain progress records for use in later benchmarking studies.
Management tracks progress...
of the modified process during the transition period and several months after full implementation. They findthat the average painting time is initially reduced to 10 minutes per hundred square feet (45 percent of theirultimate improvement goal), cost is reduced to 18 cents per square foot (35 percent of their target reduction),and durability is projected to increase to 40 months (60 percent of their improvement goal). At this rate, theteam believes that they can reach 100 percent of their improvement goals within twelve months after fullimplementation. Management also finds that the painting teams are dissatisfied with the long transition period.In response, the facilities management program accelerates the pace of the transition.
Helpful Hints
While tracking improvements in the modifiedprocess, the team should consider the following:
v effects on the process measures
v effects on other processes in the system
v organizational attitudes toward the changes
22 - Environmental Management Benchmarking Guide
The Benchmarking Model - ACTION PHASE (Post-Study)
Track Progress
�� Progress is tracked using the performance measures established in the planning phase of the
benchmarking study. Progress tracking can identify problems that may arise during the implementation
of process improvements.
1 2 3 4 5 60%
20%
40%
60%
80%
100%
Time Cost Durability
1 2 3 4 5 6
Months
Percent of Goal
Key Ideas
�� Review of the completed study is necessary to ensure that the benchmarking objectives were achieved.
�� Recalibration can act as a springboard for future benchmarking efforts.
�� Recalibration is an important benchmarking step because it provides an opportunity for improving thebenchmarking methodology.
With the modified process in place...
essential data are now available for further benchmarking efforts on the painting process. Although themodified process is much more efficient than the original, there are still more refinements that could be made.The modified process has become the baseline for a future study, and the team can now consider benchmarkingother processes in the facilities management program.
Environmental Management Benchmarking Guide - 23
The Benchmarking Model - ACTION PHASE (Post-Study)
Recalibrate the Process
�� Recalibration is the review of the completed study and the establishment of a new process baseline.
Recalibration promotes further improvement in the benchmarked process as well as enhancement of the
benchmarking method itself.
EvaluateSurface
Condition
ApplyFirst Coat
Repair Damage
Prepare Surface
Good
Bad
MODIFIED
Purchase Materials
Bad
GoodInspect
ApplicationApprovePaint Job
ApplyAdditional Coat