CA
SE S
TUD
YLarge-Scale e-Learning DeploymentIRS Increases Training Efficiency with Enterprisewide Technology Strategy
IN THIS CASE STUDYThe Internal Revenue Service (IRS) has recently begun the implementation of a new approach to employee development and agent knowledge maintenance. The incorporation of a learning content management system (LCMS) and other electronic performance support systems (EPSS) has been implemented in an attempt to update and streamline current learning and development (L&D) techniques. For this initiative, the IRS has a central goal for employees to utilize a more user-friendly system and to access up-to-date content that can be readily retrieved from a central repository. The progress on integrating the web-based learning modules and the updated procedures with this new technology are detailed within this case study. The IRS seeks to:
• Increase the availability of training and reference materials made available throughout the organization to its employees for training or practical on-the-job application;
BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
BERSIN & ASSOCIATES
PLANNING& STRATEGY
CONTENTDEVELOPMENT
LEARNINGPROGRAMS
LEARNINGTECHNOLOGY
ANALYTICS &MEASUREMENT
TALENTMANAGEMENT
FOCUS:
—Chris Howard, Principal Analyst | June 2007
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
• Decrease its budget expenditures for unnecessary travel and time expenses related to traditional training techniques;
• Develop an accessible structure of associated and monitored data; and,
• Ensure the proper consistency and reusability of the data throughout the many different processes and procedures within the IRS. e
BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
TABLE OF CONTENTSOverview of the IRS Structure 4
The Business Environment 6
Reusable Content 8
Case in Point: Reusable Content in Action 9
Workflow Learning 10
The Learning Environment 12
People 12
Process 14
Technology 15
Learning Management System 16
Results 18
Next Steps 19
Blended-Learning Approach 19
The Future: HRConnect 20
Appendix I: Table of Figures 21
About Us 22
About This Research 22
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Overview of the IRS Structure
Initially created in 1862 as a means to repay war expenses, the Internal Revenue Service (IRS) today follows a mission to provide U.S. taxpayers with top-quality service – to ensure they understand and meet their tax responsibilities through applying the tax laws with integrity and fairness to all. In 2004, the IRS collected more than $2 trillion in revenue and processed more than 224 million tax returns.�
To support its mission and ensure accountability, the IRS is divided into three commissioner-level organizations (see Figure 1):
• Commissioner;
• Deputy Commissioner for Services and Enforcement; and,
• Deputy Commissioner for Operations Support.
The organizational structure of the deputy commissioner for operations support includes a stand-alone chief financial officer (CFO) who oversees the entire organization. Under the CFO, the human capital office (HCO) oversees the human resources (HR) services and learning / education functions. The HCO was introduced a few years ago through an Office of Personnel Management (OPM) executive order and spans all 12 business units within the IRS. The centralized HCO acts as a steward on behalf of each of these business units by making decisions regarding educational technologies and policies for learning programs under its direction.
The IRS uses a distributed model of education – wherein responsibility for training is not centralized under the HCO; instead, the training functions and course development responsibilities are embedded within each of the business units – with one exception. The HCO provides all training related to general leadership competencies and is also responsible for HR specialist curriculum certificates (e.g., test item
1 Source: IRS website, “About Us: The Agency, its Mission and Statutory
Authority,” http://www.irs.gov/irs/article/0,,id=98141,00.html.
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Figure 1. IRS Organizational Chart
development and performance consulting�,�). Continuing professional education (CPE) sessions also fall under this exception. These CPE sessions are available in a number of areas (e.g., policy and program, technology, and leadership).
� “Performance consulting” is a needs assessment process that must be
completed to identify the root cause of the business problem. Working with the
line of business, performance consultants diagnose the business problem and
assess the needs, and then work with instructional designers to develop, launch,
manage and assess the training solution. Performance consulting does not
presume that the solution is training.� For more information, High-Impact Learning Measurement: Best Practices,
Models, and Business-Driven Solutions for the Measurement and Evaluation of
Corporate Training, Bersin & Associates / Josh Bersin, November 2006. Available to
research members at www.elearningresearch.com or for purchase at www.bersin.
com/measurement.
Figure 1: IRS Organizational Chart
Source: IRS, 2004.
CommissionerChief of Staff
Chief Counsel
Appeals
EEO and Diversity
Communicationsand Liaison
Research, Analysis and Statistics
National TaxpayerAdvocate
Internal Revenue ServiceDepartment of the Treasury
Office ofProfessional
Responsibility
Small Business / Self Employed
Large and Mid-SizeBusiness
Wage andInvestment
Chief InformationOfficer
Chief FinancialOfficer
Human Capital Officer
Agency-WideShared Services
MissionAssurance
CriminalInvestigation
Tax Exempt andGovernment
Entities
Services and Enforcement
OperationsSupport
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Figure 2: IRS Meta-Tagging Process
The Business Environment
In 2001, the IRS embarked on developing an e-learning strategy. At that time, the IRS was spending 65 percent of its training budget on travel, leaving only 35 percent for training delivery. The goal was set to reverse these equations and deliver more training, with an organizationwide training budget of $100 million. After an initial review, it became obvious that this overhaul would entail more than a mere shift towards reliance on online learning – this would truly be an overhaul of how the IRS manages data throughout all training efforts. The true focus would be the all-encompassing involvement of the data incorporated into and beyond the online classroom.
Figure 2: IRS Meta-Tagging Process
Source: IRS, 2004.
Internal Revenue Code
IRC
ProceduresInterpretationsAdministration
Meta-TaggedLearning Objects
into LCMS
Create New
Courses
Create Internal Revenue Manuals(IRMs)
Deliver Training
Extract RelevantObjects
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
With these first vital steps, challenges and issues were encountered, including:
• Managing the various forms of training and reference information;
• Too much of the training budget being spent on travel and not enough on learning;
• Needing more rapid and direct access to information; and,
• Cost-effectively training a large seasonal workforce.
From here, the actual challenges became clear, allowing project goals to be better defined. Due to budgetary needs, the IRS set forth long-term technological objectives, which included:
• Purchasing a user-friendly learning content management system (LCMS) as a means to efficiently develop and revise learning content;
• Creating SCORM-compliant� content meta-tagged for reuse via internally developed templates; and,
• Expanding content availability within the LCMS to all training content, not just electronic content.
In 2003, the IRS began to pursue these objectives and goals with the creation and implementation of course development templates. These XML�-based templates ensured that training would be developed consistently (i.e., formatting and presentation) – in addition to including a methodology for labeling learning content or “meta-tagging�,” that would be used throughout the organization (see Figure 2). This set the stage for
4 “Sharable Content Object Reference Model” (SCORM) is a set of specifications
for course content that produces reusable learning objects.5 “XML” is extensible markup language, which improves the functionality of
the web by allowing users to identify information in a more accurate, flexible
and adaptable way. It is “extensible” because it is not a fixed format like HTML
(which is a single, predefined markup language). Instead, XML is actually a meta-
language – a language for describing other languages – that lets users design
markup languages for limitless different types of documents.6 “Meta-tagging” (from “meta-tag”) refers to the process of creating and
storing data elements that describe the assets and modules contained in a course.
The “metadata” is used for purposes of locating the content (within a database or
repository) or customizing the course for delivery.
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
the inevitable implementation of an enterprisewide LCMS, based directly on the templates already in use. Although the templates were burdensome to use, the IRS saw them as a strategic advantage in transitioning to an LCMS and leveraging the reuse of content�. The future effectiveness of the LCMS was largely based upon practices used in current e-learning formats. A number of e-learning trends (e.g., reusable content, workflow learning / electronic performance support) were integrated within the development of the LCMS system for the IRS.
Reusable Content
For the rapid development of new and personalized training courses, emphasis was placed on “reusable content�.” These discrete learning objects� can be drawn from a central repository or database to quickly customize courses or other instructional materials (e.g., reference workbooks and job-aides), taking advantage of available technologies to better fit learners’:
• Needs;
• Preferences;
• Experience levels; and,
• Identified performance / skills gaps.
In the past, reusable content has been limited to online courses. By developing all learning content (including instructor-led courses) as reusable content, organizations are finding that reusable content can be utilized in both online and classroom settings – even in print. In addition, this method of content cataloguing creates opportunities for blended learning since the content is free from any specific format
7 For more information, Business Applications of Reusable Learning Content:
Analyses, Best Practices and Business-Driven Solutions Based on Current
Technologies and Usage, Bersin & Associates / Chris Howard, April 2007. Available
to research members at www.elearningresearch.com.8 “Reusable content” (also called “dynamic content”) is discrete learning objects
with no link to a specific course. 9 A “discrete learning object” can be a paragraph of text, a chart or diagram, or
any individual piece of content that can be linked with other objects to create any
number of training courses.
By developing all
learning content as
reusable content,
organizations are finding
that reusable content can
be utilized in both online
and classroom settings
– even in print.
BEST PRACT ICE
�BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
and can, therefore, be used in a variety of ways (e.g., job-aides, review packets and online classes). Other benefits of reusable content include:
• Reduction of development time;
• Rapid adjustment to learners’ needs; and,
• Consistency in the content and style of training.
Reusable content also facilitates a movement away from the classic asynchronous training event to a greater proportion of learning that occurs on the job.
In order to manage reusable content efficiently, Bersin & Associates suggests that it be broken down to its smallest parts (e.g., a graph, paragraph of text) and accurately meta-tagged. By breaking down content to its the simplest components, meta-tagging makes it possible for organizations to document how a piece of training content is used, as well as maximizing content reuse in the widest range of deployment methodologies.
Within this model, the IRS has implemented software by MetaSoft to link content to its source of interpretation. The creation of a manifest (for each template of data) narrows linked data to its specific use and allows the content intake department to cross-check all incoming associations for accuracy.
Case in Point: Reusable Content in Action
The IRS is responsible for abiding by and enforcing the Internal Revenue Code (IRC). Congressional legislation continually updates the IRC, creating a challenge for the distribution of updated codes to employees. The IRC tends to be cumbersome and difficult to understand. For this reason, Internal Revenue manuals (IRM) have been created to explain procedures related to tax collection, as well as to assist employees with the proper interpretation of administrative and procedural issues covered by the IRC.
Meta-tagging makes it
possible for organizations
to document how
a piece of training
content is used, as well
as maximizing content
reuse in the widest
range of deployment
methodologies.
K E Y P O I N T
�0BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Although intended to make the IRC more understandable, the sheer amount of material included creates inefficient coverage and general difficulty for employees referencing the manual. As a result of this inefficiency, employee development programs need to be directly linked to the IRM and especially to the IRC. Due to the massive quantity of material that must be covered in training and the frequency with which the IRC is updated by Congress, the IRS sought a user-friendly training solution that could easily create and update training courses.
The content is often pulled directly from the IRM. The IRS used the manual as the basis for the meta-tagging model for classroom trainers and course developers. Further, linking to the IRM ensures coverage for any topic on which the IRS might need to instruct employees. e
Workflow Learning
Other e-learning trends that have influenced the IRS’s learning strategy include “workflow learning” and electronic performance support systems (EPSS). Workflow learning refers to the methodology in which subject matter experts (SMEs) and course developers create curricula that are embedded in the workflow or closely simulate “on-the-job” experiences. It is estimated that about 90 percent of what employees learn is through informal, on-the-job training. This learning approach is designed to provide real-time training content while the trainee is engaged in the business process. Workflow learning creates a direct link between what is learned and how it is used on the job. This method increases retention of the material – and also maximizes the transfer and usage of the new skills. In addition, time and resources are saved from employees missing work or traveling to training courses.
In an effort to integrate workflow learning into organizations, EPSS has been introduced as an additional learning technology. Basically, the use of this system enables people who don’t know what they are doing to
Case in Point: Reusable Content in Action
Workflow learning
creates a direct link
between what is learned
and how it is used on the
job.
K E Y P O I N T
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
perform as though they do. Thus, EPSS systems go beyond providing information “just in case” employees need it. Further, these systems surpass simple job-aids / reference manuals that may not address the current challenge faced by an employee. Instead, the EPSS:
• Guides employees through processes;
• Simplifies work for non-experts; and,
• Encourages consistency in performance.
In 2004, the IRS began the development and requirement testing of a new LCMS system, ForceTen by Eedo. The LCMS went live in December 2005; this LCMS provided a centralized repository for training content, and a user-friendly means to quickly and efficiently develop and revise content utilizing reusable information objects.
Since the implementation of the LCMS, the IRS has leveraged reusable content in two primary ways:
• First, by linking course content to the IRM, changes in either the IRC or the manual can be quickly and automatically updated; and,
• Second, when a new development effort arises that relates to a specific part of the manual, the database can be searched for all content that is already available on the topic (e.g., auditing business losses).
This saves significant time and resources when developing new courses. Eventually, the meta-tagged content available within the LCMS will be expanded to all training content, not just electronic content. In addition to accomplishing the stated objectives by the IRS, this will also enable content to be easily reused for new training requirements / task analysis needs. Online versions of knowledge support / job-aids have also been enabled. Additional performance support and blended-learning opportunities will be developed as more employees become LCMS-trained.
EPSS systems go beyond
providing information
“just in case” employees
need it.
K E Y P O I N T
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
The Learning Environment
In addition to the long-term technological e-learning goals, one learning objective for the IRS is to go beyond blended and asynchronous learning to online embedded performance support. In order to accomplish this goal, the education and learning employees in each of the business organizations have become performance consultants – who are expected to identify opportunities for learning that can be embedded in the work processes. Eventually, continuing education at the IRS will no longer be separate learning events that require employees to be removed from their tasks; rather, employees will be able to learn while engaged in productive work.
People
The IRS community has adapted well to the e-learning approach to training. Agencies working on the “go-learn” initiative within the IRS are currently joined with the OPM to develop standard performance measures. Throughout its continued interaction, OPM has noted that some agencies are doing things a bit differently at times and is working closely with each to ensure consistency. Outside of organizational confines, the e-learning approach has been well accepted by trainees. In fact, the OPM was astounded at the usage levels of e-learning within the IRS as compared to its other customers. In August of 2005 (a peak period for mandatory briefings), the learning management system (LMS) was overwhelmed due to the high rate of simultaneous user requests. (For more information on the LMS, see section, “Learning Management System.”) Even when the IRS is not at its peak briefing period, a variety of courses are scheduled with audiences ranging from 200 to 6,000 learners. In an effort to meet the expected capacity at peak periods, the IRS is taking proactive measures to increase system capacity.
An example of the benefits resulting from the use of the LCMS and reusable learning objects within the IRS is offered by the wage and investment organization – a group within the IRS that manages the call centers that tax-payers contact during tax season. The employees in this department constantly undergo review to guarantee quality
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Figure 3: IRS Single-Delivery Learning Modules (SDLMs) Process
assurance during tax filing season. This review involves both internal representatives listening in on customer calls and external representatives (e.g., the Office of the Inspector General and the Government Accountability Office) calling in with questions. These interactions are then rated and, as necessary, employees receive refresher training.
In the past, employees would have been taken off the phones for classroom-based refresher training; for the last three tax filing seasons, the IRS has been developing single-delivery learner modules (SDLMs). Occasionally, these SDLMs are combined to include several mini-topics housed within multiple courses with knowledge checks at the end. Based on identified needs, SDLMs are created and deployed within a span of a few weeks using the IRS templates (see Figure 3). This strategy keeps employees on the phones and improves the overall quality of customer service provided to the tax-payers.
Figure 3: IRS Single-Delivery Learning Modules (SDLMs) Process
Source: IRS, 2004.
IRSWage & Investment
Organization(WIO)
Call CenterQuality Assurance
Monitoring
DefineAgent Training
Need
Meta-TaggedLearning Objects
in LCMS
Create New Customized
SDLM
Deliver on-the-Job Training to
Individual Agent
Extract RelevantObjects for
Individual Agent Training
Benefits: Ease of Composition Reduced Development Time Delivered OJT Reduced Training Cost Knowledge Checks Available Targeted to Individual Employee Need
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Process
The adjustment to the objectives related to the IRS’s e-learning strategy has resulted in:
• Changes in how the IRS adapts to the tax code;
• Changes in policy mandated by Congress;
• Compliance with federal regulations; and,
• The approach to content development.
For instance, the IRC and IRM require regular updates that align with Congressional changes to the U.S. tax code. Without utilizing reusable content and meta-tagging, updating all training materials (e.g., job-aides, classroom training and online courses) affected by changes in the IRC requires an intensive effort to perform manually. Although the 12 business units within the IRS have been required to use templates (that are considered somewhat cumbersome), the implementation of the LCMS and reusable learning objects have ultimately reduced the level of effort required to make these content changes across the board.
In response to the IRS’s need to comply with federal regulatory agencies, both content developers and vendors have been offered extensive training to adjust to the LCMS implementation. Initially, the LCMS project manager forecasted that 100 employees would be trained on the system; it is expected that several hundred employees across the 12 business units will be trained to develop content.
The development of training content within the LCMS (in compliance to Section 508 of the Rehabilitation Act�0) is one area covered in this training. When the LCMS was initially introduced, the IRS had invested many hours of training with regard to Section 508 compliance. By examining every type of object that the LCMS creates, this training
10 Section 508 of the Rehabilitation Act requires access to the federal
government’s electronic and information technology by the employees and
the public to the extent it does not pose an “undue burden.” The law covers
all types of electronic and information technology in the federal sector when
federal employees develop, procure, maintain or use such technology. For more
information, please visit http://www.access-board.gov/sec508/summary.htm.
In response to the IRS’s
need to comply with
federal regulatory
agencies, both content
developers and vendors
have been offered
extensive training to
adjust to the LCMS
implementation.
BEST PRACT ICE
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
ensured that a template was created specifically for each function that would be used at the IRS. In addition, extensive testing was conducted to ensure that the entirety of Section 508 of the Rehabilitation Act, as well as the related requirements, was instituted. Instructions for content developers were created to assist them in remaining Section 508-compliant while creating and meta-tagging course content.
Finally, a series of documents were written that discuss the course development requirements (e.g., how the course should be written) and information mapping. In addition to the technology standards for course development (e.g., Section 508 compliance), the IRS has developed a number of internal standards. For instance, standards have been developed that address how the names or social security numbers of tax-payers are used within the training materials. In this case, the name of the county in which the taxpayer resides is used in training content instead of a name or social security number. If a social security number must be used, there is specific guidance within the course development guidelines regarding which social security numbers can be used.
Technology
A combination of technologies is being employed by the IRS to accomplish its technological and educational goals. These tools are intended to assist course developers – along with helping employees and managers who may use the system for training purposes. For instance, course developers are provided with a number of tools that were developed based on a task analysis. Available through the LMS and the HCO website, these tools can assist content developers in determining the most appropriate delivery method for the information. The training courses can be organized depending on delivery type, such as on-the-job training (OJT) or online courses.
Course developers also have the tools to conduct a cost-benefit analysis projection for course development to determine the feasibility of contracting a project outside the IRS. These tools also assist in determining when and where to schedule training (e.g., per diem, housing and travel costs). In addition to the commerce feature that tracks course development efforts, contract costs and travel costs, the
Course developers
have the tools to
conduct a cost-benefit
analysis projection for
course development to
determine the feasibility
of contracting a project
outside the IRS.
BEST PRACT ICE
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
IRS has customized the system to track account codes. Although this system is not directly linked to the IRS financial system, it is expected that these links will be established in the future.
Learning Management System
The LMS was initially deployed to meet the needs of the IRS’s legacy system (essentially a training scheduling system). Most of the functionality that was initially used within the LMS was for classroom administration; however, one of the major differences in the legacy system is that it was not possible to set up or assign curricula to anyone in a particular job assignment. Plateau offers an LMS that assists organizations in identifying training requirements (e.g., job-role assignment), as well as aiding in the program deployment enterprisewide. This program also tracks scheduling, administration and completion for regulatory compliance reporting purposes. In addition to the curricula being linked to specific job assignments, the LMS can be used to schedule phases of training (e.g., a series or sequence of courses).
LMS Features
A number of features have been deployed within the LMS, such as administrator registration, manager approval registration and self-service registering for classroom training. The LMS has also been used for trainee management and will eventually be used to target training needs based on the job assignments of individual employees. In addition, the credentials of all 20,000 internal instructors�� will be tracked.
One feature of this learning management system is the ability to track groups of learners over a period of time (i.e., class functionality) to manage the learning needs of the employees. In order to maintain
11 Although the IRS supports more than 20,000 internal instructors, these
instructors are not all full-time employees. In fact, technical specialists or subject
matter experts (e.g., insurance tax, banking specialist) may be hired at certain
times of the year on relevant topic areas (e.g., training on insurance tax-auditing
techniques).
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
accurate records, the LMS receives weekly updates from the personnel system (Oracle / PeopleSoft’s HRConnect). For example, the LMS could be used to distribute an update of emergency procedures for exiting a building for a particular office location. Using this system makes it possible to identify all IRS employees who work in that building and send out the relevant information to them. Similarly, when a course is updated or revised (e.g., a new tax code), the LMS assists the administrator in determining who is scheduled to take the courses. This technology also helps determine whether an employee can finish the current training with an older version of the course or if the updated course should be completed instead – as well as whether those employees who have previously taken the course must now take the updated course. Managers may also access the system to facilitate their workgroups to determine which of their employees are enrolled for coursework and what courses they will take. Prior to this functionality, a manager had to call the education department to determine who was enrolled for coursework and which courses were being offered. Using this feature within the LMS expedites a manager’s access to this information.
The IRS has also been using the LMS to track and manage training registrations through a variety of attendance, completion and withdrawal codes (e.g., family emergency leave). As a result, it is now possible to track no-shows and last minute withdrawals. This is important to business units as instructors are often hired to train a specified number of employees (e.g., two employees) for a pre-determined wage per student. A high no-show rate results in a loss of revenue for the business unit. In order to avoid additional indirect costs (e.g., time away from auditing or hiring additional vendors), every seat in the room must be filled. During 2002 to 2003, the average classroom attendance was 13 employees. Although policies regarding late withdrawals and no-shows have been put in place in some of the business units, the functionality that tracks attendance will further facilitate the development and enforcement of these policies.
One feature that has been particularly effective in improving the no-show rate is the notification feature. The LMS not only allows trainees to self-register, the system also notifies trainees of any additional materials and pre-training assignments that will be needed for the course. Late course registrations made the process of material delivery to multiple
The IRS has also been
using the LMS to track
and manage training
registrations through a
variety of attendance,
completion and
withdrawal codes.
K E Y P O I N T
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
sites (there are nearly 100 training sites) even more logistically and administratively complicated. Now, issues (such as determining how many copies of learning materials are needed or faxing materials for the next day) have been eliminated. Notifications have also been implemented to advise managers and employees if they have been placed on waitlists, as well as to remind them of scheduled training, including location information.
Another new and well-accepted LMS feature is the catalog, which manages the courses that are offered to each individual employee. For instance, within the IRS public catalog the courses that are available to all employees can be accessed, as well as course catalogs specific to a business unit. These catalogs also contain courses that can be shared across other IRS organizations. Additionally, subject matter area catalogs (e.g., human resources) can be created, so that subtopics of classes can be searched (e.g., personnel, labor relations or training). Occasionally, newly released courses are highlighted as featured courses within the employee’s individual accessible pages.
Employee training histories and course availability are not the only things being tracked using the LMS. In fact, the credentials of all instructors have been entered into the system (e.g., classroom instructor training, virtual classroom instructor training and leadership instructor training). This is an important capability within the system, given that the IRS supports more than 20,000 internal instructors for tax and leadership training, and requires its instructors to maintain certain credentials. For instance, if an instructor has not taught for a period of time, the IRS requires the instructor to take a refresher course. Through the use of this functionality, the task of tracking which instructors are available to teach and which require refresher courses has become much more manageable.
Results
The IRS went enterprisewide with the LMS in January 2005. By the end of the fiscal year (September 2005), they had delivered more than 800,000 learning events. Of that number, more then 500,000 of the events were delivered using e-learning. Although the initial goal of implementing the e-learning strategy was to reduce the portion of
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
the training budget spent on travel, reports indicate the percentage of the budget going toward travel has not changed; however, this is not considered a problem by the HCO because they are now delivering more learning than ever before – without an increase in the budget. For instance, all IRS employees (approximately 100,000 people) must attend five to six mandatory briefings annually over a period of three months every summer. These briefings cover such topics as ethics, security and equal employment opportunity (EEO). If everyone had to travel for these mandatory briefings, the training budget would increase exponentially. Thus, although classrooms continue to be used and continue to impact the training travel budget more training is being offered using approximately the same percentage of the overall training budget. This suggests that the overall technology architecture has provided some cost-containment, otherwise the budget would be much higher.
Although the IRS does not currently have traditional measures in place, due to a mandate from OPM the agency is beginning to identify more common forms of management criteria. It is expected that the IRS will partner with the OPM and other agencies to identify measures and metrics by which to examine performance and business results.
Next Steps
Blended-Learning Approach
IRS uses a blended-learning approach that includes a mix of online, virtual classroom, classroom, interactive video teletraining and downlink sites, as well as prerequisite assignments sent out by the instructor or the LMS. Eventually, the IRS plans to expand the training content development capabilities to all its business units.
In the future, the IRS will be upgrading to Plateau’s open suite LMS version and purchasing the separate performance module. This upgrade will provide capabilities for succession planning, individual development planning, 360-degree assessments and top-down goal
Between January and
September of 2005, the
IRS had delivered more
than 800,000 learning
events – 500,000 of which
were delivered using
e-learning.
K E Y P O I N T
�0BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
alignment��. One feature that will eventually be rolled out is the job assignment profile. This functionality tracks any changes that occur within each employee’s job assignment, so that the learning plan and curriculum for that individual is automatically updated and customized based on his / her new job tasks. Further, the system automatically assesses classes that have previously been completed to determine if collateral credit can be applied to the new learning plan.
The Future: HRConnect
The IRS does not plan to conduct performance appraisals through the LMS, as this function will be managed through the HRConnect system. However, the IRS does expect that there will be a need for some exchange of information between these two systems. Eventually, the LMS may also be linked to the IRS’s competency database�� to create a clear link between each course, the needed competencies and the critical level of skills needed for a job assignment. The IRS has also been working in conjunction with the OPM to create overall government standards for external training (e.g., conferences), as well as external training requests through the vendor.
12 “Top-down goal alignment” refers to cascading goals – such that the goals
of the top-level of the organization can be implemented into each level of the
company.13 This database contains the legally defensible competencies for mission-critical
occupations that were developed in collaboration with OPM.
��BERSIN & ASSOCIATES © JUNE 2007 • NOT FOR DISTRIBUTION • LICENSED MATERIAL
LARGE-SCALE E-LEARNING DEPLOYMENT
Appendix I: Table of Figures
Figure 1: IRS Organizational Chart 5
Figure 2: IRS Meta-Tagging Process 6
Figure 3: IRS Single-Delivery Learning Modules (SDLMs) Process 13
22bersin & associates © JUne 2007 • not for distribUtion • licensed material
large-scale e-learning deployment
About UsBersin & Associates is the only research and advisory consulting firm focused solely on WhatWorks® research in enterprise learning and talent management. With more than 25 years of experience in enterprise learning, technology and HR business processes, Bersin & Associates provides actionable, research-based services to help learning and HR managers and executives improve operational effectiveness and business impact.
Bersin & Associates research members gain access to a comprehensive library of best practices, case studies, benchmarks and in-depth market analyses designed to help executives and practitioners make fast, effective decisions. Member benefits include: in-depth advisory services, access to proprietary webcasts and industry user groups, strategic workshops, and strategic consulting to improve operational effectiveness and business alignment. More than 3,500 organizations in a wide range of industries benefit from Bersin & Associates research and services. Bersin & Associates can be reached at http://www.bersin.com or at (510) 654-8500.
About This ResearchCopyright© 2007 Bersin & Associates. All rights reserved. WhatWorks® and related names such as Rapid e-Learning: WhatWorks® and The High Impact Learning Organization® are registered trademarks of Bersin & Associates. No materials from this study can be duplicated, copied, republished, or re-used without written permission from Bersin & Associates. The information and forecasts contained in this report reflect the research and studied opinions of Bersin & Associates analysts.