Where people meet potential

Tools Supporting the Modernization and Reform of Assessing and Hiring Federal Job Candidates

There has been a heavy reliance on OQs in the past as a first step in the hiring process, although they have a very low-level of predictive validity. The OQs provide only an indirect assessment of competencies and are completely open to subjectivity – there are no right or wrong answers. Relying on self-reported applicant information also allows for potential bias in the assessment process.

Let’s look at an example of an OQ question, designed to measure candidate writing ability:

Occupational Questionnaire Example – Written Communication Skill Level

Indicate the extent that best describes your highest level of experience writing Technical Reports.

A. I have not had any education, training, or experience in performing this task.

B. I have prior education or training in performing this task but have not yet performed it in a work setting.

C. I have performed this task in a work setting, but only with close supervision from a supervisor or senior employee.

D. I have routinely and independently written technical reports as a regular part of my job, and usually without the need for review by a supervisor or senior employee.

E. I am highly skilled at writing technical reports in a work setting. I have trained others in this activity and/or others have consulted me as an expert for assistance.

All of the answers provided to the candidate are based on their personal perceptions, which can also allow for “gaming the system” or answering how you think the agency wants you to respond – which ultimately will not be a true representation of the candidate.

Alternative AssessmentsThere are many alternatives to Occupational Questionnaires, including skill- and competency-based assessments, logic and reasoning tests, personality assessments, and Situational Judgment Tests (SJTs) to name a few. Here is one example that clearly improves the ability to measure communication skills compared to the previous example:

Applicant Assessment Example – Written Communication Skill Level

Select the statement that contains no grammar, punctuation, or spelling errors.

A. The Chairman’s latest speach was his best.

B. The Chairmans’ latest speech was his best.

C. The Chairmans’ latest speach was his best.

D. The Chairman’s latest speech was his best.

E. None of the above.

As you can see, the Applicant Assessment questions provide a better determination of applicants’ skill level instead of asking applicants to self-rate themselves. The same is available for items contained in supervisory level assessments. Without measuring the candidates’ ability to demonstrate that they are actually able to recognize grammatical mistakes, solve problems through critical thinking, or make the right decision when dealing with conflict, there is really no way to know if they will be able to perform the job effectively. Applicant Assessments are an invaluable tool in that regard, as Occupational Questionnaires simply cannot provide those measurements.

Notable Industrial-Organizational

Psychologists agree – what is currently being used simply

isn’t working

IntroductionThe executive order issued in June of 2020, “Modernizing

and Reforming the Assessment and Hiring of Federal Job Candidates,” is poised to greatly change federal hiring

practices. A high-level official at the United States Office of Personnel Management (OPM), John York, stated that

the “Executive Order requires agencies to develop skill- and competency-based assessments for every job they

advertise in the competitive service.” He went on to say that the order represents “the biggest shift in federal hiring in

almost 40 years.”

The order i, includes the following mandates:

“Sec. 3. Improving the Use of Assessments in the Federal Hiring Process. (a) In addition to the other requirements of this order, the Director of OPM shall work with the heads of all agencies to ensure that, within 180 days of the date of this order,

for positions within the competitive service, agencies assess candidates in a manner that does not rely solely on educational

attainment to determine the extent to which candidates possess relevant knowledge, skills, competencies, and abilities. The heads

of all agencies shall develop or identify such assessment practices.

(b) In assessing candidates, agencies shall not rely solely on candidates’ self-evaluations of their stated abilities. Applicants must clear other assessment hurdles in order to be certified for consideration.

(c) Agencies shall continually evaluate the effectiveness of different assessment strategies to promote and protect the quality and

integrity of their hiring processes.”

Traditional Hiring PracticesNotable Industrial-Organizational

Psychologists agree – what is currently being used simply isn’t working. Traditional federal hiring has

usually consisted of an Occupational Questionnaire, followed by an interview

with a hiring manager. That approach is problematic for several reasons. Let’s

start with Occupational Questionnaires (OQ), which we like to call the “Traditional

Federal Assessment.”

i https://www.federalregister.gov/documents/2020/07/01/2020-14337/modernizing-and-reforming-the-assessment-and-hiring-of-federal-job-candidates

Applicant Assessments are an

invaluable tool

A Better ApproachFederal agencies have a wide variety of valid, well-respected, and widely used selection instruments. Moreover, many are part of established integrated hiring systems; this automates the assessment and improves the overall candidate experience. Automation supports both proctored and non-proctored assessment strategies. Implementing a solid strategy and using online assessment tools can assist in reducing applicant score inflation and provide a more reliable representation of candidate quality by measuring key job competencies and critical skills.

Additionally, companies like PSI, in partnership with Monster Government Solutions, and others already have established off-the-shelf assessment batteries that have been implemented for selection programs at the entry-level and supervisory levels at Civilian agencies and the DOD for years. These batteries are part of an integrated hiring system – meaning, the assessment platform is currently integrated with the agency’s hiring platform (ATS or hiring systems). The platforms provide seamless administration and scoring integration, allowing for separate scoring or scored in combination with existing assessments.

At many federal agencies, applicant assessments are typically completed as part of a “multiple hurdle” approach. This process helps eliminate unqualified candidates quickly and identifies applicants that meet the minimum requirements of the agency to progress in the process. For example, a common three-hurdle process might consist of the following steps:

1) Minimum qualifications through use of Occupational Questionnaires

2) An assessment battery

3) Structured interview

For step two, assessment batteries can be standardized or customized based on the specific needs of an agency. Standardized test batteries that target critical job requirements such as KSAs/competencies, job knowledge, and skills are readily available and more quickly implemented. Some agencies have developed their own content which can also be built into most testing platforms.

It is important to note that in the development and implementation of the assessment batteries, job analysis and test validation methodologies are implemented in compliance with legal guidelines and best practices. This provides the benefit of selection tools that:

• Align with job analysis results

• Reduce applicant score inflation

• Reduce time spent evaluating poor applicants

• Improve hiring decisions

Popular Assessment ExamplesHere are examples of two of the most widely used off-the-shelf assessments being employed at several federal agencies:

NON-SUPERVISORY ASSESSMENT BATTERY

• Competencies measured: oral communication, writing, interpersonal skills, problem solving, teamwork, technology orientation, customer service, attention to detail, self-management

• Appropriate for ACWA and non-ACWA positions. Battery includes three tests: personality, language skills, verbal reasoning

• Approximately 200 items and one-hour administration time

SUPERVISORY ASSESSMENT BATTERY

• Competencies measured: oral communication, interpersonal skills, problem solving, decision making, customer service, leadership, planning and evaluating, influencing/negotiating, strategic thinking, human capital management

• Battery includes three tests: personality, situational judgment, reasoning

• Approximately 200 items and 1.5-hour administration time

What sources were used to develop these two assessment batteries?In order for an assessment to be used during the selection process, there is a need to ensure validity. Test battery validation relies on a content validity approach, stemming from job analysis studies and publicly available data. This provides for assessment instruments that are proven, effective, and legally defensible. For the aforementioned assessments, here are examples of institutions that participated in the studies that were conducted:

NON-SUPERVISORY OCCUPATIONAL DATA SOURCES

• Prior MGS job analysis results (30+ studies across 11 Cabinet-level agencies)

• OPM MOSAIC studies

• OPM PMF Competency Model

• OPM HR University

• Secretary’s Commission on Achieving Necessary Skills (SCANS) – Department of Labor

SUPERVISORY OCCUPATIONAL DATA SOURCES

• Prior MGS job analysis results (16+ studies across 8 Cabinet-level agencies)

• OPM Leadership and Supervisory Competency Models

• OPM Management Effectiveness Inventory

• Leadership Competency Inventory (Penn State University)

• Various agency-specific leadership competency models

There is a need to ensure validity

Connect with an expert today.

ConclusionThis Executive Order provides an opportunity for agencies to better measure a candidate’s ability to perform on the job, period. When we focus on candidates’ competencies and not their education as a primary evaluation factor, we open opportunities for individuals who might not have the same educational background but do have the skills and/or abilities to perform well. As such, there are readily available, proven assessment solutions that agencies can implement in a reasonable time period that are solid, defensible, and secure with respect to candidate and assessment data.

psionline.com/talent

PSI Services LLC

@PSIServicesLLC