Select™ for Retail Banking (v3) - OutMatch for... · 2018-10-02 · Select for Retail Banking...

Select for Retail Banking (v3) Technical Manual

Select™ for Retail Banking (v3) (Including Branch Banking Manager, Banking Sales Associate, In-Store Banking

Sales Associate and Banking Service Associate)

Technical Manual

Prepared by: Assess Systems


Copyright,1996‐2012Bigby,Havis&Associatesd/b/aAssessSystems.Allrightsreserved2

Table of Contents Overview ....................................................................................................................................................................... 4

General Background ......................................................................................................................................... 4

What Is Select For Retail Banking (v3)............................................................................................................. 4

Who Is It For? .................................................................................................................................................. 5

What Does It Measure ...................................................................................................................................... 5

Reading the Report .................................................................................................................................................... 7

What Do The Various Indices Mean? ............................................................................................................... 7

What Are the Select for Branch Manager (v3) Dimensions? ........................................................................... 8

What Are the Select for Banking Sales Associate (v3) Dimensions? .............................................................. 9

What Are the Select for In-Store Banking Sales Associate (v3) Dimensions? ................................................ 11

What Are the Select for Banking Service Associate (v3) Dimensions? ........................................................... 14

What Does It Mean When A Performance Subscale Is Flagged? .................................................................... 14

What Is The Job Tasks Section? ....................................................................................................................... 14

What Is The Experience Checklist Section? ..................................................................................................... 15

What Are The Counterproductive Behaviors? ................................................................................................. 16

What Is The Positive Response Pattern? ........................................................................................................... 16

How Do I Use The Interview Probes And/Or Structured Interview? ............................................................... 17

What Is the “Making the Hiring Decision” Chart? .......................................................................................... 18

What Are Development Suggestions? ............................................................................................................. 18

Using the Report ......................................................................................................................................................... 20

Making The Hiring Decision ............................................................................................................................. 20

Selection Recommendations ............................................................................................................................ 29

Who Should See the Select Report ................................................................................................................... 21

Validation Research ................................................................................................................................................... 22

General Background ........................................................................................................................................ 22

Development of Retail Banking Assessments .................................................................................................. 24

Select for Banking Sales Associate ................................................................................................................... 24

Select for In-Store Banking Sales Associate ..................................................................................................... 31



Select for Branch Manager ................................................................................................................................ 33

Select for Banking Service Associate ................................................................................................................ 39

Select for Retail Banking (v3) ................................................................................................................................ 44

Adverse Impact Analysis with Applicants ............................................................................................................ 45

Adverse Impact Approaches ............................................................................................................................. 45

Adverse Impact for Branch Manager ................................................................................................................ 46

Adverse Impact for Banking Sales Associate .................................................................................................... 49

Adverse Impact for In-Store Banking Sales Associate ...................................................................................... 51

Adverse Impact for Banking Service Associate ................................................................................................ 53

Adverse Impact for Banking Branch Manager Math Module ........................................................................... 55

Adverse Impact for Banking Math Module ....................................................................................................... 56

Summary ...................................................................................................................................................................... 57

Recommendation of Use ................................................................................................................................... 57

Caution Regarding Interpretation ..................................................................................................................... 57

Next Steps: Monitor Pass Rates ....................................................................................................................... 57

Reference .......................................................................................................................................................... 57

Appendix ...................................................................................................................................................................... 58

Sample Test Items ............................................................................................................................................. 59

About Assess Systems ...................................................................................................................................... 66



Overview

General Background

For over 20 years the organizational psychologists and consultants of Assess Systems™ have been involved in the design, development, and deployment of tailored selection systems for entry-level and para-professional roles across a wide variety of industries. In each case, as part of the project, we were asked to develop and criterion-validate work personality tests to aid in the selection of high potential candidates.

While in each validation study a custom selection tool was developed to meet the unique aspects of a particular job, it became increasingly apparent that these instruments shared many work-personality dimensions. From this research, we developed the foundation for the Select test batteries. Each Select survey measures both those common dimensions predictive of performance in most associate-level jobs, as well as unique dimensions specific to the particular job for which it has been validated. Below is an overview of common dimensions across Select surveys.

Select Dimensions Include:

Positive Attitude - Shows faith in self and positive outlook on self, others, and the company

Energy - Prefers a high activity level; action oriented

Frustration and Criticism Tolerance - Appears emotionally optimistic and resilient

Acceptance of Diversity - Tolerates others different from self

Self Control - Refrains from impulsive behavior

Acceptance of Structure - Accepts rules and follows direction

Empathy, Interpersonal Insight - Shows an understanding for the feelings and needs of others

Multi-Tasking - Enjoys handling multiple tasks simultaneously

Integrity - Possesses a strong work ethic

Influence - Enjoys influencing the actions of others

Sociability - Displays an ease within a social context and enjoys interacting with others

Good Judgment - Shows a tendency towards objective thinking based on fact

The Select Associate Screening System is a family of related tests designed to measure important, work-related personality characteristics in applicants for entry-level jobs. It allows employers to quickly and easily screen for characteristics associated with productivity, cooperative work behaviors, and integrity. The following pages provide an overview and validation research for the Select for Retail Banking surveys. For a full listing of other surveys available in the Select Associate Screening System, please see page 66 of this manual.

What Is Select For Retail Banking (v3)?

Select for Retail Banking (v3) is a group of personality-based surveys designed to measure characteristics that have been found to predict job effectiveness for positions in retail banking work environments.

Through our research in retail banking, we have identified several key personality characteristics that contribute to job success. Select for Retail Banking (v3) allows you to hire managers who will lead a team of banking sales associates, emphasize overall branch performance, juggle multiple demands, effectively lead and coach others, and demonstrate the drive to succeed. For banking sales associates, it allows you to hire employees who are sales



oriented, will work cooperatively, and will positively interact with customers. Select for Retail Banking (v3) will also help you to screen out applicants with low integrity and those who are poorly suited for these types of roles.

Who Is It For?

The Select for Retail Banking (v3) suite has been validated for use with people who work both in traditional, stand-alone branches and in-store branches that offer a sales-oriented, cross-functional approach to service and sales with customers.

Select for Banking Sales Associates (v3) is appropriate for those associates who not only handle daily customer transactions, but who are responsible for selling retail banking services such as loans, investments, checking services, etc., and who are provided incentives for promoting products and services. (For Teller positions, see Select for Banking Service Associates which is an adapted version of our customer service survey.) Select for In-store Sales Associates (v3) is appropriate for those associates who work as a sales associate within an in-store banking environment. (For Teller positions, see Select for Banking Service Associates which is an adapted version of our customer service survey.) Select for Branch Managers (v3) is appropriate for those managers who manage the daily operations of the branch and who are also required to promote customer services. These managers work in an environment where they are given responsibility for making decisions regarding customer services, are provided incentives for the performance of their branch, and are required to work closely with their team in order to be successful. Select for Banking Service Associates (v3) is appropriate for service only positions, such as Tellers, and is an adapted version of our Select for Customer Service, which includes a standard customer service performance indicator, and customized modules for retail banking (willingness to do job tasks, interview, and retail banking associate math module).

What Does It Measure?

Standard Modules

Performance and Integrity: Select for Retail Banking (v3) measures personality characteristics related to effective job performance in retail banking roles. The two central indices that are reported in each survey are performance and integrity. Each Performance Index has been validated to predict the traits associated with successful performance in the specific role. The Integrity Index has been validated to predict the attitudes, personal integrity, and work ethic required in most associate and entry-level manager roles. Three surveys have been developed to meet the specific needs within a retail banking environment. A fourth survey, appropriate for Tellers in pure service-only positions, is adapted from our standard customer service survey. Below is a listing of these surveys.

Select for Banking Sales Associates

Select for In-store Sales Associates

Select for Branch Managers

Select for Banking Service Associates



Optional Modules

The following are optional modules offered within the Select system. These modules may be added to your survey or customized to fit your needs. Contact your human resource department or Select distributor for more information.

Retail Banking Math: Two retail banking-focused modules are available: Retail Banking Associate Math and Retail Banking Manager Math. These modules indicate a measure of an individual’s ability to solve math problems relevant to the retail banking environment. Both modules include items around basic arithmetic, percentages, financial word problems, making change and number matching. The Retail Banking Manager Math module also includes items that require candidates to analyze tables and charts relevant to understanding branch sales reports. Job Tasks (Associate Roles Only): The job task module is an online interview which asks the candidate about his or her willingness to perform important tasks related to the job. This module includes questions related to retail banking environments, such as a candidate’s willingness to handle difficult customers, suggest other services, aisle sell to customers (in-store only), etc. Depending upon the survey, these job tasks are tailored to each of the key banking associate roles: sales, in-store and service. Job Experience Checklist (Branch Manager Only): This module is an online interview which asks the candidate about his or her prior experience performing important managerial job activities and functions. This module includes question related to retail banking branch environments, such as a candidate’s experience reviewing and responding to branch performance reports, etc. Counterproductive Behaviors: An additional module available with Select for Retail Banking is the Counter-productive Behaviors module (CPB). There are two versions of this module, attitude-based or admissions-based. Both of these are assessments of counterproductive behaviors and include topic areas of drug use, theft, job commitment, work ethic, resistance to direction, safety, etc. Attitude-based questions ask individuals about their beliefs while admissions-based questions ask individuals about prior use or experience (these are relevant to drug use and theft). Caution: Admissions-based questions relating to drug use and theft are prohibited in some states/territories. Check regulations particular to your legal jurisdiction before implementing.



Reading The Report

The Select for Retail Banking reports provide information to help facilitate the selection of top performing candidates. The reports are designed to provide information regarding a candidate’s personality characteristics and highlight areas that may merit attention in the interview process.

What Do the Various Indices Mean?

Random Response Index:

The Select for Retail Banking (v3) surveys contain a validity check to ensure that the candidate is not responding to the items randomly. Special items in the survey determine if the applicant is correctly reading the survey and sufficiently attending to the

survey items. Inappropriate answers to these statements will result in a survey which is “invalid.” This may indicate that: 1) the applicant has limited language or reading skills, 2) the applicant did not carefully read the survey, or 3) the applicant may need visual accommodation. We recommend that you attempt to determine the reason for this invalid result and, if necessary, re-survey the applicant providing them with assistance or accommodation.

Retail Banking Math Index:

The Retail Banking Math Index is a measure of basic math calculations and basic numerical reasoning skills relevant to the retail banking environment. If the candidate receives a score in the “Needs Training” category, he/she may have difficulty with these types of calculations on the job without additional training.

Integrity Index:

The Integrity Index measures the candidate’s attitude related to personal integrity and work ethic. High scores indicate a positive work ethic and low scores indicate a tendency to “get by” and not “give it their all.” While not all applicants with scores in the AVOID range will have low integrity, our research suggests that many will. Pursuing applicants who score in the GOOD range

should increase your chances of hiring someone with a positive work ethic. (Example is from Select for Banking Service Associate (v3).)

Performance Index:

The Performance Index measures those personality characteristics that differentiate good retail banking candidates from poor ones. Those scoring in the higher ranges on the Performance Index (OK and GOOD/BETTER) are more likely to possess the important traits

necessary to perform well on the job. Try to avoid hiring applicants with scores in the AVOID range. (Example is from Select for Branch Managers (v3).)

Random Response

Valid Invalid

X

Math Index

16

Needs Training Okay

X

0 - 14 15 - 18

Integrity Index Score:

9

Avoid Good

X

0 - 7 8-13

Performance Index

35

Avoid OK Good Better

X

0 - 30 31 - 32 33- 36 37-43



What Are The Select for Branch Managers (v3) Dimensions?

The following table lists personality dimensions found to consistently predict performance in these jobs. These dimensions describe the characteristics which make up the Performance and Integrity Indices.

Select for Branch Managers (v3) Dimensions

Performance Index Undesirable Scores Desirable Scores Representative

Behaviors

Leadership: Inclination to coach and lead

others

Little preference to coach and lead others

Comfortable in a coaching and leadership role

Entrepreneurial Spirit – must run their branch as their own business, take initiative to put together branch business plans

Responsibility: Takes personal responsibility;

self-motivated

May hesitate to take personal ownership and responsibility

Comfortable taking personal ownership and responsibility

Takes pride/ ownership in success of branch

Preference for Objective Measures:

Preference for objective performance measures

Not comfortable being evaluated on objective performance measures

Comfortable being evaluated on objective performance measures

Analytical

Can review and understand monthly branch performance reports and identify trends, issues, etc. – can translate this into individual goals

Flexible Thinking: Can think beyond “black-or-

white”, is not a rigid thinker and can interpret information

More comfortable making decisions when facts are available

Comfortable making decisions based on experience and intuition, when facts are not available

Is big picture oriented

Can conceptually see how pieces fit together, how to operate their branch to meet regional/bank goals, etc

Energy: Activity level, action orientation

Slow pace, low stamina or low energy reserve

Energy, drive to get the job done.

Self-starter

Enjoys an upbeat, fast-paced environment

Interpersonal Influence: Can influence others in a sales

role

May be uncomfortable asserting oneself in a sales-oriented environment

Comfortable asserting oneself in a sales-oriented environment

Possesses strong persuasion, selling skills

Social Comfort: At ease with others; enjoys social

interaction

May be uncomfortable interacting with people they do not know

Enjoys talking to and interacting with others

Builds strong relationships within the community, shopping centers, businesses, etc.

Resilience: Ability to handle frustration and

pressure

Sensitive to frustration Resilient: not easily upset. Upbeat outlook

Can work effectively during high periods of stress or customer activity

Integrity Index Undesirable Scores Desirable Scores Representative

Behaviors

Integrity/Work Ethic: Positive outlook on working hard

and earning one's pay

“Get by” attitude; negative attitudes about work and the company

Enjoyment of work; work integrity; honest effort

Good work ethic

Hard worker

Dependable; on time every time



What Are The Select for Banking Sales Associate (v3) Dimensions? The following table lists personality dimensions found to consistently predict performance in these jobs. These dimensions describe the characteristics which make up the Performance and Integrity Indices.

Select for Banking Sales Associate (v3) Dimensions


Behaviors


role



Can effectively influence and persuade clients or customers to increase sales

Positive Service Attitude:

Is positive about other people and the service role

Defensive, negative outlook

Positive outlook on self, others and the service role

Positively influences others

Customer service oriented

Is perceptive about people and is motivated to help them




Self-starter

Has the energy that is needed to work at a steady pace when required.

Actively puts in effort

Social Comfort: At ease with others; enjoys social

interaction



Comfortable approaching and interacting with customers

Accommodation to Others: Desire to accommodate to the

needs of others

May be uncooperative Enjoys serving others and working cooperatively

Recognizes and understands customer needs

Delivers service that exceeds customer expectations

Dependability: Follows through on tasks;

conscientious

May not always follow through on all tasks

Dependable and follows through

Can be counted on to keep commitments

Is reliable


pressure

Sensitive to frustration Resilient: not easily upset.

Upbeat outlook

Stays positive and motivated in the face of difficulty, stress, or rejection

Process Oriented: Can work within ordered

processes and procedures; is organized

Uncomfortable with process and procedures

Enjoys work environments with process and procedures

Is careful and detail-oriented

Does not make careless errors

Keeps workspace and work tasks organized to improve work efficiency





Follows procedures and accepts direction



Acceptance of Diversity: Tolerance of others different from

self

Impatient

Annoyed with others who are different from oneself (e.g., foreigners or people poorly informed about products or services).

Cooperative

Tolerant of others

Treats every customer and coworker equally


Behaviors





Good work ethic

Hard worker




What Are The Select for In-Store Sales Associate (v3) Dimensions? The following table lists personality dimensions found to consistently predict performance in these jobs. These dimensions describe the characteristics which make up the Performance and Integrity Indices.

Select for In-Store Sales Associate (v3) Dimensions


Behaviors


role



Can effectively influence and persuade clients or customers to increase sales



Defensive, negative outlook Positive outlook on self, others and the service role


Consistently displays a positive attitude about the customer, their sales role and the job




Self-starter

Has the energy that is needed to work at a steady pace when required.

Actively puts in effort

Social Comfort: At ease with others; enjoys

social interaction



Comfortable approaching and interacting with customers


needs of others




Dependability: Follows through on tasks;

conscientious

May not always follow through on all tasks

Dependable and follows through

Can be counted on to keep commitments

Is reliable


pressure

Sensitive to frustration Resilient

Not easily upset

Upbeat outlook


Process Oriented: Can work within ordered

processes and procedures; is organized

Uncomfortable with process and procedures

Enjoys work environments with process and procedures

Is careful and detail-oriented

Does not make careless errors

Keeps workspace and work tasks organized to improve work efficiency





Follows procedures and accepts direction



Acceptance of Diversity: Tolerance of others different

from self

Impatient

Annoyed with others who are different from oneself (e.g., foreigners or people poorly informed about products or services).

Cooperative

Tolerant of others

Treats every customer and coworker equally.

Multi-Tasking: Ability to handle multiple tasks at

one time

Difficultly in “switching gears” when under stress;

Narrow task focus

Flexible

Handles multiple tasks at one time

Can effectively handle competing priorities and juggling multiple tasks without becoming flustered

Self Reliance: Handles tasks independently with little support from others

May seek assistance or assurance from others before completing tasks, even when they do not need it

Is likely to work independently

Should complete tasks with little assistance or guidance from others

Knows when to solve the problem themselves and when to ask for help

Takes the initiative to get things done, learn new things, and take on additional responsibilities or tasks.

Does not wait for others to provide them with direction


Behaviors





Good work ethic

Hard worker




What Are The Select for Banking Service Associate (v3) Dimensions? The following table lists personality dimensions found to consistently predict performance in these jobs. These dimensions describe the characteristics which make up the Performance and Integrity Indices.

Select for Banking Service Associate (v3) Dimensions


Behaviors

Energy:

Activity level, action orientation


Energy, drive to get the job done

Self-starter

Maintains a high activity level for an extended period of time

Performs consistently repetitive tasks and remains effective

Frustration Tolerance:

Resilience when faced with frustrations and irritations as

opposed to feeling their effects

Sensitive to frustrations Resilient

Not easily upset.

Upbeat outlook



needs of others




Acceptance of Diversity: Tolerance of others different

from self.

Impatient, annoyed with others who are different from oneself (e.g., foreigners or people poorly informed about products or services).

Cooperative

Tolerant of others

Treats every customer and coworker equally.



Defensive, negative outlook

Positive outlook on self, others and the service role


Consistently displays a positive attitude about the customer and their role


Behaviors

Integrity/Work Ethic: Positive outlook on working hard and earning one's pay



Good work ethic

Hard worker




What Does It Mean When A Performance Subscale Is Flagged?

In addition to providing an overall Performance Index score, Select for Retail Banking (v3) surveys also provides subscale information on each of the characteristics that make up the Performance Index. For each subscale, the candidate’s scores will fall into the OK or FLAG category. If a candidate scores unfavorably on a subscale, there will be an “X” next to that subscale under the FLAG heading. For example, if Suzanne Example responded to statements related to Energy in an unfavorable way, there would be an “X” next to Energy on her report. This “Flagged” subscale means that Suzanne may have difficulty maintaining the sense of urgency needed to quickly fulfill customer requests. At the end of the report, suggested interview questions pertaining to the flagged subscales are provided. (The example below is from the Banking Sales Associate (v3) survey.)

It is important to remember that although an applicant’s overall Performance Index score may fall in the OK or GOOD/BETTER range, he or she may still have a low score on one or a few of the subscales. This indicates that, while he or she may exhibit most of the characteristics identified as important, the applicant may have some weaknesses which can affect overall performance. Any flagged subscales should be followed-up in the interview to confirm whether or not a low score will be problematic.

What Is the Job Task Section? (Optional Module)

The Job Task section is an optional module for the Banking Sales Associate, In-store Sales Associate, and Banking Service Associate survey. This section asks about the candidate’s willingness to do tasks related to a retail banking environment as well as willingness to work shifts and weekends. Areas where the candidate expresses reluctance should be probed further in the interview. (These items can be customized for your organization.) The following table is from the Banking Sales Associate (v3) survey.

How willing are you to … Would do it and

enjoy it Would do it

Would do it, but not like it

Would not want to do it

Would not do it

Work weekdays? X

Work weekends? X

Commit to being on time, every time? X

Be required to meet sales quotas? X

Adjust work schedule on short notice? X

Keep branch space organized and tidy? X

Subscale OK Flag

Interpersonal Influence X

Positive Service Attitude X

Energy X

Social Comfort X

Dependability X

Resilience X

Process Focused X

Preference for Objective Measurement X

Acceptance of Diversity X

Accommodation to Others X



Serve or assist customers? X

Handle cash transactions? X

Answer telephone inquiries? X

Handle customer complaints? X

Promote services to customers? X

Operate computer and software systems?

X

Make cold calls to current and potential customers?

X

Help implement marketing promotions? X

Make service recommendations to customers?

X

Contact potential customers to explain the variety of banking services you

offer?

X

Work in an environment that requires you to rely on and work closely with co-

workers as part of a team?

X

The table above reports the candidate’s stated willingness to do tasks commonly required in jobs similar to this one. Indications of reluctance should be probed during the interview. Note: Example is from the Select for Banking Sales (v3).)

What Is the Experience Checklist Section? (Optional Module)

The Experience Checklist is an optional module for the Branch Manager survey and asks about the candidate’s experience conducting tasks related to managing a branch, such as managing associates, organizing promotions, etc. Areas where the candidate indicate moderate to extensive experience should be probed further in the interview. (These items can be customized for your organization.)

How much experience do you have ... None Minimal Moderate Extensive

Hiring new associates? X

Disciplining associates? X

Firing associates? X

Setting work schedules? X

Implementing sales promotions? X

Training staff? X

Doing direct sales? X



Coaching others to sell? X

Handling teller transactions? X

Handling customer complaints? X

Maintaining site appearance? X

Doing administrative tasks? X

Prospecting for new customers? X

Reviewing and responding to branch performance reports?

X

Working cooperatively with others? X

Being required to meet sales quotas? X

The table above reports the candidate’s stated experience with common branch management tasks. If he/she has moderate or extensive experience in an area, ask for more specific information during the interview.

What Are Counterproductive Behaviors? (Optional Module)

The Counterproductive Behaviors module will highlight certain items within the Select for Retail Banking (v3) surveys if they are answered in an undesirable way. The items within the attitude-based and admissions-based versions of this module are related to attitudes and behaviors regarding drug use, theft, safety, and similar topic areas. One should use this information to further investigate potential problem areas during the interview or through reference checks.

In this section, undesirable responses by the application to drug use, theft, job commitment, work ethic, resistance to direction, etc. questions are presented. The total number of survey questions for each topic is given in parentheses. The candidate selected an undesirable response or answered agree to the following:

Drug Use (5 possible questions)

Which best describes your current involvement with illegal drugs (for example: speed, cocaine, marijuana, heroin, etc.)? Frequent, but I don’t let it interfere with my work.

Using marijuana illegally during work…. Has no effect on me.

What Is The Positive Response Pattern?

Within the Select for Retail Banking (v3) surveys there are statements designed to determine if an individual is trying to present themselves in a more positive manner, or put their “best foot forward.” If these items are answered in an overly positive manner, a special warning will be produced on the report. In these instances, a special effort should be made to verify the results of the report with other information such as interview impressions and reference checks.



Positive Response Pattern

Use Scores Obtained From This Survey Cautiously

This applicant answered the survey in a positive manner. Such positive responses may suggest one or more of the following:

1. He is very positive in his view of himself, others and life in general.

2. He deliberately tried to present himself in a positive manner.

3. He does not have much self-insight and may see few personal weaknesses or areas for self-improvement.

The possibilities listed above should be carefully considered, and a special effort should be made to verify with other information such as additional interview impressions or reference checks.

How Do I Use The Interview Probes And/Or Structured Interview?

The Select report provides interview probes or an interview guide to aid in conducting a structured, behavioral interview that is job relevant. The interview guides were developed specifically for the financial sales associate and branch manager roles. The guide may be tailored to include your company's custom interview.

Interview Probes

If a candidate flags any of the Performance sub-scales, the Select Report will include follow-up interview probes. These are presented to assist the interviewer in asking questions about potential liabilities or characteristics which may merit additional attention and consideration. In some instances, an individual may have developed a strategy to compensate for particular weaknesses. For example, a candidate who scores in the low range on the measure of Frustration Tolerance may become easily discouraged or overly sensitive to job frustrations. However, she may have developed a strategy for using stress management techniques to help maintain her calm and optimism. The interview probes included in the report can help to confirm or deny whether a potential liability exists.

Sample Interview Probe

Low Process Focus

This person appears to prefer a work environment that is free of procedures, routine and predictability. Once the newness of the job wears off, he/she may be easily bored with or fail to consistently perform the more routine aspects of the job. Alternatively he/she may resist following standard procedures.

Interview Probe:

Look for a tendency in his/her work history to change jobs frequently. Prove for the reasons behind each job change.

Ask him/her to describe the type of work he/she likes best and least. Listen for indications that he/she desires more freedom and variety in his/her work than this job may provide.

After describing the standard duties and requirements of the job, ask him/her to describe the parts he/she will like and the parts he/she many not like. Listen for indications that he/she may find the job too monotonous or that he/she may have difficulty following standard procedures and methods.

Probe suggestions allow the interviewer to determine:

To what extent is the characteristic likely to be observed in day-to-day job performance?

What impact (if any) does the characteristic have on job performance, work relationships, problem solving, etc.?



To what extent, or by what method, does the person “manage” or control the behavior

Structured Interview (Optional)

A detailed interview guide will be printed in the back for the Select for Retail Banking reports. This guide will include:

A script to open and close the interview.

A series of job-related interview questions with guidance as to what to listen for in the candidate’s responses.

Interview Probes based on survey results (see above).

Rating Worksheet to help evaluate the candidate’s overall fit for the role.

By using this structured interview guide consistently (meaning you ask the same questions of every candidate), you will bring increased objectivity to the process.

Sample Structured Interview Questions:

While asking the following interview questions, be sure to listen and probe in the following areas highlighted by Select:

Low Energy Level

Low Frustration Tolerance

Tell me about your responsibilities in your previous (or current) job. Which did you like most and why? Also, which did you like least and why? (Listen for relevant work experience and likes and dislikes that may or may not fit this job.)

What were your most significant accomplishments or successes in this (these) jobs? (Listen for work ethic, values, commitment.)

Why are you considering leaving or why did you leave your current/last job? (Listen for reliability, job fit.)

What is the “Making the Hiring Decision” Chart?

The Select report also provides users with a hiring decision chart that identifies key steps in the selection process (e.g. reviewing the application, prescreen, test results, background checks, etc.) and a chart to document notes and overall recommendations. The chart can serve as an efficient way of consolidating notes/observations from the selection process to assist in making the hiring decision.

What Are Development Suggestions?

As an optional resource to assist in on-boarding your new hire, a link for developmental suggestions will be provided at the end of the Select report. Developmental suggestions will only appear for performance sub-scales that are flagged from the assessment. The suggestion will highlight the flagged sub-scale and provide guidance on how to further develop in this area, including on-the-job activities. Before using these, please contact your organization’s Human Resource department to ensure this is an option your company is using as part of their process. These suggestions should not be provided to applicants who were not selected.



Sample Development Suggestion:

Low Energy/Work Pace

A review of your answers to the assessment suggests that your work style is likely to be methodical and slow-paced. If you need to improve in this area, consider the following suggestions:

Make a “to do” list of your tasks at the beginning of each day. Rank them in order of importance and focus on the highest priority ones first.

Set deadlines for completing a task and stick to these. Sometimes it also helps to make a time schedule for each and every step involved

If you find yourself lacking energy during the day, try to determine why and address the cause: Do you get enough sleep or exercise? Is there something on your mind that is draining your energy?

Don’t jump around among several unfinished tasks. It’s easier to keep your focus when it isn’t divided. Finish the most important tasks and then be purposeful in your intent to complete the rest.



Using The Report

Making the Hiring Decision

As you use Select in your hiring process, please keep the following in mind:

Select results should be one voice in the selection process. Recommendations from Select should be combined with evaluations of work experience and education, and results from interviews with hiring managers, to make a final decision.

Select should be an aid to the well-reasoned judgment of the hiring manager, not a replacement for this judgment

Use the interview suggestions. It may be that a candidate has learned how to manage or compensate for certain weaknesses, and using the probes found in the report may help the interviewer to determine whether or not a particular weakness will have an impact on the individual's job effectiveness.

Selection Recommendations

Using Select for Branch Managers (v3)

Using an “Avoid the AVOIDs” strategy for both Integrity and Performance, 22% of applicants will be screened out as being poor “fits” for the job without introducing significant adverse impact (see Adverse Impact table on page 45). Using this criteria for screening out candidates will reduce interviews with applicants who are likely to be a poor job fit and will improve your selection decisions. To be more selective in your selection process, a “Raising the Bar” strategy may be used. This strategy is a more stringent screening process in that it raises the cutoff scores on the Performance Index to only include individuals in the GOOD/BETTER ranges so that approximately 30% of applicants will be screened out without introducing significant adverse impact. Therefore, this strategy imposes a higher standard in screening, but is dependent upon a sufficient applicant flow.

Using Select for Banking Sales Associates (v3)

Using an “Avoid the AVOIDs” strategy for both Integrity and Performance, approximately 20% of applicants will be screened out as being poor “fits” for the job without introducing significant adverse impact (see Adverse Impact table on page 48). Using this criteria for screening out candidates will reduce interviews with applicants having a poor job fit and will improve your selection decisions.

To be more selective in your selection process, a “Raising the Bar” strategy may be used. This strategy is a more stringent screening process in that it raises the cutoff scores on the Performance Index to only include individuals in the GOOD/BETTER range so that approximately 34% of applicants will be screened out without introducing significant adverse impact. Therefore, this strategy imposes a higher standard in screening, but is dependent upon a sufficient applicant flow.

Using Select for In-store Banking Sales Associates (v3)

Using an “Avoid the AVOIDs” strategy for both Integrity and Performance, 22% of applicants will be screened out as being poor “fits” for the job without introducing significant adverse impact (see Adverse Impact table on page 50). Using this criteria for screening out candidates will reduce interviews with applicants having a poor job fit and will improve your selection decisions.



To be more selective in your selection process, a “Raising the Bar” strategy may be used. This strategy is a more stringent screening process in that it raises the cutoff scores on the Performance Index to only include individuals in the GOOD/BETTER range so that approximately 34% of applicants will be screened out without introducing significant adverse impact. Therefore, this strategy imposes a higher standard in screening, but is dependent upon a sufficient applicant flow.

Using Select for Banking Service Associates (v3) (adapted from Select for Customer Service)

Using an “Avoid the AVOIDs” strategy for both Integrity and Performance, approximately 17% of applicants will be screened out as being poor “fits” for the job without introducing significant adverse impact (see Adverse Impact table on page 52). Using this criteria for screening out candidates will reduce interviews with applicants having a poor job fit and will improve your selection decisions. To be more selective in your selection process, a “Raising the Bar” strategy may be used. This strategy is a more stringent screening process in that it raises the cutoff scores on the Performance Index to only include individuals in the GOOD range so that approximately 27% of applicants will be screened out without introducing significant adverse impact. Therefore, this strategy imposes a higher standard in screening, but is dependent upon a sufficient applicant flow.

Who Should See The Select Reports?

Because of the nature of appraisal information and the dangers of its misuse, the Select report must be kept confidential and its contents restricted to those who have direct responsibility for decision making. The Select reports SHOULD NOT be shown to the individual nor contents discussed with him or her. The results of Select belong to the hiring organization. The Statement of Informed Consent, designed to notify the candidate of the above, is included in both the on-line and paper and pencil surveys. The candidate must read and agree to this Statement of Informed Consent before completing the test. (Note: When entering paper and pencil results, a Select report should not be processed unless the Statement of Informed Consent has been signed by the applicant -- first page of the survey or answer sheet.)



Validation Research

General Background

The following section will present approaches for developing and validating pre-employment tests and provides a detailed description of the method used in developing and validating the Select surveys. (Additional sections of this manual will present research results for the development and validation of the Select for Retail Banking surveys.) Assess Systems validation research has shown Select traits to predict performance (i.e., the criterion) across a number of customer service, sales, and entry-level manager jobs. There are two main approaches to criterion-related validation studies: predictive and concurrent designs.

Predictive

In a predictive strategy, candidates are tested before they are hired, and performance measures are collected later, after the candidates have been hired and on the job long enough to accurately assess their job performance.

Concurrent

In a concurrent strategy, the relationship between predictor instruments and criterion job performance is verified by testing a sample of current employees. The test performance of these employees is correlated with measures (existing performance evaluations or special purpose ratings) collected “concurrently” or in the same general time period as test performance. A concurrent validation strategy is most often chosen because of its relative efficiency and to allow the test to be used for selection sooner. The flowchart on the next page shows the main work steps in a concurrent validation study, and a brief description of each step follows.

Below are brief definitions of the main work steps:

Job Analysis: Job analysis is the process whereby the knowledge, skills, abilities, and personality factors (KSAPs) necessary for effective job performance are identified. Through focus groups with job content experts (often supervisors, trainers, and top performing employees), desired attributes, as well as those characteristics that lead to poor performance, are surfaced.

Desired Attribute Dimensions: Through job analysis, a list (or lists) of the desired KSAPs for a specific position is generated. This list is consolidated and kept as a record of the job relatedness of the developed and validated selection test.

Preliminary Test Battery: The list of desired attributes helps in the selection of test items and scales which could potentially be used to identify top performing job candidates. These items and scales are the content of the preliminary test battery which is later validated with the incumbent sample.

Job Analysis

Identify Desired Attribute Dimensions

Assemble Preliminary Test Battery

Choose Sample

Administer Test Battery

Collect Job Performance Measures

Statistical Analysis(Validity / Adverse

Impact / Utility)

Final Test Battery



Choose Sample: In order to “test the test,” a large, representative sample of current employees is asked to complete the preliminary battery. This sample is ideal when made up of both good and poor performers and when there is diversity in terms of ethnicity, age, and gender.

Administer Test Battery: The preliminary test battery is then given to the incumbent sample. It is important that employees know that they are helping to “test the test” and that none of their responses will be used for anything other than research purposes.

Collect Job Performance Ratings: Concurrently with administering the test battery, incumbent job performance measures are collected. Specially designed rating forms are used and supervisors are taught how to rate performance as objectively as possible. In some studies, truly objective data can be collected; for example, Select has been validated against individual call revenue in an airline reservations job.

Statistical Analysis: Analyses are completed to determine the relationship between job performance measures and test results. The primary measure of this relationship is the correlation coefficient. Correlations vary from -1.0 to +1.0.

+1.0 = perfect positive relationship (high test score = high performance.)

0 = no relationship.

-1.0 = perfect negative relationship (high test score = low performance.)

p-value (probability value) indicates level of statistical significance - at a minimum, this should be smaller than .05. (.05 means that the probability of the observed result occurring by chance is 5 out of 100.)

Expected range of correlation coefficients in this type of study: .2 to .4.

After the best predicting scales are combined to form the Performance Index, the indices are analyzed for adverse impact. If adverse impact is found, indices are often modified to ensure the rate of selection does not work to the disadvantage of members of an ethnic, gender, or age group.

Final Test Battery:

Once the best items and scales have been analyzed and selected, a final test battery can be designed. This process generally involves finalizing scoring software and paper and pencil tests.



Development of Retail Banking Assessments

Assess Systems has been working across a variety of service and sales-oriented industries since the late 1980s to identify those candidates who will be successful in roles that require customer interaction, growing sales, leading a team, and working in a team-oriented environment. Across these various validation studies, correlations between predictor indices and job performance (ratings of service and sales behaviors by supervisors and objective sales data) have ranged between .28 and .47.

Drawing from this experience, Assess Systems began working with financial institutions to develop job specific surveys for the retail banking environment. While custom surveys were initially developed for in-store banking purposes, we subsequently completed a broader analysis of the data, across both in-store and traditional branch samples, to develop and validate the generic Select for Retail Banking suite of products. The following section describes the broader research studies that led to the generic surveys.

Study One: Development and Validation of the Banking Sales Associate Survey (Performance Index and Math Module)

Working with a retail banking consulting firm, Assess Systems enlisted the help of four different regional retail banking brands to develop the Select for Banking Sales Associate Survey. Each of these brands included small branch (in-store) and larger, traditional branch locations. Through job analysis, a list of attributes desired in the role was developed. This list, in turn, guided the selection and development of a preliminary survey battery which was used as part of a criterion-related, concurrent validation study. This battery contained scales previously validated in other service and sales oriented industries as well as experimental scales and items.

Validation Samples:

Two representative samples of incumbents were used in the validation analysis. These samples were drawn from across various retail banking branches and regions. The following describes some of the characteristics of the sample:

Four different regional, retail banking brands

Sales-orientation across all branches, from front line associates to managers

Locations across the Southeast and Northeast regions of the U.S.

Regional banking brands, later acquired by a large national banking institution

Physical branch size: o Sample one: small-size, in-store branches o Sample two: mid to large-size, traditional stand-alone branches

In both samples, current associates completed the preliminary survey. At the same time special performance ratings were collected from regional managers and/or branch managers and raters were assured that their ratings were being collected for research purposes only.

In both samples we collected ratings from regional managers regarding each participant’s overall performance, sales performance and potential for promotion.

In the small branch sample we also collected ratings from direct supervisors (branch managers) on these same overall criteria.

In the larger traditional branch sample we were able to collect additional ratings regarding sales and work behaviors such as customer service, initiative, etc.



After reviewing the ratings collected from both regional managers and branch managers in the small branch sample it was determined that we would use the regional manager ratings as the primary criterion measure. This determination was made for the following reasons:

Regional managers were the common rater group between both samples.

A greater number of participants were rated by the regional managers than the branch managers allowing for a larger validation sample.

The ratings provided by regional managers showed better variability in ratings and better differentiation between top and bottom performers in the small branch sample.

# of Employees

Rated

Overall Performance

Sales Performance

Potential for Promotion

Mean SD Mean SD Mean SD

Supervisor ratings 79 3.43 1.01 3.05 1.02 3.30

1.81

Regional Manager Ratings

148 2.88 .99 2.88 1.05

2.85

1.06

Ratings ranges from “1” = Poor, “5” = Good After matching survey and regional manager performance data and removing individuals with random response patterns the final data set was 128 for the small branch sample and 82 for the traditional branch sample. Demographics for both samples are presented in the table below:

Demographics of Validation Samples:

Gender Sample One

Small Branch Sample Two

Traditional Branch

Male 47 22

Female 79 54

Prefer Not to Say 2 6

Total 128 82



Ethnicity Sample One


Traditional Branch

African American 18 9

Caucasian 99 61

Hispanic 0 3

American Indian/Alaskan native 3 1

Asian/Pacific Islander 1 0

Other 1 1


Total 128 82

Age Sample One


Traditional Branch

0-19 1 0

20-29 83 17

30-39 21 15

40-49 7 23

50-59 9 18

60 or over 2 2


Total 128 82

Validity Analysis – Banking Sales Associate Performance Index

Using the regional manager ratings as criteria and the incumbent personality data, various statistical analyses were then conducted to determine the best predictor set of test items and scales in the validation sample. This best predictor set was then analyzed for adverse impact to ensure no potential bias. The resulting performance (work characteristics) index is significantly correlated with key measures of job performance in both samples. Expected correlations for a study of this type are between .20 to .40. Correlations Between Performance Criteria and Performance Index for Banking Sales Associate Studies

Criterion Rating Sample One Small Branch

(n=128)

Sample Two Traditional Branch (n=82)

Overall Performance .40** .34**

Sales Performance .31** .36**



Potential for Promotion .42** .39**

Customer Service (--) .40**

Results-Orientation (--) .27*

Decision-Making (--) .25*

Initiative (--) .27*

* Correlations significant at p< .05 ** Correlations significant at p< .01; (--) indicates that the criterion rating was not gathered in that study

The Performance Index was then divided into recommended ranges (Avoid, Okay, Good and Better) and mean differences in performance were examined. The graphs below show the mean performance ratings for each of the Banking Sales Associate cut-off scores on the Performance Index for each of the samples. In each case the differences between the Avoid categories and the Okay/Better categories were significant at p<.05.

Difference in performance means based on Performance Index Recommendation

Sample One: Small Branch Associates Sample Two: Traditional Branch Associates


Sample One: Small Branch Associates Sample Two: Traditional Branch

0

0.5

1

1.5

2

2.5

3

3.5

4

Overall Performance

2.67

3.23.57

Avoid

Okay

Better

0

0.5

1

1.5

2

2.5

3

3.5

4

Sales Performance

2.62

3

3.57

Avoid

Okay

Better

0

0.5

1

1.5

2

2.5

3

3.5

4

Overall Performance

2.8

3.93 4

Avoid

Okay

Better

0

0.5

1

1.5

2

2.5

3

3.5

4

Sales Performance

2.98

3.68 3.7

Avoid

Okay

Good



Associates

Validity Analysis – Banking Math Module

During the job analysis phase of the retail banking research studies, a sample of job-related math items were developed to measure the types of math related problems banking sales associates and branch managers are required to solve on the job. The appropriateness of these items to the content of the job was confirmed by job content experts. The following are examples of the types of items that are included:

Sample Basic Math Ability Items:

1. $12, 476 x 7.8% =

2. 1,298 is what percentage of 1,500?

3. A customer has a starting balance in his checking account of $567.46. He has a check to deposit into his checking account for the amount of $1,547.52, and wants $250 cash back. What is the balance of his checking account after this transaction?

4. Your bank runs a promotion for businesses that open checking accounts for their employees. It offers a 50% reduction of the $7.00 per person set-up fee if more than 5 employees join. If a business has 7 employees, how much would they pay in set-up fees?

These items were administered as part of the preliminary survey completed by job incumbents in both samples. At the same time criterion ratings of math ability were collected from managers. Various statistical analyses were conducted to determine the best predicting set of math items. This set was then analyzed for adverse impact to ensure no potential bias. The resulting set was found to be correlated with manager ratings of overall performance and math ability.

0

0.5

1

1.5

2

2.5

3

3.5

4


2.63

3.23.62

Avoid

Okay

Better

2.3

2.4

2.5

2.6

2.7

2.8

2.9

3

3.1


2.56

3 3.01

Avoid

Okay

Good



Correlations Between Performance Criteria and Banking Sales Associate Math Index

Criterion Rating

Sample One

Small Branch

(n=81)

Sample Two

Traditional Branch

(n=70)

Math Ability .23* .21*

Overall Performance .18* .19*

Potential for Promotion .12 .10

* Correlations significant at p< .05

The Math Index was then divided into recommendation ranges and (Needs Training and Okay) and mean differences in performance were examined. The graph below shows the mean performance ratings for each of the two ranges on the Math Index for each sample. In each case differences were significant at p<.01.

Difference in performance means based on Math Index Recommendation Sample One: Small Branch Associates Sample Two: Traditional Branch Associates


3.2

3.3

3.4

3.5

3.6

3.7

3.8

3.9

Math Ability

3.45

3.82

Needs Training

Okay

2.7

2.75

2.8

2.85

2.9

2.95

3

3.05

Overall Performance

2.83

3.03

Needs Training

Okay

3.4

3.5

3.6

3.7

3.8

3.9

Math Ability

3.57

3.9

Needs Training

Okay

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Overall Performance

3.01

3.45

Needs Training

Okay




2.78

2.8

2.82

2.84

2.86

2.88

2.9

Potential for Performance

2.83

2.9

Needs Training

Okay

2.8

2.9

3

3.1


2.9

3.1

Needs Training

Okay



Study Two: Tailoring of Retail Banking Associate Survey Performance Index for In-Store Sales Associates

As part of the job analysis for the Retail Banking Survey, Assess Systems was given the opportunity to examine a derivative of this environment – in-store banking. During the job analysis it became apparent that there were some conditions unique to the in-store banking environment which might require customization of the performance index. For example:

In-Store Banking Environment Traditional Retail Banking Environment

Smaller, tighter work environment Physically more spacious; greater freedom

In-store associate is required to juggle more types of transactions and tasks

Specific job duties and transactions (e.g., account set-up, teller, investments, etc.)

No on-site specialists, which requires the in-store associates to be more independent

More supervision by management

Requires more versatility and multi-tasking More focused

More out-of-branch selling (aisle selling) Destination customers

While the Banking Sales Associate survey predicts success with in-store associates (see Study One, Sample One), Assess Systems found through subsequent analysis that the predictions for this somewhat unique environment could be improved by tailoring the survey. By re-examining the in-store associate sample data only, we found that the addition of two scales, Self-Reliance and Multi-Tasking improved the predictiveness of the Performance Index. Inclusion of these scales was further supported by the job analysis in this unique environment. In-store associate roles are less specialized and the job requires greater initiative and versatility on the part of the candidate.

Correlations Between Performance Criteria and the In-Store Sales Associate Performance Index

Criterion Rating In-Store Sales Associate

Index (n=128)

Overall Performance .47**

Sales Performance .40**

Potential for Promotion .44**

** Correlations significant at p< .01


The In-Store Associate Performance Index was then divided into recommended ranges (Avoid, Okay, Good and Better) and mean differences in performance were examined. The graphs below show the mean performance ratings for each of the In-store Sales Associate cut-off scores on the Performance Index for the sample. In each case these differences were significant at p<.01.

The In-Store Sales Associate Survey was tailored to reflect the in-store environment:

Customized Willingness To Do Job Tasks

Customized Structured Interview Questions

0

0.5

1

1.5

2

2.5

3

3.5

4

Sales Performance

2.65

3

3.55

Avoid

Okay

Good

0

0.5

1

1.5

2

2.5

3

3.5

4

Overall Performance

2.69

3.33 3.55

Avoid

Okay

Good

0

0.5

1

1.5

2

2.5

3

3.5

4


2.65

3.333.65

Avoid

Okay

Good



Study Three: Development and Validation of the Branch Manager Survey (Performance Index and Branch Manager Math Module)

Assess Systems has been working with a variety of service and sales-oriented industries since the late 1980s to identify those candidates who will be successful in supervisory management roles that required leading a small to medium sized team of sales-oriented associates, efficiently operating a retail store, managing resources and growing sales. Assess Systems was asked to build upon this foundation to validate a tool specific to retail banking branch managers. Working with a retail banking consulting firm, Assess Systems enlisted the help of four different regional retail banking brands to develop the Select for Banking Branch Manager Survey. Each of these brands included small branch (in-store) and traditional banking locations. Through job analysis, a list of attributes desired in the role was developed. This list, in turn, guided the selection and development of a preliminary survey battery which was used as part of a criterion-related, concurrent validation study. This battery contained scales previously validated in other retail management roles as well as experimental scales and items.

Validation Samples: The validation samples consisted of a representative sample of incumbents across various retail banking branches and regions. The following describes some of the characteristics of the sample:

Four different retail banking brands

Sales-orientation across all branches, from front line associates to managers

Locations across the Southeast and Northeast regions of the U.S.

Regional banking brands, later acquired by a large national banking institution

Physical branch size: o Study one: small-size, in-store branches o Study two: mid to large-size, traditional stand-alone branches

In both samples, current branch managers completed the preliminary survey and at the same time special performance ratings were collected from their direct supervisors (regional managers.) In all cases raters were assured that their ratings were being collected for research purposes only and were guided to avoid potential rating errors. After matching survey and performance data and removing individuals with random response patterns the final data set was 152 for the small branch sample and 76 for the traditional branch sample. Demographics of the samples are presented in the following table:

Gender Sample One


Traditional Branch

Male 52 22

Female 95 54


Total 152 76



Ethnicity Sample One


Traditional Branch

African American 21 9

Caucasian 122 61

Hispanic 0 3

American Indian/Alaskan native

0 1

Asian/Pacific Islander 2 0

Other 2 1


Total 152 76

Age Sample One

Small Branch

Sample Two

Traditional Branch

20-29 68 17

30-39 50 15

40-49 22 23

50-59 6 18

60 or over 1 2


Total 152 76

Validity Analysis – Banking Branch Manager Performance Index

Various statistical analyses were conducted to determine the best predictor set of test items and scales in the validation sample. This best predictor set was then analyzed for adverse impact to ensure no potential bias. The resulting work characteristics are statistically and significantly correlated with key measures of job performance. (Expected correlations for a study of this type are between .20 and .40.)


Correlations Between Performance Criteria and Performance Index for Branch Manager Study

Criterion Rating Sample One

Small Branch (n=152)

Sample Two Traditional Branch

(n=76)

Overall Performance .37** .33**

Sales Performance ..40** .34**

Potential for Promotion .36** .31**

Customer Service .20* .27*

Managerial Skills/Team Leader .31* .28*

Commitment to Results .18 .40**

Decision Making/Analytic Skills .22* .19

Initiative (--) .38**

Work Ethic .36** (--)

Coaching & Developing Others .22* (--)

Commitment to Results .22* (--)

* Correlations significant at p< .05 ** Correlations significant at p< .01 (--) indicates that the criterion rating was not gathered in that study

The Performance Index was then divided into recommended ranges (Avoid, Okay, Good and Better) and mean differences in performance were examined. The graphs below show the mean performance ratings for each of the Branch Manager cut-off scores on the Performance Index for the sample. In each case these differences were significant at p<.01. Difference in performance means based on Performance Index Recommendation

Sample One: Small Branch, Managers Sample Two: Traditional Branch, Managers

3.1

3.2

3.3

3.4

3.5

3.6

3.7

Overall Performance

3.36

3.65 3.7

Avoid

Okay

Good

0

0.5

1

1.5

2

2.5

3

3.5

4

Overall Performance

3.21

3.88 3.98

Avoid

Okay

Good





Validity Analysis – Banking Branch Manager Math Module

During the job analysis phase of the retail banking research studies, a sample of job-related math items were developed to measure the types of math related problems banking sales associates and branch managers are required to solve on the job. The appropriateness of these items to the content of the job was confirmed by job content experts. The following are examples of the types of items that are included:

Sample Branch Manager Math Item:

March April May June

Total Customers 270 274 248 256

Total Deposits $76,091 $75,888 $66,381 $68,321

Total Loans $17,091 $17,502 $14,571 $15,814

Table BM2

Based on Table BM2, in what month was the average deposit per customer the highest?

0

1

2

3

4


3.05

3.82 3.88

Avoid

Okay

Better

0

1

2

3

4

Sales Performance

3.25

3.61 4

Avoid

Okay

Good0

1

2

3

4

5

Sales Performance

2.86

3.84.13

Avoid

Okay

Good

3.3

3.4

3.5

3.6

3.7

3.8

3.9


3.55

3.85 3.89

Avoid

Okay

Better



These items were administered as part of the preliminary survey completed by job incumbents in the sample one (small branch) study. At the same time criterion ratings of math ability were collected from supervisors. Various statistical analyses were conducted to determine the best predicting set of math items. This set was then analyzed for adverse impact to ensure no potential bias. The resulting set was found to be correlated with manager ratings of overall performance and math ability.

Correlations Between Performance Criteria and Branch Manager Math Index:

* Correlations significant at p< .05

The Math Index was then divided into recommendation ranges and (Needs Training and Okay) and mean differences in performance were examined. The graph below show the mean performance ratings for each of the two ranges on the Math Index. In each case these differences were significant at p<.01. Difference in performance means based on Math Index Recommendation

Sample One: Small Branch Managers

Criterion Rating Sample One

Small Branch (N=107)

Math Ability .22*

Overall Performance .20*

Potential for Promotion .18*

3.35

3.4

3.45

3.5

3.55

3.6

Overall Performance

3.43

3.58

Needs Training

Okay

3.5

3.55

3.6

3.65

3.7

3.75

Math Ability

3.58

3.73

Needs Training

Okay



33.05

3.13.15

3.23.25

3.33.35

3.43.45

3.53.55


3.18

3.51

Needs Training

Okay



Study Four: Development of the Banking Service Associate Survey (Adapted from Select for Customer Service)

Since the mid-1980s, the organizational psychologists and consultants of Assess Systems have been involved in the development of tailored selection systems for client companies. In recent years, these systems were increasingly focused on the selection of candidates for associate-level jobs. In each case, as part of the project, we were asked to develop and validate work personality tests. These projects were conducted in a variety of industries. While in each validation study a custom selection tool was developed to meet the unique aspects of a particular job, it became increasingly apparent that these instruments shared in common many work-personality dimensions. Candidates who possess these personality characteristics are more apt to provide excellent service to others and to extend themselves to meet the customer’s needs. These characteristics form the foundation for the Select for Customer Service Survey and its variants (Select for Hospitality, Select for Receptionists and Select for Personal Service). The Select for Customer Service Technical Manual describes the body of research (six studies) which has lead to the development of our customer services surveys. In each of these studies, a concurrent validation study was conducted whereby participants completed a preliminary survey which included items measure work ethic/integrity, and at the same time supervisors provided ratings of performance. In the initial study various statistical analyses were conducted to determine the best predictor set of test items and scales in the validation sample. This best predictor set was then analyzed for adverse impact to ensure no potential bias. The resulting work characteristics and Performance Index are statistically and significantly correlated with key measures of job performance. In subsequent studies the Performance Index was cross-validated against measures of job performance. (Expected correlations for a study of this type are between .20 and .40). Across studies, our research found that a core set of personality characteristics (positive service attitude, energy, accommodation to others, acceptance of diversity, frustration tolerance and integrity) predicted successful job performance across these service jobs. In some studies we also found that the addition of a sociability component improved the predictability of the survey, while in others, job requirements supported the use of a math module. The Select for Customer Service Technical Manual provides a detailed summary of each of these studies, while the table below presents the correlations of the Performance against the key measure of overall job performance in each study. Correlations Between Performance Criteria and the Customer Service Performance Index

Study One

Airline Reservation

Agents (n=1085)

Study Two

Security Monitoring Personnel

(n=127)

Study Three

Retail Clerk/Cashier

(n=79)

Study Four

Retail Personal Services (n=104)

.34 .26 .28 .22

All correlations significant at p<.05

Further tailoring is also reflected in the Job Task portion of the survey and the Structured Interview where the content has been modified to more closely reflect the activities and behaviors associated with specific roles.



Tailoring of the Select for Customer Services Survey for Retail Service Associates

The Select for Banking Service Associate Survey is another variant to the Select for Customer Service, Survey. Like all of the Select for Customer Service surveys, the Select for Banking Service Associate Survey will allow you to hire associates who will work cooperatively, exceed customer expectations and foster new and repeat business. It will also help you to identify those applicants with low integrity and who are poorly suited for a customer service role. The Select for Banking Service Associates has been tailored for retail banking environments in the following ways:

Customized Willingness to do Job Tasks that reflect teller-oriented environment

Customized Structured Interview questions



Study Five: Development and Validation of the Integrity Index Used in the Select for Retail Banking Suite of Surveys

All of the Select surveys involving customer facing roles (e.g., Select for Customer Service, Select for Retail and Select for Call Centers) including Select for Retail Banking share a common Integrity Index that was developed and cross-validated in several service oriented roles. The integrity items were developed to assess a candidate’s tendency to “get by” and not to “give it their all” or take advantage of the company.

In each of these studies, a concurrent validation study was conducted whereby participants completed a preliminary survey which included items measure work ethic/integrity, and at the same time supervisors provided ratings of performance. In the initial study various statistical analyses were conducted to determine the best predictor set of test items to measure the integrity construct. This best predictor set was then analyzed for adverse impact to ensure no potential bias. The resulting Integrity Index statistically and significantly correlated with key measures of work ethic and honest. In subsequent studies the Integrity Index was cross-validated against measures of work ethic and honesty. (Expected correlations for a study of this type are between .20 and .40).

The Select for Customer Service Technical Manual provides a detailed summary of each of these studies, while the table below presents the correlations of the Integrity Index against the key measure of overall job performance in each study.

Correlations Between Work Ethic/Honesty Performance Criteria and the Integrity Index

Study One

Convenience Store Employees

(n=416)

Study Two

Retail Clerk/Cashier

(n=79)

Study Three

Security Monitoring Personnel

(N=127)

Study Four

Personal Services Retail Chain

(N=104)

.24 .39 .38 .15

All correlations significant at p<.05



Utility

Utility estimates the potential impact of implementing a selected procedure. The tables and charts provided below highlight the type of performance difference found in current samples when the Select test is implemented. Based on the research sample, using the Select for Retail Banking Performance Indices to screen-out those scoring in the AVOID range should improve the selection process. Below are utility results found for each of the Retail Banking tools.

For Banking Sales Associates:

60% of poor performers were screened out by the Select survey, while improving the selection of top performers by 54%.

80% of poor performers were screened out at the Avoid and Okay range, while only 8% of poor performers score in the Better range.

Applying the “Avoid the AVOIDs” practice, poor performers were reduced by 60%.

Similarly, comparisons of those recommended by the survey (Okay and Good) vs. those not recommended (Avoid) show that the survey results clearly distinguish between those rated as Star Performers by managers (ratings of 5) and those rated as Poor Hires (those rated as 1 or 2) by their managers.

Chances of being successful in the role based on survey recommendation:

Avoid Okay Good

15% 30% 70%

For Branch Managers:

64% of poor performers were screened out by the Select survey, while improving the selection of top performers by 28%.

84% of poor performers were screened out at the Avoid and Okay range, while only 15% of poor performers score in the Better range.

Applying the “Avoid the AVOIDs” practice, poor performers were reduced by 50%

Similarly, comparisons of those recommended by the survey(Okay and Good) vs. those not recommended (Avoid) show that the survey results clearly distinguish between those rated as Star Performers by managers (ratings of 5) and those rated as Poor Hires (those rated as 1 or 2) by their managers.

0%10%20%30%40%50%60%70%80%90%

Poor Hires Star Performers

81%

10%16%

30%

3%

60%

Avoid

Okay

Good



Chances of being successful in the role based on survey recommendation:

Avoid Okay Good

35% 60% 75%

0%

20%

40%

60%

80%

Poor Hires Star Performers

Avoid

Okay

Good



Select for Retail Banking (v3)

An extensive review was made on the applicant database for Retail Banking cases gathered to date. Pass rates and adverse impact were analyzed for the survey and when using the “Avoid the AVOIDs” strategy, the survey was within EEOC guidelines. However, recalibrations of cut-scores at the scale and at the Performance Index level were now possible to allow for additional selectivity when screening candidates. Below are screenshots of the updated performance indices by survey.

Select for Banking Branch Managers version 3.0

Performance Index Score:

32

Avoid Okay Good Better

X

0-30 31-32 33-36 37-43

Select for Banking Sales Associate version 3.0


42


X

0-35 36-38 39-44 45-48

Select for In-Store Banking Sales Associate version 3.0


46


X

0-41 42-44 45-50 51-55

Select for Banking Service Associate version 3.0


25


X

0-21 22 23-24 25



Adverse Impact Analysis with Applicants

Adverse Impact Analysis Approaches

Adverse Impact (AI) basically refers to a substantially different rate of selection in hiring, promotion or other employment decisions which works to the disadvantage of members of a race, sex or ethnic group (Uniform Guidelines Questions & Answers #10). Adverse Impact Analysis is particularly important to employers who are governed by the regulations of the Office of Federal Contract Compliance Programs (OFCCP) however it is equally applicable under Equal Employment Opportunity Commission (EEOC) who also investigates claims of discrimination in the workplace. As specifically defined by Federal Contract Compliance Manual (FCCM), Chapter 7, Adverse Impact is defined as: “Definition: Adverse impact is a substantially different rate of selection in hiring, promotion, transfer, training or in other employment decisions which works to the disadvantage of minorities or women. If such rate is less than 80 percent of the selection rate of the race, sex, or ethnic group with the highest rate of selection, this will generally be regarded as evidence of adverse impact. Adverse impact analyses based on the 80% rule may be buttressed by a test of statistical significance. For further discussion, see Section 7E06.” Measurement of AI generally falls under two approaches a) evaluation of practical significance or more commonly termed the Four-Fifths Rule as referenced by the OFCCP in their Definition of AI and b) tests of statistical significance.

Practical significance is a legal term that refers to a difference that would be considered substantial by a reasonable person. There is no precise test for practical significance. The 80% Test or Four-Fifths Rule was originally adopted as a test of practical significance. The 80% Test essentially says that adverse impact is suggested if the pass rate of a focus group (e.g. women) is less than 80% of the pass rate of the majority group (e.g. men). Put a different way; is there more than a 20% difference in the pass rates of the two groups? The 80% Test is still used as a useful indicator of practical significance. However, one must look at other factors in combination with the 80% rule – such as the sample size, stability of the trend over time, etc. In some cases, a difference of more or less than 20% may be considered to be insignificant in practice.

Statistical significance refers to a difference that is substantial enough that it is unlikely to have occurred by random chance. A variety of tests exist for examining statistical significance with the two most commonly used being the 2 standard deviation (SD) test, also called the Z test, and Fisher’s exact test (FET). Both approaches examine the relationship between two variables to determine whether a difference in employment decision rates is likely due to chance. Most often the OFCCP has used Z tests (the 2 standard deviation test) as the primary measure of adverse impact particularly for large samples while the Fisher’s exact test (FET) is used in small samples (sample sizes of less than 30).

The reason that the courts have generally applied both statistical and practical significance standards in cases of alleged adverse impact is that statistical significance tests alone can be misleading in some circumstances. In particular, sample size and composition have a substantial impact on tests of statistical significance. A statistically significant finding of adverse impact in a small sample may be suspect because these results tend to be unstable and may not replicate over time. Likewise, in large samples, even very small differences that are insignificant in practice will meet the threshold of statistical significance. The following Table displays the difference in passing rates, or Impact Ratio, that is necessary to attain statistical significance for a given sample size. The example makes several assumptions (footnoted below) for the sake of simplicity.



Table 1: Minimum Significant Pass Rates*

Sample Size (n) Minimum Significant Pass

Rate Difference Minimum Significant

Impact Ratio

5000 3% 0.97

1000 6% 0.94

500 8% 0.92

100 19% 0.81

50 27% 0.73

* Using Fisher Exact (2-tail), assuming equal division of sample between focal and majority group and a majority pass rate of 75%.

The example in Table 1 highlights two important points. First, even very small differences in pass rates will result in a finding of statistically significant adverse impact when the sample size is large. For instance, with a sample size of 5000, a difference of just 3% will result in a finding of statistical significance even through the impact ratio under the 80% is well within guidelines. Second, even a large difference in pass rates would not result in a finding of statistical significant adverse impact when the sample is small. For instance, with a sample size of 50, a difference of 27% or more is needed to result in a finding of statistical significance. Note that this level of difference would result in a violation of the 80% Test (IR < 0.80) without reaching statistical significance. As Biddle (2006) points out, “under most circumstances…analysis should result in a firm finding of both statistical and practical significance to constitute a solid finding of adverse impact.” In addition, according to Biddle (2006) “…when practices, procedures, or tests show signs of adverse impact (or may show adverse impact if greater sample sizes were evaluated), the employer should either consider conducting validation studies or using alternative employment practices that have less adverse impact.”

Adverse Impact Analysis – Banking Branch Managers

To examine the issue of adverse impact for the Select for Banking Branch Managers survey, selection impact ratios for the Performance Index were estimated using a sample of applicant data collected for the standard Select assessment (n= 278). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.

Candidate results were categorized as follows:

Performance Index: Avoid (0-30), Okay (31-32) and Good/Better (33-43).

Integrity Index: Avoid (0-7) and Good (8-13)



Adverse Impact Analysis

Analysis was completed on each of the individual indices and scoring recommendations (Performance and Integrity) to review for adverse impact. As can be seen in the upper portion of the table, no significant adverse impact was found for the individual indices (Performance and Integrity) for ethnicity or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Avoiding the Avoids”. “Recommendation 1” in the tables indicates the candidate scored in the Okay/Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 78.3% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. In the second analysis of Adverse Impact we examined Recommend rates using a more stringent screening criterion on the Performance Index. Under this scenario “Recommendation 2” indicates the candidate scored in the Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 69.8% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.



Percent of Candidates Passing Select for Banking Branch Managers

Index All Applicants

(n=278)

White

(n=211)

Minority

(n=57) Ratio*

Men

(n=134)

Women

(n=140 Ratio*

Performance Index

0-30 Avoid -- -- -- -- -- -- --

31-32 Okay 85.9% 85.2% 90.9% 1.07 83.0% 86.2% 1.04

33-43 Good/Better 76.1% 75.4% 83.8% 1.11 72.8% 78.5% 1.08

Integrity

0-7 Avoid -- -- -- -- -- -- --

8-13 Good 93.2% 91.5% 98.2% 1.07 93.3% 92.9% 1.00

Combined**

Avoid -- -- -- -- -- -- --

Recommendation 1:

PI: Okay/Good/Better, I: Good 78.3% 75.9% 87.2% 1.15 75.6% 80.2% 1.06

Recommendation 2:

PI: Good/Better, I: Good 69.8% 68.1% 79.5% 1.17 67.4% 71.3% 1.06

Note: Impact Ratios between .80 and 1.20 are considered acceptable under EEOC Guidelines. Recommendation 1 indicates the candidate scored in the Okay/Good/Better range on the Performance Index and the Good range on the Integrity Index. Recommendation 2 indicates the candidate scored in the Good/Better range on the Performance Index and the Good range on the Integrity Index.



Adverse Impact Analysis – Banking Sales Associates

To examine the issue of adverse impact for the Select for Banking Sales Associates survey, selection impact ratios for the Performance Index were estimated using a sample of applicant data collected for the standard Select assessment (n= 6775). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.





Analysis was completed on each of the individual indices and scoring recommendations (Performance and Integrity) to review for adverse impact. As can be seen in the upper portion of the table, no significant adverse impact was found for the individual indices (Performance and Integrity) for ethnicity or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Avoiding the Avoids”. “Recommendation 1” in the tables indicates the candidate scored in the Okay/Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 79.9% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. In the second analysis of Adverse Impact we examined Recommend rates using a more stringent screening criterion on the Performance Index. Under this scenario “Recommendation 2” indicates the candidate scored in the Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 66.1% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.



Percent of Candidates Passing Select for Banking Sales Associates


(n=6755)

White

(n=3144)

Minority

(n=3454) Ratio*

Men

(n=2475)

Women

(n=4225) Ratio*

Performance Index

0-35 Avoid -- -- -- -- -- -- --

36-38 Okay 67.1% 81.6% 80.2% .98 82.3 79.8 .97

39-48 Good/Better 80.7% 68.6% 65.8% .96 69.4 65.7 .95

Integrity

0-7 Avoid -- -- -- -- -- -- --

8-13 Good 94.9% 96.3% 93.6% .97 94% 95.4% 1.01

Combined**

Avoid -- -- -- -- -- -- --

Recommendation 1:

PI: Okay/Good/Better, I: Good 79.9% 80.3% 78.1% .97 80.2% 78.4% .98

Recommendation 2:

PI: Good/Better, I: Good 66.1% 67.9% 64.5% .94 68.2% 64.8% .95




Adverse Impact Analysis – In-Store Banking Sales Associates

To examine the issue of adverse impact for the Select for In-Store Banking Sales Associates survey, selection impact ratios for the Performance Index were estimated using a sample of applicant data collected for the standard Select assessment (n= 6775). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.





Analysis was completed on each of the individual indices and scoring recommendations (Performance and Integrity) to review for adverse impact. As can be seen in the upper portion of the table, no significant adverse impact was found for the individual indices (Performance and Integrity) for ethnicity or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Avoiding the AVOIDs.” “Recommendation 1” in the tables indicates the candidate scored in the Okay/Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 78.4% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. In the second analysis of Adverse Impact we examined Recommend rates using a more stringent screening criterion on the Performance Index. Under this scenario “Recommendation 2” indicates the candidate scored in the Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 66.3% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.



Percent of Candidates Passing Select for In-Store Banking Sales Associates


(n=6755)

White

(n=3144)

Minority

(n=3454) Ratio*

Men

(n=2475)

Women

(n=4225) Ratio*

Performance Index

0-41 Avoid -- -- -- -- -- -- --

42-44 Okay 80.0% 81.0% 79.3% .98 81.6% 79.1% .97

45-55 Good/Better 67.2% 68.7% 66.0% .96 68.9% 66.2% .96

Integrity

0-7 Avoid -- -- -- -- -- -- --

8-13 Good 94.9% 96.3% 93.6% .97 94% 95.4% 1.01

Combined**

Avoid -- -- -- -- -- -- --

Recommendation 1:

PI: Okay/Good/Better, I: Good 78.4% 79.8% 77.3% .97 79.5% 77.3% .97

Recommendation 2:

PI: Good/Better, I: Good 66.3% 68.0% 64.9% .95 67.9% 65.4% .96




Adverse Impact Analysis – Banking Service Associates

To examine the issue of adverse impact for the Select for Banking Service Associates survey, selection impact ratios for the Performance Index were estimated using a sample of applicant data collected for the standard Select assessment (n= 463). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.


Performance Index: Avoid (0-21), Okay (22) and Good/Better (23-25).



Analysis was completed on each of the individual indices and scoring recommendations (Performance and Integrity) to review for adverse impact. As can be seen in the upper portion of the table, no significant adverse impact was found for the individual indices (Performance and Integrity) for ethnicity or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Avoiding the Avoids.” “Recommendation 1” in the tables indicates the candidate scored in the Okay/Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 82.7% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. In the second analysis of Adverse Impact we examined Recommend rates using a more stringent screening criterion on the Performance Index. Under this scenario “Recommendation 2” indicates the candidate scored in the Good/Better range on the Performance Index and the Good range on the Integrity Index. Under these criteria 72.6% of the total sample would fall in the Recommend category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.



Percent of Candidates Passing Select for Banking Service Associates


(n=463)

White

(n=352)

Minority

(n=96) Ratio*

Men

(n=65)

Women

(n=386) Ratio*

Performance Index

0-21 Avoid -- -- -- -- -- -- --

22 Okay 82.7% 84.4% 76.0% .90 81.5% 82.6% 1.01

23-25 Good/Better 72.5% 73.9% 65.6% .89 70.7% 72.2% 1.02

Integrity

0-7 Avoid -- -- -- -- -- -- --

8-13 Good 99.6% 99.4% 100.0% 1.01 96.9% 100.0% 1.03

Combined**

Avoid -- -- -- -- -- -- --

Recommendation 1:

PI: Okay/Good/Better, I: Good 82.7% 84.4% 76.0% .90 81.5% 82.6% 1.01

Recommendation 2:

PI: Good/Better, I: Good 72.6% 73.9% 65.6% .89 70.8% 72.3% 1.02




Adverse Impact for Banking Branch Manager Math Module: To examine the issue of adverse impact for the Select for Banking Branch Manager Math survey, selection impact ratios for the Math Index were estimated using a sample of applicant data collected for the index (n= 976). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.


Banking Branch Manager Math Index: Needs Training (0-10), Okay (11-17).


Analysis was completed on the scoring recommendations (Math) to review for adverse impact. As can be seen in the table, no significant adverse impact was found for the individual index (Math) for ethnicity, or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Needs Training”. Under these criteria 90.6% of the total sample would fall in the Okay category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.

Percent of Candidates Scoring “Okay” on Banking Branch Manager Math Module


(n=278)

White

(n=211)

Minority

(n=57) Ratio*

Men

(n=134)

Women

(n=140) Ratio*

Math Index

0-10 Needs Training -- -- -- -- -- -- --

11-17 Okay 90.6% 91.0% 87.7% .96 95.5% 86.4% .90

Note: Impact Ratios between .80 and 1.20 are considered acceptable under EEOC Guidelines.



Adverse Impact for Banking Math Module: To examine the issue of adverse impact for the Select for Banking Math survey, selection impact ratios for the Math Index were estimated using a sample of applicant data collected for the index (n= 976). Recognizing the limitations of using a test of statistical significance on large samples our general approach is to apply the practical significance standard when examining large data sets such as the Select database. Using this approach the hiring recommendations made in the Select Manual will be within EEOC guidelines. (Should an organization desire to use more stringent criteria for screening candidates we recommend that AI be examined at the local level before proceeding.) Demographic information was collected during the administration of the survey and/or as part of the application process. Ethnicity and gender were examined on candidates that provided the information during the assessment process.


Banking Math Index: Needs Training (0-14), Okay (15-18).


Analysis was completed on the scoring recommendations (Math) to review for adverse impact. As can be seen in the table, no significant adverse impact was found for the individual index (Math) for ethnicity, or gender. In addition, analysis of Adverse Impact for the most conservative screening approach was evaluated – that of “Needs Training”. Under these criteria 89.1% of the total sample would fall in the Okay category and no significant adverse impact was found for ethnicity or gender for the combined recommendation. The values obtained for the test of practical significance are within acceptable ranges based on EEOC guidelines. Organizations could utilize this selection criterion without introducing significant adverse impact on protected groups.

Percent of Candidates Scoring “Okay” on Retail Banking Math Module


(n=976)

White

(n=663)

Minority

(n=296) Ratio*

Men

(n=440)

Women

(n=529) Ratio*

Math Index

0-14 Needs Training -- -- -- -- -- -- --

15-18 Okay 89.1% 90.8% 85.5% .94 91.4% 87.1% .95

Note: Impact Ratios between .80 and 1.20 are considered acceptable under EEOC Guidelines.



Summary The following conclusions were drawn based on the creation of the Select for Retail Banking (v3) assessments:

Those that possess the characteristics measured are rated significantly better in terms of: overall customer service and overall performance.

Those that score in the recommended ranges on the assessment are evaluated as better performers.

Recommendation of Use

The Select for Retail Banking (v3) surveys measure characteristics and skills that have been found to predict job effectiveness for similar types of roles. Through our research, we have identified differences amongst the characteristics that often make an associate more effective in one role versus another. The Select for Retail Banking (v3) reports are designed to provide information regarding a candidate’s personality characteristics and highlight areas that may merit attention in the interview process. The following are recommendations on how best to utilize the information from the report:

In prioritizing Retail Banking candidates, follow the “Avoid the AVOIDs” strategy; these are candidates who score an “Avoid” on the Performance Index or an “Avoid” on the Integrity index.

Should the organization desire to implement a “Raise the Bar” strategy, we strongly advise that an evaluation be conducted on the organization’s applicant population to estimate pass rates and potential adverse impact to ensure that selection strategies meet EEOC guidelines.

For those candidates who score in the “Good/Better” ranges, use the assessment results in conjunction with other appropriate information from the application process (e.g., education, prior experience, interview impressions, etc.) to make the final hiring decision.

Utilize the job task and follow-up interview probes to gain further information from the candidate.

Only use the assessment for the appropriate Retail Banking positions.

Caution Regarding Interpretation

The assessment reports generated by the use of the Select system contain information about individuals that can aid the organization in making selection decisions. Any judgments made in a report are probabilistic only and not deterministic. Ultimate performance on the job is a combination of many factors, including past experience, background, training, organizational conditions, management, etc. The organization must be careful not to overemphasize specific statements or results, but rather consider the total person's suitability for the specific position in the company environment. To minimize chances of erroneous decisions, the organization should integrate the information gained in each report with information from other sources (e.g., interview impressions, references, work experience, job performance, work habits, etc.).

Next Steps: Monitor Pass Rates

Assess Systems will continue to monitor the overall pass rates of applicants tested with the Select for Retail Banking (v3) surveys and refine and edit as needed to ensure risk for adverse impact is reduced.

Reference

Biddle, Dan (2006). Adverse Impact and Test Validation: A Practitioner’s Guide to Valid and Defensible Employment Testing (Second Edition). Aldershot, Hampshire, London: Gower Publishing.



APPENDIX



Sample Test Items

Sample Select Measures

Personality Scales (sample items):

Counterproductive Behaviors (sample items and responses):



Job Tasks (items and sample responses from Select for Banking Branch Manager (v3)) :



Job Tasks (items and sample responses from Select for Banking Sales Associate (v3)) :



Job Tasks (items and sample responses from Select for In-Store Banking Sales Associate (v3)) :



Job Tasks (items and sample responses from Select for Banking Service Associate (v3)):



Making the Hiring Decision Chart:



Development Suggestions (sample suggestions):



About Assess Systems

Assess Systems is a consulting company that provides products, services and general consulting to organizations.

Our focus is on the people in organizations - which ones to hire or promote and when - how to train and develop them to their fullest potential - and how to organize them and the systems around them to maximize their collective potential.

We are organizational psychologists, management consultants, software designers and service professionals working together to serve the business needs of our clients.

Available Select Surveys The following is a list of surveys available in the Select Associate Screening System:

Select for Administrative Support

Select for Banking Branch Managers

Select for Banking Service Associate

Select for Banking Sales Associate

Select for In-Store Banking Sales Associates

Select for Call Centers - Inbound Combo

Select for Call Centers - Inbound Sales

Select for Call Centers - Inbound Service

Select for Call Centers - Outbound Sales

Select for Call Centers - Sales Combo

Select for Convenience Store Managers

Select for Convenience Store Associates

Select for Customer Service

Select for Entry-Level Retail Managers

Select for Health Care

Select for Help Desk Agents

Select for Hospitality

Select for Leasing Agents

Select for Personal Service

Select for Production & Distribution

Select for Retail Clerks/Cashiers

Select for Retail Sales Associate

Date post:	03-Apr-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Select™ for Retail Banking (v3) - OutMatch for... · 2018-10-02 · Select for Retail Banking...

Documents