finding

qualified

candidates?

protecting public safety and welfare?

The Four Immutable Laws of Certification Success

Kryterion Psychometric Services

warding off government regulation?

establishing authority and leadership?

Transcript: Law #1: Identify the Problem the Market Will Pay You to Solve

Thanks for stopping by!What follows is a transcript of the video, Law #1:

Identify the Problem the Market Will Pay You to Solve. There’s a reason Leslie selected this topic for the first law. It’s foundational. You can get many parts of a credentialing program right, but if miss this first law, your program is doomed. You’ll be trying to fix a problem that

your market doesn’t have. Your program will suffer the same

fate as the call-center certification mentioned in the transcript that follows. That certification “died on the vine,” to use Leslie’s own words A sad ending after a considerable sum of money had already been spent.

Take heart. If you do get Law #1 right, your certfication program will be positioned for success. Many successful consumer products alleviate or eliminate pain. Think of your certification in those same terms. It’s a product that must solve a pain point in your specific market.

You’re likely here because you watched a video on our Kryterion blog about The Four Immutable Laws of Certification Success. More precisely, the video was called Law #1: Identify the Problem the Market Will Pay You to Solve.

It’s the first of four video excerpts from an hour-long presentation at this year’s Association of Test Publisher’s (ATD) conference. The speaker was Kryterion’s own Chief Psychometric Officer, Leslie Thomas, Ph.D.

Leslie was the perfect person to deliver the presentation. She’s a psychometrician with more than 20 years’ experience developing and re-engineering certification programs across multiple industries. She and her team at Kryterion help organization’s make the best decisions possible for their credentialing programs, always with speed and cost savings in mind.

Leslie Thomas, Ph.D., Chief Psychometric OfficerKryterion, Inc. Association of Test Publishers Conference

IntroductionI want to welcome everyone here. Thank you very much for taking time out of your busy schedules to join me and the Kryterion Team for the next 60 minutes.

My goal today is to provide you with tips, tactics and strategies for credentialing success that you can take back home and use right away.

One of my own hopes, whenever I attend a conference, is to get information that I can take back and use. If I can do that, the event is worth the price of admission.

My name is Leslie Thomas. I head up Test Development and Psychometric Services at Kryterion. I’ve been with the company for about 15 years.

I’m going to be providing a lot of information to you today. I actually have a sticky note up here saying “Slow Down” because I have a tendency to speak quickly. So if I start going too fast today, please let me know.

We’ll have questions and answers at the end of today’s session. Also, if there’s something you want to ask me, but you’d rather not ask it in front of the group, please feel free to stop by the Kryterion booth #514 in the exhibition hall.

If you see me in the hall, feel free to stop me. I am a little near-sighted so you may have to make it a little obvious. If I don’t see you, it’s not that I’m ignoring you, it’s that my eyes have been bothering me lately. contacts.

Law #1: Identify the Primary Problem the Market Will Pay You to Solve 2

Kryterion Psychometric Services

Question: How many of you have worked with or are currently working with a psychometrician?

OK, it looks like most of you. So usually the biggest question I get—and I still get this from my Mom—is what is a psychometrician and what is psychometrics? Through the years, I’ve been introduced at conferences as the Chief “Psychotronics” Officer. All kinds of names.

Basically, psychometrics is the science of psychological testing. We test everything from personality tests and IQ tests to job-knowledge tests. Any kind of testing that you can imagine.

We work with organizations like yours to help design, develop, analyze, score and maintain tests.

A common misconception is that in order to help you develop an exam, we need to be domain experts ourselves. That’s not the case. We’re experts in working with the client’s subject matter experts to figure out what needs to be measured and the best way to measure it.

Psychometricians facilitate the process and do a lot of statistical analyses on the back end to figure out if what’s been developed is working as intended.

Leslie’s BackgroundSo, just a little bit about myself as context for today’s

talk. I’ve been in the industry for a little over 20 years. I started off at a company called ACT, formally known as American College Testing. I was right out of graduate school. Actually I hadn’t graduated yet. I worked with a lot of traditional programs. At the time, I had the luxury of having data sets of 20,000 people. It was wonderful.

I’ve worked with traditional health care organizations, social workers, dieticians and professional engineers. I’ve worked with the Automotive Service Excellence group. I worked with licensure programs as well.

I then moved on to another company called Brainbench, which had an adaptive testing driver. I had done some adaptive testing when I was at ACT.

Brainbench was later sold to PreVisor then SHL and then CEB. But Brainbench was a different type of

environment because they were for-profit. At the time, they were working with employers to identify qualified professional job candidates. They also worked in the IT

certification space. If you’ve worked in that area, you

know that the exam content changes quickly, and that was one of the things that we had to figure out how to accommodate.

After Brainbench, I had my own consulting business for a couple of years and worked with Lehman Brothers helping them develop a selection system for their IT people and their traders. I also worked with the Army on a career-interest inventory, helping them funnel down and streamline their 200+ MOS categories.

And then I landed at Kryterion where, as mentioned, I head up Test Development and Psychometric Services.

I tell you all of this to give you some context about my interesting career path. Most of us have had interesting paths. Regardless of the organization I worked with or the content area or domain area of the exam, I started to see patterns from my birds-eye view of working with different programs in different industries.

And those are some of the things that I’ll talk to you about today.

The 4 Immutable Laws will help you: ● Design a successful and sustainable credentialing

program and/or improve an existing one. Much of what we discuss today applies to an existing program as well as to new programs. Credentialing is not a one-and-done effort. It requires constant feeding and care.

● Better communicate the value of your credentialing program

● Recruit and get the most out of your subject matter experts. Your best and brightest SMEs are always the busiest. This presentation will talk about how best to recruit them.

● Create a higher-quality credentialing program in less time and at lower cost.

Law #1: Identify the Primary Problem the Market Will Pay You to Solve 3

Kryterion Psychometric Services

Why do we build credentialing programs? Your “why” is very important. It’s the foundation of your program. You have to be clear about your goal/outcome. If you get this wrong, everything else will be wrong, too.

At ATP, you may hear different reasons for developing a credentialing program like…

● Protecting the public safety and welfare ● Elevating the status of a profession ● Helping employers find qualified candidates ● Mitigating risk and driving compliance ● Increasing brand loyalty. There are many reasons for developing a credential.

A well-designed program can meet many, if not most, of the particular market needs above, but the programs that are really successful realize that in order to be successful, you need to be very clear about your “why.”

You have to be very clear as to the primary market need that you’re trying to solve. And more importantly, the need must be one that the market will pay for.

So, even though we want to elevate the status of a particular profession, if I ask test takers if they’re willing to pay money and spend time to take this credential in order to elevate the profession, they’re not willing to do that. The status of their profession is not a real problem.

Likewise, if you work for companies that send their people to courses and pay for the credential, they (the companies) are going to want to have a clear business need or return on investment in order to do so.

Treat a credentialing program like a productOne of the things I’ve found, especially when working with programs of a nonprofit nature, is that you need to realize that, at the end of the day, your credentialing program is a product, and you need to think about it like a product.

Case Study #1: Call Center Certification for Upper Level ManagersThe sponsor wanted to raise the status of the call-center industry as a whole, drive best practices across the industry and help upper-level call-center professionals differentiate themselves in the market. All lofty goals.

The problem was that the sponsor targeted the wrong job role. Job demand was strong for call-center professionals, who were more than aware of the shortage because of all of the headhunters calling them and offering them top dollar. Upper-level call center professionals, therefore, saw little need for the credential.

Employers were so desperate to find qualified candidates that they frankly didn’t care that prospects were certified.

The sponsor was trying to solve a problem that didn’t need a solution. Their program ended up withering on the vine. The sponsors had used the “if we build it, they will come” rationale for developing the credential. They never did the market research that would’ve revealed

Law #1: Identify the Primary Problem the Market Will Pay You to Solve 4

Kryterion Psychometric Services

that they were targeting the wrong job role. All of the wasted resources—money, manpower, consultants, staff, time and effort—could have been saved by a simple survey of market participants.

Case Study #2: Automotive Industry Certification for Automobile TechniciansThe problem for both consumers and employers was how to find qualified mechanics.

I worked on this credentialing program while I was with American College Testing. The Automotive Service Excellence Program was, at the time, the largest volun-tary credentialing program out there.

The certification allowed certificants to distinguish themselves in the market. It also helped employers make better hiring decisions. It also allowed them to promote the quality of their service to the marketplace. Lastly, the certification instilled confidence in consumers who understood that more highly trained mechanics could actually help save time and money.

The certification was a success because it fixed a problem that the market would pay for. The certification addressed a real pain point in the marketplace.

Identify the primary problem the market will pay you to resolve

● Get market feedback early and often. Make sure that your credential addresses a problem worth solving.

● Market pain points will streamline test-development and identify ROI metrics

● Communicate how your program addresses specific problems/pain points to increase its credibility and market value.

● “Your opinion, although interesting, is irrelevant.” It’s only your customer’s opinion that matters. Yours, while interesting, doesn’t.

Talk to your customers Surveying your test candidates is inexpensive, so include a survey at the end of your exam. Talk to your test taker. It’s easy from the comfort and security of your office to think you know what your market needs. But once you start to get feedback and input from your market, you’ll discover that market needs may not be what

you thought they were. Given the time and expense of developing a credentialing program, you need to be making the best decisions possible throughout. Many of them need to be informed by your market.You need to understand your market’s pain for two other reasons. First of all, it will help bring focus when you do your test development. Second, it will help you identify potential ROI metrics.

How do we measure ROI for a credentialing program?The first thing is to focus on is the market pain. For example, a common pain point for companies is how to find a qualified employee to fill a position. Go out to companies and ask them, “What is it about this job role that candidates don’t have the required knowledge and skills to perform the position?”

Get specific feedback. Which knowledge and skill gaps are preventing the employer from finding a qualified candidate? What outcome measures are available that would have the biggest impact in determining the knowledge and skill gaps, for example, in areas of public safety and welfare.

I have worked with successful programs for hospitals, banks and insurance companies to identify outcome

Law #1: Identify the Primary Problem the Market Will Pay You to Solve 5

Kryterion Psychometric Services

measures so that what they’re measuring on the exams is important to consumers and the companies.

Another market pain for IT companies is that in order to drive brand loyalty and product adoption, they must offer customer support, which can get expensive.

To lower these costs, an analysis of support tickets will identify common knowledge and skill gaps of their customers. For IT companies, the customer support department is often the best source of information for real-world scenarios about who’s calling in and the problems (gaps) they’re having. Knowing the gaps will inform training, certification and other things.

Once you start identifying those data measures, which become metrics, effectively, you can link them back to some sort of ROI. This can be messy. It’s not a precise process. Start with the data. See what it tells you.

Feel the painDive deeply into market needs, which I call pain points. People will study for and pay for a credential because they’re being driven by a pain point that needs resolution.

A certification is a product and needs to be marketed as a product that is eliminating pain, solving a market need or bestowing benefits. You then need to communicate those messages in the language of the marketplace (consumers).

I have worked with a number of companies over the years that had a great, quality product (certification) and were measuring all of the right things, but they weren’t communicating with their market in a way that was making the product visible and known.

It’s helpful to know how your market thinks about things and the language that they use when they talk about pain points. Knowing this will improve your marketing.

Identify the primary problem the market will pay you to resolve

● Do your research: competitive and “self” analysis. ● The market may reveal ideal conditions for a new cert ● Customer driven ● Captive audience (e.g., used for state licensure) ● Military/government adoption ● Partner with a well-known entity.

One of the first things that I ask new clients is whether they have any competition. If the answer is no, that could be a red flag, so more market research may be needed. I recommend a self-analysis. See what social media is saying about your program or those of your competitors.

Ideal circumstances: When market conditions advocate for certification:

Customer driven. Car manufacturers partnered with ASE to certify car mechanics. This was a mutually beneficial relationship. It works best when there is a pervasive need. If GM withdrew their financial support, for example, you would want to know that there’s still enough demand among technicians or other car companies to keep the certification going.

Captive audience. If you’re a certification sponsor for a profession that later becomes regulated by the state, the state may decide to adopt your certification as the new licensure test. in which case, everyone who wants to practice the profession must take your exam.

Military/government adoption. Helping veterans transition from military careers to civilian life can boost your certification if it assists in that transition.

Partner with a well-known entity. You can approach big companies and offer to partner with them. You’ll be the essential third party that will develop a credential for the benefit of the industry and the company. Promoting the certification will be much easier when it’s backed by a recognized company with authority and credibility in the marketplace. END

Law #1: Identify the Primary Problem the Market Will Pay You to Solve 6

Kryterion Psychometric Services