ASSESSING THE NATIONAL ENTRIES

Brian Walkerbrian.walker5@btinternet.com

Objective of the assessment The Assessment Process The procedure to be adopted Timescales Accessing the entries Examples and discussion

Outline

To review the entries and “validate” the self assessments to enable a short list to be

selected.

Objective

To provide an independent view of the entrant’s maturity level to assist them in

improving their operation

Primary Objective

Secondary Objective

Process

The companies Self-Assess their maturity level against a series of statements and provide evidence to support this level

Assessors check the companies assessment against the evidence they provide

Assessors’ role is to check that the maturity level selected looks reasonable, i.e. not obviously wrong

The Assessors do not rescore the entry from scratch based on the evidence provided

Assessment Aims

The aim is to select a short list of companies for each category

We need confidence that the best entry in that category will be among the companies short-listed

Companies differ and the score alone, while an indicator, may not provide the complete picture

Comments

Comments, both on individual questions and overall will be very helpful To explain any scoring changesTo assist with shortlistingTo focus on appropriate issues during the

review visitTo provide feedback to the company

What information will Assessors have?

Companies will indicate their assessment by selecting one of 5 statements showing a progressively higher maturity level

They will also provide evidence to support this selection by ticking relevant statement(s) in relation to the core questions and by providing free text evidence

The amount of evidence that companies can provide is limited, hence full proof of their own assessment is not possible or expected

Company Entry Screen – Core Questions

Click most appropriate

maturity level

Completion by Company

Click evidence pre-set that

applies

Completion by Company

Add free text evidence

Completion by Company

Navigation

Moving between

statements in Section

Navigation

Tabs to sections

Assessment Procedure 1

Read the general information first to get a feel for the company and its operations

Check each question, noting the maturity level selected

Check whether the evidence is reasonably supportive of the maturity level selected

Assessment Procedure 2

The lowest two maturity levels require no written evidence to support them

Higher maturity levels should be supported by evidence that is relevant and indicates that the maturity level has been achieved

The highest level needs more specific justification rather than general claims

Note the maturity level statements are generic and may not match every company. Sensible parallels need to be drawn in some cases.

Assessment Procedure 3

Where the evidence generally supports the maturity level selected – no action required

For higher maturity levels some evidence is needed but don’t look for complete confidence

Reduce score by one level where evidence is clearly inadequate – add comment to explain

In exceptional cases reduce further but only where evidence clearly indicates a lower level – again comments should explain reasons

Companies do score themselves too low so be prepared to raise scores when appropriate – comment should explain reason

Assessment Procedure 4

“Not applicable” (N/A) is an option for all questions

Entrants need to explain why the statement is “Not Applicable” if selecting this

Not Applicable means that it is not relevant to the type of business and its operations

It is not a valid response where the company has no system when a similar “best practice” company would have

As previously in MX if N/A is accepted the question is ignored in calculating percentages

Assessors need to check that the reason given is valid and N/A is a reasonable response

Not Applicable responses

Assessment Procedure 5

Two or three Assessors will be asked to assess each entry independently

A Lead Assessor will compare the assessments and adjudicate where necessary to produce an assessed report and score

The MX Board will review the top scoring entries, including any comments from Assessors to determine a short-list of companies to be visited.

Timescales

The Awards have now been launchedThe final date for entries is 23 May 2014Entries will sent for assessment as receivedFew expected much before closing dateAssessors need to complete assessment by

13 June 2014, earlier if possibleThe Short-list will be announced by 20 June

2014Vital that if Assessor is unable to undertake

an allocated Assessment they notify staff immediately

System Log -in Screen

Screen following Log-in

Screen following Log-in

Click to access entries to be

assessed

The companies you are to assess listed here

Assessor 1 & 2 here when assigned. Click to select

company

Assessor’s Entry Sheet

Assessor’s Entry Sheet

Click to access

statementsClick to see

summary report

Initial Screen for Company

Takes you to the questions

Details about company

Initial Screen for Company

Do not click back from this

page or questions

Initial Screen for Company

Click Return to Administration

to go back

Example showing Maturity Level, Pre-set Evidence and Free text Evidence

Entry Options

As with previous MX companies can enter for overall or individual categories

Category entry consists of some Core questions plus some additional questions

To be eligible for overall winner, company must complete all Core questions plus some categories

If entering only Categories then only the relevant questions need to be answered

Entrants Screen

Core questionsMX Category

Questions

Colour coded

Core questions attempted

Category attempted

Drop down to select

What have companies entered?

Entry Screen for extra question for MX Category

Suggestions for evidence – no pre-sets

2000 characters for evidence

instead of 1000

Tabs go to contributing

questions from Core set

Core questions retain pre-set

evidence

Also allow 2000 character evidence

Assessor’s view of Statement Page

Category notes apply to whole section so comment must indicate statement it applies to: “e.g. SM 1 No financial benefits evidenced, 5 reduced to 4”

Assessor’s comments go

here

Assessor changes

maturity level if reqd.

Key Points

Change higher maturity levels only if clearly not supported by the evidence

Be prepared to increase score where justified

But don’t rescore entry just on the basis of the evidence

Comments are invaluable whether maturity level changed or not

Make comments brief, specific and constructive

Any questions?

Admin details from staff

Thank you