+ All Categories
Home > Documents > EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract...

EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract...

Date post: 20-Jun-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
38
European Agency for Safety and Health at Work Version January 2016 Bilbao, 21 May 2018 Our ret: PRU(201 8) D/909 I Contract references EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type of procedure: Open 2 Working method The evaluation was performed by an evaluation committee appointed by the responsible authorising officer. The members of the evaluation committee have held meetings according to the following schedule: : Date Place Topic discussed 11/04/2018 EUOSHA premises Evaluation of the exclusion and selection criteria 26/0412016 EUOSHA premises Evaluation of award criteria 3 Access to market The evaluation committee examined whether the tenderers have access to the market of the European Union pursuant to Articles 119 and 120 of the Financial Regulation. All of them were found to have access to EU public procurement contracts. The results of the verification are as follows: No Name Place of establishment Access to Comments or domicile market I Jegistry(20l8)N9B1 Yes No comments 2 RegistTy(2018)N982 . Yes No comments 3 (Registry(2018)N983 Yes No commenls 4 —. Registry(2018)N991 Yes No comments 5 (Registry(2018)A/1050) Yes No comments 4 Verification of exclusion criteria (declaration) Sanilago de Compostela 12. 5th floor 48003 Bllbao Spain Tef +34 944 358 400 Fax +34 944 358 401 n(ormation(Eosha europa cu http //osha europa cu
Transcript
Page 1: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

European Agencyfor Safety and Healthat Work

Version January 2016

Bilbao, 21 May 2018

Our ret: PRU(201 8) D/909

I Contract references

EVALUATION REPORT

Title of contract ESENER 3

Reference: EUOSHN2O1 7IOPIFISEIOO48

Contracting authority: EUOSHA

Type of procedure: Open

2 Working method

The evaluation was performed by an evaluation committee appointed by the responsibleauthorising officer.

The members of the evaluation committee have held meetings according to the followingschedule:

: Date Place Topic discussed

11/04/2018 EUOSHA premises Evaluation of the exclusion and selection criteria

26/0412016 EUOSHA premises Evaluation of award criteria

3 Access to market

The evaluation committee examined whether the tenderers have access to the market of theEuropean Union pursuant to Articles 119 and 120 of the Financial Regulation.

All of them were found to have access to EU public procurement contracts. The results of theverification are as follows:

No Name Place of establishment Access to Commentsor domicile market

I Jegistry(20l8)N9B1 Yes No comments

2 RegistTy(2018)N982 — . Yes No comments

3 (Registry(2018)N983 Yes No commenls

4 —. Registry(2018)N991 — Yes No comments

5 (Registry(2018)A/1050) Yes No comments

4 Verification of exclusion criteria (declaration)

Sanilago de Compostela 12. 5th floor 48003 Bllbao SpainTef +34 944 358 400 Fax +34 944 358 401n(ormation(Eosha europa cu http //osha europa cu

Page 2: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Version January 2016

European Agencyfor Safety and Healthat Work

The provision of declarations on honour, dated and signed, stating that the tenderers andsubcontractors as provided in the tender specifications are not in one of the exclusion orrejection situations referred to in Articles 106 and 107 of the Financial Regulation, has beenverified.

The results of the verification of exclusion criteria are as follows:

(Registry(2018)N9B3 Yes

,.Registry(20l 8)N991

(Registty(201 B)N1 050)

S Verification of selection criteria (declaration)

The provision of declarations on honour, signed and dated, stating that the tenderers andsubcontractors as provided in the tender specifications fulfil the selection criteria has beenverified.

As requested in the procurement documents, the following tenderers were asked to provide thefollowing documents for the purpose of verification of selection criteria:

- No 1 PRU(201 8)D1685 request to on 13 April 2018 to provide the duly signed declarationon honour on exclusion and selectbun criteria for consortium partner

The document has been requested by e-mail with the above Adonis number, Tenderers wereasked to provide the document within 7 days.

The following tenderers provided the documents requested in the procurement documents:

- No I REG(2018)A!1 177 from on 18 April2018.

The results of the verification of selection criteria are as follows:

SanUago do Compostoa 12 5th floor -48003 Bbao Sp&nTot •34 944 356 400 Fax +34 944 356 401n?arrnaton4osha europa eu http ,/osha europa eu

ND Name Declaratio Further request Accepted, Commentsn receIved (dale + ARES rejected -

in due reference) orform excluded

Jegistry(2018)A/981 No 13 AprIl 2018 Accepted Thedeclaration of

PRU(2018)D/685 honour fromone of theconsortium

.artners

ihad

nor been dullyfilled in. It wasprovided filledin, signed andstamped viaemail on 18April 2018,REG(201 8)N1‘77

2 RegIstry(2O18)N982 Yes —

3

4

5

Yes

Yes

2

Page 3: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Version January 2016

European Agencyfor Safety and Healthat Work

No Name Declaration Further Accepted Commentsreceived in due request or

form [and rejecteddocuments

required in thetender

specifications]

1 Registry(2018)N951 No 13 April Accepted

PRU(2018)0/685

2 (Reglstry(2018)AI982 Yes

3 Registry(2018)N983 Yes

4 (Registry(2018)N991 Yes

S (Reglstry(2018)N1050) Yes

6 Compliance with technical specifications

Compliance of tenders with the tender specifications was checked.

All tenders were found to be compliant with the minimum requirements of the tenderspecifications. The results of the compliance check are as follows:

Tender Name Compliant CommentsNo

1 p(Registsy(2o18)A/9s1 Yes

2 ,Registry(2018)N982 Yes

3 (Registry(2016)N983 Yes

4 egistry(2018)N991 Yes

S 4Registry(2018)N1050) Yes

7 Evaluation of award criteria

Requests for correction of obvious clerical errors have been sent on 30 April 2018 by emailPRU(2018)D1186 in accordance with Article 96(2) of the Financial Regulation to eachconcerned tenderer with a time limit of 7 days to provide a reply.

The following tenderers provided the requested corrections:

- No 3 ‘on 4 May 2018 REG(2018)N1 345

The final tenders were evaluated and ranked on the basis of the announced award criteriawithout modification. Detailed evaluations are attached as an Annex.

8 Abnormally low tender(s)

Santiago de Compostela 12 5th floor 48003 Bilbao SpainTel .34 944 358 400 Fax +34 944 358 401nformation@osha europa cu http llosha europa. Cu

Page 4: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Version January 2016

European Agencyfor Safety and Healthat Work

Not applicable

9 Score and ranking

Detailed marks per tender:

Criterion I Criterion 2 Criterion 3Tender Total score Reaches or

Name Max 30 Max 35 Max 35 for quality not minimumNoMm 15 Mm 16 Mm 18

criterIa scores

1 16 20 24 60 Yes(gislry(2018)A/981

2 1,, 18 24 25 57 Yes(Reg;stry(201 8)N982

3 21 29 31 61 Yes(Registry(201 8)N983

4 22 25 27 74 Yes(Regislry(201 8)A1991

5(Regisiry4uIo)A/1050) 27 33 34 94 Yes

The tenders that passed all minimum quality levels are ranked following the formula indicatedin the tender specifications:

Scare for tender = Quality score of the tender*0.70 + (lowest price offerlprice offer oftender) 1 000.30

Total score

Tenderfar quality Score for price

No Name criteria Price of the Finaltender (Financial Rankingsc ore

(Quality score*1 00*0.30)score’0,70)

1(Registry(20I8)A/981 42.00 €3,646,440.00 28.81 70.81 5

2 46,90 €4,229,612.00 24.84 71.74 4(Registry(201 8)A1982

3 -

(Registry(2018)ANb356.70 €3,501,654.80 30.00 86.70 2

4

- C,(egistry(2018)A/991 51.80 €4277630.00 24.56 76.36 3

st50) 65.80 €4,329,100.00 24.27 90.07 1

Sanhago do Compostela 12, 5th floor 48003 Bhbao SpainTel +34 944 358 400 Fox +34 944 358 401informationoshaeuropo cu hop /iosha europa cu

Page 5: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Version January 2016

European Agencyfor Safety and Healthat Work

10 Award

Considering the results of the evaluation as described above, the Evaluation Committeeproposes to award the contract to the following tenderer

Tender No 5

Kantar Public

Kantar Deutschland GmbH

Landsberger Strasse 284

80687 Munich

Germany

Price: EUR 3060,700

The proportion of the contract to be subcontracted is estimated to be 70% of the contractvalue.

The above-mentioned tender is compliant with the tender specifications and offers the bestprice quality ratio.

The Early Detection and Exclusion System has been checked.

Sant’aQD de Cornpost&a IZ 5th rocr - 48003 6ba SprTel .349442584&0 Faxt349443584019ñ7rrnator:3csha urooa cu rtt !!osha eurcoa Cu

5

Page 6: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Version January 2016

European Agencyfor Safety and Healthat Work

Name, date and signature of the evaluation committee members:

SinIiago d& Conposo!a 12 5th floor 48003 Sbao 5p&’

Tel ‘31 943 358 400 Fax +31 944 358 401

nIormotonosha europa eu fl1p iiosha europa eu0

Page 7: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Açj

tncy

)It

sSu

kty

nJ

IItc

hhJ

otwmk

Ope

nte

nd

erp

roce

du

reE

UO

SHN

2OI7

IOP/

F/SE

1004

8—T

echn

ical

eval

uati

on:

CR

ITE

RIO

NC

OM

ME

NT

S

CR

ITE

RIO

N1

EF

FIC

IEN

CY

AN

DQ

UA

LIT

YO

FT

HE

PR

OJE

CT

MA

NA

GE

ME

NT

:-

SC

OR

E:

16O

UT

OF

30P

OIN

TS

Str

engt

hs

The

app

roac

hto

proj

ect

man

agem

ent

and

wor

kor

gani

sati

on,

incl

udin

gth

ere

sourc

eal

loca

tion

,th

efe

asib

ilit

yof

the

prop

osed

tim

etab

lean

dth

eco

ntin

genc

ypl

anfo

runfo

rese

enev

ents

orco

mpl

icat

ions

,in

clud

ing

pro

cedure

sfo

rqu

alit

yco

ntro

lan

assu

rance

.(1

0po

ints

)5

Ext

ent

and

qual

ity

ofth

eco

rete

am,

incl

udin

gth

eco

mpo

siti

onin

term

sof

exp

erti

sean

dsk

ills

,al

loca

tion

orro

les

and

resp

onsi

bili

ties

ofal

lte

amm

ember

sas

wel

las

reso

urc

eba

ck-u

p(1

0po

ints

)5

The

over

all

appr

oach

topr

ojec

tm

anag

emen

tan

dw

ork

orga

nisa

lion

are

prop

erly

desc

ribe

dal

soin

term

sof

role

san

dre

spon

sibi

liti

esacro

spa

rtne

rs.

The

prop

o5ed

team

isal

loca

ted

tosp

eéif

icta

sks

(que

stio

nnai

re,

desi

gnan

dtr

ansl

atio

n;sa

mpl

ing

and

wei

ghti

ng;

scri

ptin

gan

dàa

tapr

oces

sing

),un

der

the

dire

ctio

nof

aP

roje

ctD

irec

tor

supp

orte

dby

aP

roje

ctL

eade

r:B

oth

P7oj

ect

Dfr

ecto

ran

dP

roje

ctL

eade

rar

eal

sore

spon

sibl

efo

rQ

ualit

yC

ontr

olpr

oced

ures

.

Aco

mpr

ehen

sive

qual

ity

assu

ran

cepl

anis

repoit

ed,

indl

udin

gre

spon

sibi

liti

esan

dqu

ality

indi

cato

rsan

dri

sks.

Wea

knes

ses

The

eval

uati

onw

ould

have

wel

com

edm

ore

clar

ityco

ncer

ning

the

reso

urce

allo

cati

on,

whi

chis

only

pres

ente

dfo

rth

eco

rele

am.

Inad

diti

onto

this

,th

enu

mbe

rof

days

allo

cate

dto

each

core

team

mem

ber

seem

sin

cons

iste

ntw

ithth

ety

pean

ddu

mbe

rof

task

sto

beca

rrie

dou

tan

dth

eov

eral

ldu

rati

onof

the

proj

ect.

Whi

leth

ero

les

and

resp

onsi

bili

ties

ofth

eco

rete

amar

eli

sted

,th

eev

alua

tion

com

mit

tee

wou

ldha

veap

prec

iate

da

desc

ript

ion

ofth

ere

lati

onsh

ips

betw

een

the

task

-bas

edte

ams.

Con

cern

ing

the

prop

osed

tim

etab

le,

the

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edfu

rthe

rde

tail

sco

ncer

ning

the

tim

ebe

twee

nth

epi

lot

and

mai

nfi

eldw

ork

(thr

eem

onth

s)as

itm

aypr

obab

lyle

adto

fiel

dwor

kta

king

plac

ein

sum

mer

.

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edm

ore

deta

ils

onth

eco

ntin

genc

ypl

an.

Whi

leth

equ

ality

assu

rance

isw

ell

expl

aine

d,it

was

foun

dto

bea

bit

stan

dard

and

not

deve

lope

din

view

ofth

enee

ds

ofE

SE

NE

R-3

.

Str

engt

hs

The

offe

rpr

esen

tsa

good

mix

ofsk

ills

inte

rms

ofpr

ojec

tm

anag

emen

t,re

port

ing

and

tech

nica

lex

pert

ise

inte

rms

ofC

AT

I/C

AW

Iim

plem

enta

tion

and

man

agem

ent,

asw

ell

asO

SH

expe

rtis

e.

Page 8: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

tender

procedure

EU

OS

HN

2O

I7IO

PIF

IS

E1004B

—T

echn

ical

ev

alu

atio

n:

-

Jim

Wet

ym

14H.

oIth

aW

wk

Wea

kn

esse

s

iM

ore

clar

ityw

ould

have

been

wel

com

edas

toth

ero

les

ofth

eco

rete

amm

embe

rsan

dth

ere

lati

onsh

ips

betw

een

them

.

Con

cern

ing

the

com

posi

tion

ofth

eco

rete

am,

grea

ter

expe

rtis

ew

ould

have

been

expe

cted

spec

ific

ally

inth

em

anag

emen

tof

larg

e,in

tern

atio

nal

surv

eys.

The

evat

uati

onco

mm

itte

eex

pres

sed

conc

ern

over

the

limite

dnu

mbe

rof

days

allo

cate

dto

the

proj

ect

dire

ctor

and

the

proj

ect

man

ager

,w

hich

dono

tse

emap

prop

riat

efo

rth

esp

ecif

icit

ies

ofE

SE

NE

R-3

.

Mor

eex

pert

ise

onsa

mpl

ing

and

wei

ghti

ngw

outd

have

been

expe

cted

and

agr

eate

ral

loca

tion

inte

rms

ofnu

mbe

rof

days

ofth

eap

poin

ted

team

mem

bers

.

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edde

tail

son

the

allo

cati

onof

num

ber

days

for

the

non-

core

team

mem

bers

.

The

prop

osal

does

not

incl

ude

are

sour

ceba

ck-u

ppl

an.

Str

engt

hs

The

prop

osal

cove

rsal

lth

eco

untr

ies

asre

quir

edin

the

tend

ersp

ecif

icat

ions

thro

ugh

the

subc

ontr

acti

ngto

fiel

dwor

kpr

ovid

er’

whi

chop

erat

esfr

omfo

urhu

bsan

dL

\an

dfo

rea

chhu

ba

Pro

ject

man

ager

and

aFi

eldw

ork

supe

rvis

orar

eap

poin

ted.

The

inte

rvie

wer

sar

ena

tive

spea

ker

san

dfo

rea

chco

untr

yth

ere

isa

nati

veev

alua

tor

for

qual

ityco

ntro

lpu

rpos

es.

Qua

tity

ofth

epr

opos

edne

twor

kof

fiet

dwor

kre

sear

chce

ntre

s,th

eT

henu

mbe

rof

CA

TIw

orks

tati

ons

(350

)is

deem

edto

been

ough

for

the

requ

irem

ents

ofth

ena

tion

alpr

ojec

tm

anag

ers

and

the

fiel

dwor

ksu

perv

isor

s(1

0po

ints

)6

-su

rvey

.

Wea

kn

esse

s

Div

isio

nof

resp

onsi

bili

ties

betw

een

the

hubs

nee

ds

tobe

clea

r.T

hesp

litof

inte

rvie

wer

sco

veri

ngth

esa

me

coun

try

acro

ssdi

ffer

ent

loca

tion

sca

nre

sult

inla

ckof

coor

dina

tion

and

the

eval

uati

onco

mm

itte

eex

pre

ssed

som

edo

ubts

abou

tit.

The

invo

lvem

ent

ofin

terv

iew

ers

with

two

mon

ths

ofex

peri

ence

isno

tde

emed

tobe

suff

icie

ntfo

rth

ere

quir

emen

tsof

this

proj

ect.

CR

ITE

RIO

N2

WO

RK

PAC

KA

GE

A:

QU

EST

ION

NA

IRE

AN

DSA

MPL

ING

:-S

CO

RE

:20

OU

TO

F35

PO

INT

S

Qua

lity

ofth

epr

opos

alto

the

ques

tion

nair

ere

visi

on,

incl

udin

gth

esu

gges

ted

appro

ach

tone

wto

pics

,th

est

rate

gyto

gath

erin

form

atio

nT

hepr

opos

allo

oks

into

mar

gina

lim

prov

emen

tsin

the

intr

oduc

tion

and

cont

act

phas

eof

the

from

non-r

esponden

tsan

dth

epr

e-te

st(1

0po

ints

)4

surv

eyas

wel

las

onth

ein

stru

ctio

nsfo

rth

ein

terv

iew

ers.

ersa

mfr

ngpf

mic

roes

tabl

ishm

ents

inth

epe-

test

J5ou

tof

the

25

Page 9: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Wea

knes

ses

The

prop

osal

toth

equ

esti

onna

ire

revi

sion

was

dee

med

tobe

wea

kby

the

eval

uati

onco

mm

itte

e,w

ithfe

wde

tail

sgi

ven

beyo

ndth

ecr

iter

iafo

rth

eli

tera

ture

revi

ewfo

rne

wto

pics

and

outc

ome

ques

tion

s.

On

com

para

bili

ty,

the

eval

uali

onco

mm

itte

ew

ould

have

expe

cted

mor

ede

tail

s,pa

rtic

ular

lyre

gard

ing

MS

Es.

Sim

ilarl

y,m

ore

wou

ldha

vebe

enw

elco

med

onga

ther

ing

info

rmat

ion

from

non-

resp

onde

nts.

The

appr

oach

toth

epr

e-te

stw

asno

tsu

ffic

ient

.T

heev

alua

tion

com

mit

tee

expre

ssed

conc

erns

over

carr

ying

out

the

inte

rvie

ws

inE

ngli

shon

ly,

asth

ispr

even

tsfr

omte

stin

gth

ecu

ltur

alco

mpo

nent

san

dth

esp

ecif

icla

ngua

gen

uan

ces.

Str

engt

hs

Qua

lity

ofth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

—w

hile

ensu

ring

inle

rnat

iona

lco

mpa

rabi

lity

-,in

clud

ing

the

nati

onal

ad-h

ocm

odul

es,

asw

ell

asth

epro

pose

dap

pro

ach

totr

ansl

atio

nan

dth

epi

lot

test

(10

poin

ts)

6

Qua

lity

ofth

esa

mpl

ing

stra

tegy

,in

clud

ing

the

prop

osed

bu

sin

ess

reg

iste

rs,

the

sam

plin

gde

sign

invi

ewof

min

imis

ing

sele

ctio

nbi

asan

dth

epo

tent

ial

impa

cts

onco

mpa

rabi

lity

with

ES

EN

ER

-2(1

5po

ints

)10

The

over

all

appr

oach

toth

etr

ansl

atio

n,le

dby

—an

appl

ying

Lhe

TR

AP

Uap

proa

ch,

isde

emed

tobe

very

soun

dan

dre

leva

nt,

inlin

ew

ithth

ete

chni

cal

spec

ific

atio

ns.

The

desc

ript

ion

ofth

epr

opos

alto

the

nati

onal

ad-h

ocm

odul

esm

eets

the

requ

irem

ents

ofth

ete

chni

cal

spec

ific

atio

ns.

For

the

pilo

tte

st,

atle

ast

10m

icro

esta

blis

hmen

ts(o

utof

30)

are

prop

osed

tobe

inte

rvie

wed

inea

chof

the

coun

trie

s.M

ost

ofth

ein

terv

iew

ers

invo

lved

inth

epi

lot

will

bein

inor

der

tom

inim

ize

the

num

ber

oftr

aini

ngs

and

tole

tth

eco

rete

amli

sten

toth

ep

ilo

flu

esti

on

nai

res.

Wea

knes

ses

The

appr

oach

toth

ena

tion

alad

-hoc

mod

ules

isin

line

with

the

tech

nica

lsp

ecif

icat

ions

but

addi

tion

alde

tail

sw

ould

have

been

wel

com

edby

lhe

eval

uati

onco

mm

itte

e.T

heov

eral

lap

proa

chto

the

pilo

tw

asve

rysu

ccin

ctan

dm

ore

deta

ils

wou

ldha

vebe

enI

appr

ecia

ted

byth

eev

alua

tion

com

mit

tee.

Str

engt

hs

The

eval

uati

onco

mm

itte

eva

lued

posi

tive

lyth

ege

nera

lap

proa

chto

the

sam

plin

gst

rate

gy,

build

ing

onth

eex

peri

ence

from

ES

EN

ER

-2.

The

reis

are

leva

ntdi

scus

sion

onre

liabi

lity,

whi

chaf

fect

sIh

est

rati

fica

tion

ofna

tion

alsa

mp

les

acro

ssse

cto

ran

dsi

zecl

asse

s.T

hete

nder

erap

pea

rsto

prop

ose

adi

spro

port

iona

test

rati

fica

tion

bysi

zean

dto

have

itto

oby

sect

orin

the

coun

trie

sw

ithle

ssth

an75

0in

terv

iew

s,in

view

ofin

crea

sing

the

relia

bilit

yof

the

find

ings

for

smal

lse

ctor

s.

-E

uic

taA

tiu

y

JI,,

s.I,

,oa

4H

tthh

/ai

Wor

k

Ope

nte

nd

erp

roce

du

reE

UO

SH

N2O

I7IO

PIFI

SEIO

O48

—T

echn

ical

eval

uati

on:

prop

osed

. aso

und

roun

dq

go

Ith

ere

qufr

edda

taso

urce

s,in

clu

gan

esti

mat

ion

ofth

e-

Page 10: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Lpe

m1L

nEy

Open

tender

pro

cedure

EU

OSH

AI2

OI7

IOPI

FISE

IOO

48—

Tec

hn

ical

eval

uat

ion

:j

i.S0

T.ty

ctd

Ubh

,JQ

IWh

7mis

sing

data

Laes

tim

ate

the

num

ber

ofes

tabl

ishm

ents

,co

mbi

ning

the

SB

San

dth

eL

FS.

A

rele

vant

poin

tis

mad

eon

the

type

ofdi

stri

buti

onfu

ncti

onof

esta

blis

hmen

tsac

ross

size

clas

ses.

The

reis

are

leva

ntap

proa

chto

the

sam

ple

plan

spe

rco

untr

y,ai

min

gto

min

imis

eth

eva

rian

ceof

desi

gnw

eigh

ts.

The

offe

rsar

gues

that

the

min

imum

sam

ple

size

per

coun

try

shou

ldat

leas

tbe

672

(96

x7)

.F

urth

erto

this

,th

eof

fer

prop

oses

toin

clud

eth

eag

eof

the

esta

blis

hmen

tsin

the

wei

ghti

ng.

The

prop

osed

sam

ple

fram

esar

epr

ovid

edby

Sam

ple

Sol

utio

nsan

dar

eat

the

esta

blis

hmen

tle

vel—

whi

chis

posi

tive

—an

dth

eap

proa

chis

cent

rali

sed,

toge

ther

with

The

reis

apr

oper

desc

ript

ion

ofth

eco

mpa

rabi

lity

ofth

epr

opos

edap

proa

ch,

argu

ing

that

the

chan

ges

—sa

mpl

efr

ames

,scr

eeni

ngfo

rno

n-ex

isti

ngnu

mbe

rs,

impr

ovem

ents

inco

ntac

tph

ase

—w

illin

crea

seth

ere

liabi

lity

ofth

efi

ndin

gsan

dhe

nce,

ther

ew

illbe

anef

fect

onco

mpa

rabi

lity

.

-

Wea

kn

esse

s

Whi

leth

epr

opos

alis

rele

vant

,th

eev

alua

tion

com

mit

tee

wou

ldha

vew

elco

med

mor

ecl

arity

rega

rdin

gth

efi

nal

sam

ple

desi

gnpr

opos

ed.

Inlin

ew

ithth

is,

mor

ein

form

atio

nw

ould

have

been

wel

com

edab

out

the

esta

blis

hmen

tle

vel

sam

ple

fram

espr

ovid

edby

CR

ITE

RIO

N3

WO

RK

PAC

KA

GE

8:T

RA

ININ

G,

IMPL

EM

EN

TA

TIO

NA

ND

DE

LIV

ER

Y:

-S

CO

RE

:24

OU

TO

F35

PO

INT

S

Str

engt

hs

The

eval

uati

onco

mm

itte

eva

lued

posi

tivel

yth

eus

eof

ace

ntra

tise

dda

taco

llec

tion

plat

form

r’

both

CA

TIan

dC

AW

I,w

hich

enab

les

the

‘rea

l-tim

e’tr

acki

ngof

the

sam

ple,

reco

rdin

gin

form

atio

non

cont

acts

and

non-

cont

acts

.Si

mila

rly,

the

prop

osed

appr

oach

toth

epr

ogra

mm

ing

ofth

em

aste

rqu

esti

onna

ire

isso

und.

Qua

lity

ofth

est

rate

gyfo

rfi

eldw

ork

prep

arat

ion,

part

icul

arly

the

The

reis

atr

aini

ngse

min

arpr

opos

edw

ithth

eco

repr

ojec

tte

am,

the

dire

ctor

a!op

erat

ions

appro

ach

totr

aini

ngan

dth

eco

ntac

tst

rate

gy(o

rre

duci

ngno

n-re

spon

se,

and

the

qual

ityco

ntro

lm

anag

erat

,—

plus

site

man

ager

san

dtw

osu

perv

isor

sfr

om

incl

udin

gth

eon

line

mod

ule

(10

poin

ts)

7ea

chhu

b.A

lso,

two

fiel

dwor

kbr

iefi

ngs

are

envi

sage

d(f

orth

epi

lot

and

the

mai

nfi

eldw

ork)

,he

ldby

am

embe

rof

the

core

team

,si

tem

anag

eran

dna

tive

supe

rvis

or.

The

eval

uati

onco

mm

itte

eap

prec

iate

dth

epr

opos

edap

proa

chto

impr

ovin

gth

eco

ntac

tst

rate

gy.

Off

erin

g15

call

atte

mpt

s,ov

erdi

ffer

ent

tim

esan

dda

ysan

dta

rget

edto

sect

orpr

acti

ces,

send

ing

the

intr

oduc

tion

lett

erw

hile

onth

eph

one

with

the

gate

keep

eror

resp

onde

nt,

are

rele

vant

prop

osal

s.

For

the

onlin

equ

esti

onna

ire,

the

mo

nil

org

ofin

vila

tion

sse

ntou

tdugfi

eid

work

isw

ell

Page 11: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

I.,Sc

fVy

D4

Iciw

ca

Open

ten

der

pro

cedure

EU

OSH

A)2

0171

OPI

FISE

IOO

4B—

Tec

hnic

alev

alua

tion

:

expl

aine

d,w

itha

rem

inde

raf

ter

thre

eda

ys,

and

afo

llow

-up

tono

n-re

spon

seon

day

8.S

ubje

ctto

addi

tion

alqu

ality

cont

rol

mea

sure

s,re

spo

nd

ents

may

beca

lled

back

.

Wea

knes

ses

Qua

lity

ofth

e,p

rop

ose

dfi

eldw

ork

exec

utio

n,de

scri

bing

the

over

all

appro

ach

tog

uar

ante

ehi

gh-q

uali

tyin

terv

iew

san

dm

inim

ise

no

nre

sponse

bias

incl

udin

gth

em

onit

orin

g,co

ntro

lan

dre

port

ing

ofth

efi

eldw

ork

pro

cess

(15

poin

ts)

10

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edm

ore

deta

ils

conc

erni

ngth

etr

aini

ng.

IIw

asd

eem

edto

bege

nera

lfo

rth

ein

terv

iew

erta

sks.

Itis

not

enti

rely

clea

rto

the

eval

uati

onco

mm

itte

ew

hen

CA

WI

isto

beof

fere

d,on

lyas

ala

stre

sort

orbe

fore

—if

the

resp

onde

ntha

sno

ten

ough

tim

eon

the

phon

e.

Str

engt

hs

Fiel

dwor

kis

tobe

carr

ied

out

from

four

mai

nhu

bs,

with

nati

vein

terv

iew

ers.

Inad

diti

onto

this

,na

tive

eval

uato

rsar

eap

poin

ted

for

each

coun

try,

whi

chis

valu

edpo

siti

vely

byth

eev

alua

tion

com

mit

tee.

The

reis

aso

und

appr

oach

tosa

mpl

em

onit

orin

g—

byce

ll—

whi

chis

cent

rali

sed

and

perf

orm

edon

ada

ilyba

sis,

and

envi

sagi

ngan

asse

ssm

ent

whe

n80

%th

resh

old

has

bee

nre

ache

d.T

here

ison

line

acce

ssto

real

-tim

epr

ogre

sspe

rco

untr

y,pl

usw

eekl

yup

date

dre

port

s.

The

reis

ago

od,

cent

rali

sed

appr

oach

toqu

alit

yco

ntro

l,ch

ecki

ng11

-12%

ofal

lin

terv

iew

sco

nduc

ted

and

intr

oduc

tion

conv

ersa

tion

s.D

epen

ding

onth

efi

eldw

ork

phas

eco

ntro

lis

tofo

cus

ondi

ffer

ent

issu

es.

For

CA

WI,

ave

rifi

cati

onca

llis

prop

osed

for

outl

iers

—ex

pect

edfo

r15

-20%

ofC

AW

Iin

terv

iew

s—

,pe

rfor

min

ga

chec

kof

four

ques

tion

s.

Wea

knes

ses

Qua

lity

ofth

eap

pro

ach

toda

tapr

oces

sing

,de

live

ryan

dst

orag

e,in

clud

ing

wei

ghti

ngan

dth

eim

pact

onco

mpa

rabi

lity

with

ES

EN

ER

-2,

asw

ell

asth

ere

port

ing

(10

poin

ts)

7

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edm

ore

clar

ityre

gard

ing

the

loca

tion

ofth

ena

tion

alte

ams

ofin

terv

iew

ers

and

whe

ther

nati

onal

team

sar

esp

read

acro

ssdi

ffer

ent

hubs

Str

engt

hs

The

appr

oach

toda

taco

ding

isco

mpa

tibl

ew

ithE

SE

NE

R-2

and

mat

ches

the

requ

irem

ents

ofth

ete

chni

cal

spec

ific

atio

ns.

Whe

reap

plic

able

,co

nsis

tenc

ych

ecks

will

bepr

ogra

mm

edin

the

ques

tion

nair

e.D

ata

vali

dati

onto

bepe

rfor

med

atva

riab

lean

dre

spon

dent

leve

l.T

heof

fer

prop

oses

vali

dati

onru

les

tobe

Inte

grat

edas

part

ofth

eC

AT

Ipr

ogra

mm

ing

plus

anS

PS

Ssy

ntax

tobe

agre

edw

ithE

U-O

SHA

for

post

-hoc

vali

dati

on,

whi

chis

soun

d.T

heev

alua

tion

com

mit

tee

valu

edpo

siti

vely

the

poin

ton

the

data

tobe

stor

edin

for

two

year

s.

The

appr

oach

tow

eigh

ting

follo

ws

the

one

adop

ted

inE

SE

NE

R-2

,w

ithth

eus

eof

gene

rali

sed

regr

essi

ones

tim

atio

nrn

ptho

dsfo

rpo

st-s

trat

ific

atio

nw

eigh

ts.

The

auxi

liar

y

Page 12: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Jlr

5ofe

Iym

idH

rhh

Jolw

mk

Op

ente

nder

pro

cedure

EU

OS

HA

I2O

I710

P1F1

5E10

048—

Tec

hn

ical

eval

uat

ion

;

vari

able

sar

ees

tabl

ishm

ent

age

and

regi

on.

The

offe

rpr

opos

esto

add

answ

ers

rela

ted

to

the

wil

ling

ness

toco

oper

ate

(per

cept

ion

ofin

terv

iew

erad

ded

atth

een

dof

each

inte

rvie

w).

Adi

scus

sion

ofth

eco

mpa

rabi

lity

betw

een

wei

ghti

ngin

ES

EN

ER

-2an

dE

SE

NE

R-3

is

prov

ided

too,

whi

chth

eev

alua

tion

com

mit

tee

appr

ecia

tes.

Wea

kn

esse

s

Whi

leth

eda

tava

lidat

ion,

proc

essi

ng,

stor

age

and

deli

very

are

pres

ente

d,ad

diti

onal

deta

ils

wou

ldha

vebe

enap

prec

iate

dby

the

eval

uati

onco

mm

itte

e,al

sofo

rth

ew

eigh

ting

appr

oach

and

the

repo

rtin

g.

TO

TA

L6

00

UT

OF

IOD

Sig

ned

by:

Page 13: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

‘6JI;I

!nn

A&

n[y

3fe

,StF

.rynd

11.4

thI

wW

c,k

Ope

nte

nd

erpro

cedure

EU

OSH

AI2

OI7

IOP/

F/SE

IOO

4B—

Tec

hnic

alev

alua

tion

:—

CR

ITE

RIO

NC

OM

MEN

TS

CR

ITE

RIO

NI

EFF

ICIE

NC

YA

ND

QU

AL

ItY

OF

THE

PRO

JEC

TM

AN

AG

EMEN

T:-

SCO

RE

:18

OU

TO

F30

POIN

TS

Str

engt

hs

The

reis

ade

tail

edde

scri

ptio

nof

the

appr

oach

toth

ew

ork

orga

nisa

tion

and

the

proj

ect

man

agem

ent,

whi

chis

deem

edso

und

byth

eev

alua

tion

com

mit

tee.

The

prop

osed

tim

etab

lese

ems

feas

ible

and

inlin

ew

ithle

nder

requ

irem

ents

The

app

roac

hto

proj

ect

man

agem

ent

and

wor

kor

gani

sati

on,

incl

udin

gth

ere

sou

rce

allo

cati

on,

the

feas

ibil

ity

ofth

epr

opos

edti

met

able

and

the

con

tingen

cypl

anfo

runfo

rese

enev

ents

orco

mpl

icat

ions

,in

clud

ing

pro

cedure

sfo

rqu

alit

yco

ntro

lan

das

sura

nce

.(1

0po

ints

)6

Ext

ent

and

qual

ity

ofth

eco

rete

am,

incl

udin

gth

eco

mpo

siti

onin

term

sof

exper

tise

and

skill

s,al

loca

tion

orro

les

and

resp

onsi

bili

ties

ofal

lte

amm

ember

sas

wel

las

reso

urc

eba

ck-u

p(1

0po

ints

)6

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

eco

ntin

genc

ypl

an,

with

ade

scri

ptio

nof

the

reso

urce

back

-ups

.T

here

isa

soun

dan

dde

tail

edap

proa

chto

qual

ity

cont

rol,

incl

udin

gth

eap

poin

tmen

tof

aqu

alit

ydi

rect

or,

and

alon

gIh

ecr

iter

iaof

the

ES

S,

with

ade

tail

edqu

alit

yas

sura

nce

plan

cove

ring

the

risk

asse

ssm

ent

and

miti

gatin

gac

tion

per

task

,in

clud

ing

resp

onsi

bili

ties

for

deli

vera

bles

and

qual

ityin

dica

tors

.

Wea

knes

ses

Whi

leth

ere

isa

clea

rde

scri

ptio

nof

the

role

sof

the

supp

orti

ngpa

rtne

rsls

ubco

ntra

ctor

s—

and

‘,th

eev

alua

tion

com

mit

tee

wou

ldha

vea

red

ated

mor

ecl

arity

asto

the

spec

ific

role

san

dre

spon

sibi

liti

esw

ithin

the

core

team

(an

d

a-

Fur

ther

toth

is,

itw

ould

have

been

wel

com

eto

have

mor

ede

tail

son

the

rela

tion

ship

betw

een

the

task

-bas

edsu

bte

ams

and

the

resL

Con

cern

ing

the

reso

urc

eal

loca

tion

,th

eev

alua

tion

com

mit

tee

expre

ssed

som

edi

ffic

ultie

sto

asse

ssth

enu

mbe

rof

days

allo

cate

dto

each

team

mem

ber

and

whe

ther

they

are

cons

iste

ntw

ithth

ety

pean

dnu

mbe

rof

task

sto

beca

rrie

dou

t.T

hein

form

atio

nis

prov

ided

per

task

rath

erlh

anin

divi

dual

team

mem

ber.

Whi

leth

eri

skas

sess

men

tis

very

deta

iled

,th

eev

alua

tion

com

mit

tee

has

expre

ssed

som

eco

ncer

nsab

out

itsco

vera

geof

the

spec

iflc

itie

sof

ES

EN

ER

-3.

Str

engt

hs

The

eval

uati

onco

mm

itte

eva

lued

posi

tive

lyth

eex

tent

and

expe

rtis

eof

the

core

team

,pa

rtic

ular

lyin

term

sof

proj

ect

man

agem

ent,

asw

ell

asth

epr

opos

edre

sour

ceba

ck-u

p.

Page 14: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Op

ente

nd

erpro

cedure

EU

OSH

A12

OI7

/OP1

F/SE

/004

8—T

ech

nic

alev

alu

atio

n:

—.

6tS

d,ty

onil

itdi

h.J

niwmb

Wea

kn

esse

s

As

poin

ted

out

abov

e,m

ore

clar

ityw

ould

have

been

wel

com

eby

the

eval

uati

onco

mm

itte

ere

gard

ing

the

allo

cati

onof

days

for

each

team

mem

ber.

Som

eco

ncer

nsw

ere

exp

ress

edab

out

the

mix

ofsk

ills

inth

epr

opos

edte

am,

whi

chw

asde

emed

very

stro

ngfo

rth

epr

ojec

tm

anag

emen

t,th

epr

e-te

stan

dth

etr

ansl

atio

n.H

owev

er,

high

erex

pert

ise

wou

ldha

vebe

enap

prec

iate

din

term

sof

surv

eyde

sign

,m

anag

emen

tan

dan

alys

is,

asw

ell

asth

emat

ickn

owle

dge

ofO

SH

.

Str

engt

hs

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

eco

vera

geof

all

the

coun

trie

sas

requ

ired

inth

ete

chni

cal

spec

ific

atio

nsfr

omth

efo

urhu

bsan

dth

eap

poin

tmen

tof

apr

ojec

tm

anag

eran

dfi

eldw

ork

supe

rvis

orin

each

ofth

ehu

bs.

Fur

ther

toth

is,

aco

untr

yco

ordi

nato

ran

dan

oper

atio

nsm

anag

erar

eap

poin

ted

too,

whi

chis

valu

edpo

sitiv

ely

byth

eev

alua

tion

com

mit

tee.

The

num

ber

ofw

orks

tati

ons

isab

out

350,

whi

chis

cons

iste

ntw

ithth

enu

mbe

rof

inte

rvie

ws

tobe

carr

ied

out

and

the

timel

ine

prop

osed

.

The

eval

uati

onco

mm

itte

eva

lues

posi

tive

lyth

eC

lE

ngli

shle

vel

ofth

epr

ojec

tm

anag

ers

Qua

lity

ofth

epr

opos

edne

twor

kof

fiel

dwor

kre

sear

chce

ntre

s,th

ean

dfi

eldw

ork

supe

rvis

ors,

and

the

depl

oym

ent

ofna

tive

spea

king

inte

rvie

wer

sof

all

the

nati

onal

proj

ect

man

ager

san

dth

efi

eldw

ork

supe

rvis

ors

(10

poin

ts)

Bla

ngua

ges

cove

red.

Wea

kn

esse

s

The

eval

uati

onco

mm

itte

ew

ould

have

appr

ecia

ted

mor

ede

tail

sto

this

spec

ific

crite

rion

.M

ore

clar

ityw

ould

have

been

help

ful

rega

rdin

gth

elo

cati

onof

the

man

ager

san

dsu

perv

isor

san

dw

hile

thei

rex

peri

ence

isve

ryre

leva

nt,

mor

ede

tail

sas

tow

how

illbe

cove

ring

wha

tco

untr

yw

ould

have

been

appr

ecia

ted.

Ats

o,th

eap

proa

chto

qual

ityco

ntro

lat

—is

very

thor

ough

but

mor

eta

ilori

ngto

the

spec

ific

itie

sof

ES

EN

ER

-3w

ould

have

been

appr

ecia

ted.

The

eval

uati

onco

mm

itte

eex

pre

ssed

som

eco

ncer

nsas

toth

epa

rtic

ipat

ion

ofin

terv

iew

ers

with

two-

mon

thex

peri

ence

.

CR

ITER

ION

2W

OR

kPA

CK

AG

EA:

QU

EST

ION

NA

IRE

AN

DSA

MPL

ING

:-S

CO

RE

:24

OU

TO

F35

PO

INT

S

Qua

lity

ofth

epr

opos

alto

the

ques

tion

nair

ere

visi

on,

incl

udin

gth

eS

tren

gths

sugges

ted

appro

ach

tone

wto

pics

,th

est

rate

gyto

gath

erin

form

atio

nT

heap

proa

chto

revi

seth

equ

esti

onna

ire,

prop

osin

gth

equ

esti

onna

ire

appr

aisa

lsy

stem

,fr

omnon-r

esponden

tsan

dth

epr

o-te

st(1

0po

ints

)7

seem

edre

leva

ntto

the

eval

uati

onco

mm

itte

e.S

ugge

stio

nsto

revi

seth

equ

esti

onna

ire

alon

gth

elin

esof

the

tend

ersp

ecif

icat

ions

are

pres

ente

dan

dex

tens

ivel

yex

plai

ned.

Page 15: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

The

eval

uati

onco

mm

itte

eap

prec

iate

dth

epr

opos

als

tore

view

the

ques

tion

nair

ein

orde

rto

incr

ease

the

resp

on

sera

te,

part

icul

arly

look

ing

atth

esu

rvey

intr

oduc

tion

text

.Si

mila

rly,

the

ques

tion

sto

ask

tono

n-re

spon

dent

san

dth

eju

stif

icat

ion

ofth

epr

opos

alto

focu

son

OS

Hm

anag

emen

tqu

esti

ons

(bea

ring

inm

ind

resp

onse

leve

lsin

ES

EN

ER

-2)

was

posi

tive

lyva

lued

byth

eev

alua

tion

com

mit

tee.

The

reis

also

ago

odun

ders

tand

ing

ofsu

rvey

ing

smal

lest

esta

blis

hmen

ts,

adap

ting

the

intr

oduc

tory

text

and

sugg

esti

ngso

me

inte

rvie

wer

hint

sfo

rso

me

spec

ific

item

s.

The

eval

uati

onco

mm

itte

eva

lued

the

poin

ton

focu

sing

the

effo

rts

inre

achi

ngth

eta

rget

resp

onde

ntas

soon

aspo

ssib

lein

stea

dof

prov

idin

gm

any

deta

ils

togat

ekee

per

s.T

heap

proa

chto

the

defi

nitio

nof

new

ques

tion

sw

asd

eem

edso

und.

with

acl

earl

yde

fine

dcr

iter

ia,

and

tobe

vali

date

din

the

cogn

itiv

epr

e-te

st.

The

eval

uati

onco

mm

itte

eco

nsid

ered

rele

vant

the

cons

ider

atio

nsm

ade

conc

erni

ngth

ein

clus

ion

ofou

tcom

equ

esti

ons,

tryi

ngto

gath

eras

muc

hin

form

atio

nas

poss

ible

over

the

phon

e,of

feri

ngth

epo

ssib

ilit

yto

com

plet

eth

em

issi

ngin

form

atio

non

line

.T

heap

proa

chto

the

pre-

test

isde

emed

stro

ng,

incl

udin

gth

etr

ansl

atio

nby

as

wel

las

the

sug

ges

ted

coun

try

cove

rage

.T

horo

ugh

desc

ript

ion,

incl

udin

gth

ean

alyt

ical

fram

ewor

kte

mpl

ate,

pro

cess

carr

ied

out

byex

peri

ence

dre

sear

cher

str

aine

db

y•

via

web

inar

.

Wea

knes

ses

Whi

leth

eap

proa

chto

the

pre-

test

isco

nsid

ered

tobe

stro

ng,

furt

her

deta

ils

wou

ldha

vebe

enw

elco

me

onw

how

illbe

inte

rvie

wed

and

how

resp

on

den

tsw

illbe

appr

oach

ed.

Whi

leth

ecr

iter

iato

defi

nene

wqu

esti

ons

isre

leva

nt,

the

spec

ific

exam

ples

toth

ene

wqu

esti

ons,

whi

char

ein

line

with

the

ones

incl

uded

inth

ete

chni

cal

spec

ific

atio

ns,

are

deem

eda

bit

vagu

e.T

hetw

oad

diti

onal

topi

cson

nano

mat

eria

lsan

dgr

een

jobs

are

appr

ecia

ted

byth

eev

alua

tion

com

mit

tee

but

som

eco

ncer

nw

asex

pre

ssed

abou

tth

epr

opos

edfo

rmul

atio

nof

the

ques

tion

s.

Str

engt

hs-

Qua

lity

ofth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

—w

hile

ensu

ring

inte

rnat

iona

lco

mpa

rabi

lity

-,in

clud

ing

the

nati

onal

ad-h

ocm

odul

es,

asw

elt

asth

epro

pose

dap

pro

ach

totr

ansl

atio

nan

dth

epf

lol

test

(10

poin

ts)

9

The

over

all

appr

oach

toth

etr

ansl

atio

n,le

dby

subc

ontr

acto

r—

and

appl

ying

the

TR

APO

appr

oach

,is

deem

edto

beve

ryso

und

and

rele

vant

,in

line

with

the

tech

nica

lsp

ecif

icat

ions

.

The

eval

uati

onco

mm

itte

eva

lued

posi

tive

lyth

epr

opos

alto

the

nati

onal

ad-h

ocm

odul

es,

with

am

axim

umdu

rati

onof

two

min

utes

and

are

com

men

dati

onto

have

them

atth

een

din

whi

chw

oudaf

fect

corn

para

bfli

y,T

cp.p

i!.t

9i!

roduce

the

Cut

o*on

Agt

a.y

J(a

,So

f.tg

and

Hnh

bJ

aTW

ab

Ope

nte

nd

erpro

cedure

EU

OSH

N2O

I7IO

PIF!

SE10

048—

Tec

hnic

alev

alua

tion

:

Page 16: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

tender

pro

cedure

EU

OS

HN

2OI

TIO

P/F1

SEIO

O48

—T

ech

nic

alev

alu

atio

n:

Jfaa

Sd.:y

aa4

H.ah

Sat

Wk

mod

ule1

part

icul

arly

ifth

ere

are

repe

titi

ons

with

core

ques

tion

s,is

appr

opri

ate.

The

aim

san

dap

proa

chof

the

pilo

tte

star

ere

leva

ntan

dpo

sitiv

ely

valu

edby

the

eval

uati

on

com

mit

tee.

30in

terv

iew

spe

rco

untr

yco

veri

ngal

lse

ctor

s,si

zes

—ov

ersa

mpl

ing

ofth

em

icro

esta

blis

hmen

ts-

with

avi

ewto

test

the

func

tion

ing

ofC

AT

WCA

WI,

the

ques

tion

nair

e,no

n-

resp

onse

,th

ein

form

atio

nfr

omno

n-re

spon

dent

s,ou

tcom

equ

esti

ons,

inte

rvie

wer

san

d

qual

ityco

ntro

l.R

esul

tsto

beus

edfo

ran

yre

quir

edch

anges

toth

equ

esti

onna

ire.

Wea

kn

esse

s

The

eval

uati

onco

mm

itte

eco

nsid

ered

that

invi

ting

the

Foc

alPo

inL

sto

the

web

inar

s,w

hile

bein

ga

reas

onab

leid

ea,

ispr

obab

lyno

tve

ryre

alis

tic.

Str

engt

hs

The

offe

rspr

ovid

esa

list

ofsa

mpl

ing

fram

esby

coun

try,

upda

ted

from

thos

eus

edin

ES

EN

ER

-2,

prop

osin

gso

me

chan

ges,

whi

chis

posi

tivel

yva

lued

byth

eev

alua

tion

com

mit

tee.

The

reIs

aso

und

intr

oduc

tion

tosa

mpl

ing,

base

don

the

Tot

alE

rror

Mod

el,

acco

unti

ngfo

rso

me

ofth

em

ain

issu

es.

The

offe

rpr

opos

esa

mul

tist

rati

fied

rand

omsa

mpl

ing:

26st

rata

(4si

zecl

asse

sx

7se

ctor

grou

ps)

asin

ES

EN

ER

-2bu

tpr

ovid

esan

alte

rnat

ive,

whi

chw

ould

have

bene

fite

dfr

omQ

uali

tyof

the

sam

plin

gst

rate

gy,

incl

udin

gth

epr

opos

edbu

sine

ssm

ore

deta

ils

and

acr

itica

las

sess

men

tfo

ra

prop

erev

alua

tion

.re

gist

ers,

the

sam

plin

gde

sign

invi

ewof

min

imis

ing

sele

ctio

nbi

asan

d—

fl--—

-—

——

--—

----—

--—

-—

-----—

-—

—---—

--—

-—

—--—

—-

---

----—

--

--

--

the

pote

ntia

lim

pact

son

com

para

bili

tyw

ithE

SE

NE

R-2

(15

poin

ts)

8-

Wea

kn

esse

s

As

poin

ted

out

abov

e,th

esa

mpl

ing

desi

gnw

asno

ten

tire

lycl

ear

alth

ough

the

eval

uati

onco

mm

itte

eun

ders

tood

that

the

tend

erer

initi

ally

sugges

tsto

follo

wth

esa

me

appr

oach

than

-th

eon

ein

ES

EN

ER

-2.

The

eval

uati

onco

mm

itte

eex

pres

sed

conc

erns

over

the

scre

enin

gpr

oced

ure

tose

lect

one

esta

blis

hmen

tin

mul

ti-un

itco

mpa

nies

.T

hesu

gges

ted

appr

oach

,sn

owba

llin

g,(a

skin

gco

mpa

nies

tose

lect

the

esta

blis

hmen

tto

bein

terv

iew

ed),

can

lead

tosi

gnif

ican

tbi

as,

asit

isa

non-

rand

ompr

oced

ure.

Mor

ede

tail

sw

ould

have

been

wel

com

eon

the

com

para

bili

tyw

ithE

SE

NE

R-2

.

CR

ITE

RIO

N3

WO

RK

PA

CK

AG

ES

:T

RA

ININ

G,

IMPL

EM

EN

TA

TIO

NA

ND

DE

LIV

ER

Y:

-S

CO

RE

:25

OU

TO

F35

PO

INT

S

Qua

lity

ofth

est

rate

gyfo

rfi

eldw

ork

prep

arat

ion,

part

icul

arly

the

Str

engt

hs

appro

ach

totr

aini

ngan

dth

eco

ntac

tst

rate

gyfo

rre

duci

ngno

n-re

spon

se.

The

over

all

appr

oach

tofi

eldw

ork

prep

arat

ion

ispo

siti

vely

valu

edby

the

eval

uati

on

incl

udin

glh

eon

line

mod

ule

(10

poin

ts)

7co

mm

itte

e.T

heus

eof

asi

ngle

soft

war

efo

rm

anag

ing

the

sam

ple,

cont

acti

ngth

eun

its,

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

___

prog

ram

min

gth

ecA

T!

and

CA

WI

ques

tionnai

res

and

reco

rdin

ath

eda

ta,

acro

ssal

l

Page 17: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

tu.p

nn

oniy

Jfa

rSaI

.ryaa

dfir

dib

i,IW

mk

Ope

nte

nd

erp

roce

dure

EU

OSH

N2O

I7!O

P/FI

SEIO

O48

—T

echn

ical

eval

uati

on:

coun

trie

san

dfo

rth

ew

hole

surv

eyis

very

rele

vant

.

The

reis

ade

tail

edde

scri

ptio

nof

the

scri

ptin

gof

the

mas

ter

ques

tion

nair

ean

dth

etr

ansl

ated

vers

ions

(inC

AT

Ian

dC

AW

I),

prov

idin

gE

U-O

SHA

the

poss

ibil

ity

tote

stth

esc

ript

,w

hich

isw

elco

me

byth

eev

alua

tion

com

mit

tee.

The

data

reco

rdin

gm

atri

xte

mpl

ate

with

all

surv

ey-r

elat

edda

taan

dpa

rada

tais

prop

osed

tobe

the

sam

eas

inE

SE

NE

R-2

.

The

appr

oach

totr

aini

ngis

pres

ente

dw

ithsu

ffic

ient

deta

ils

and

incl

udes

two

sem

inar

sfo

rm

anag

ers

and

supe

rvis

ors

inE

ngli

sh,

plus

grou

pbr

iefi

ngfo

rin

terv

iew

ers

(one

for

each

lang

uage

).In

terv

iew

ers

are

also

trai

ned

toco

nver

tre

fusa

lsan

dre

duce

non-

resp

onse

L(con

tact

stra

tegy

),w

Nch

isva

lued

posi

tive

lyby

the

eval

uati

onco

mm

itte

e.

Wea

knes

ses

Whi

leth

eov

eral

lap

proa

chto

trai

ning

isso

und,

the

eval

uati

onco

mm

itte

efo

und

the

prop

o5ed

trai

ning

sem

inar

sto

beto

osh

ort.

The

eval

uati

onco

mm

itte

eto

oex

pre

ssed

som

edo

ubts

asto

the

opti

onfo

rre

spo

nd

ents

tovi

sual

tyfo

llow

the

ques

tion

nair

eon

thei

rsc

reen

duri

ngth

ein

terv

iew

,as

itm

aybe

mor

eco

nfus

ing

thai

clar

ifyi

ng.

Con

cern

ing

the

offe

rof

the

onli

nesu

rvey

opti

on,

the

eval

uati

onco

mm

itte

ew

asno

ten

tire

lyco

nvin

ced

abou

tth

epr

opos

al,

sugg

esti

ngto

inte

grat

eit

asa

refu

sal

conv

ersi

onst

rate

gyra

ther

than

ala

stop

tion

for

non-

resp

onde

nts.

-

Qua

lity

ofth

epro

pose

dfi

eldw

ork

exec

utio

n,de

scri

bing

the

over

all

Fiel

dwor

kis

tobe

carr

ied

out

byfr

omfo

urhu

bs:

—L

aan

dap

pro

ach

toguar

ante

ehi

gh-q

uali

tyin

terv

iew

san

dm

inim

ise

non-

JThe

offe

rp

rese

nts

mea

sure

sfo

rfi

eldw

ork

qual

ity

cont

rol,

incl

udin

gth

em

onit

orin

gre

sponse

bias

incl

udin

gth

em

onit

orin

g,co

ntro

lan

dre

port

ing

ofth

eof

inte

rvie

wer

s—

with

the

use

ofan

inte

grat

edto

olfo

rfi

eldw

ork

mon

itori

ng-,

wee

kly

repo

rts,

fiel

dwor

kp

roce

ss(1

5po

ints

)10

feed

back

sess

ions

inca

seof

nega

tive

resu

lts

plus

ase

ries

ofm

easu

res

tom

axim

ise

resp

on

sera

tes,

whi

chha

vebe

ende

scri

bed

too

unde

r‘F

ield

wor

kpr

epar

atio

n’an

dar

ede

emed

tobe

rele

vant

.

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

est

rate

gyto

redu

ceno

n-re

spon

se,

spec

ific

ally

the

man

agem

ent

ofth

esa

mpl

e(s

ampl

ere

leas

esby

coun

try,

bas

edon

resp

onse

rate

s)an

dfo

cusi

ngef

fort

son

the

hard

erto

reac

h,w

hich

isso

und.

The

cont

act

stra

tegy

isre

leva

ntto

o,in

clud

ing

the

calli

ngti

mes

,th

e10

cont

act

atte

mpt

spe

run

it(w

ithtw

oex

cept

ions

,w

hich

are

reas

onab

le),

plus

the

info

rmat

ion

emai

lan

dth

ew

ebpa

ge.

Reg

ardi

ngth

ere

fusa

lco

nver

sion

stra

tegi

es,

ther

eis

afa

irpo

int

mad

eon

the

limite

din

form

atio

nco

llec

ted

onth

ere

asons

for

refu

sals

inE

SE

NE

R-2

and

the

poss

ible

reas

ons,

with

way

sto

over

com

eth

em,

whi

chis

soun

dan

dre

leva

nt.

Page 18: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nder

pro

cedure

EU

OSH

N2O

I7/O

PIF/

SEIO

O4B

—T

echn

ical

eval

uati

on:

jkr

Soi

Hyd

H,d

th.J

&W

wk

Wea

kn

esse

s

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

efu

rthe

rde

tail

sto

the

prop

osed

appr

oach

tofi

eldw

ork

exec

utio

n.T

hem

easu

res

desc

ribe

dto

incr

ease

resp

onse

rate

s,w

hile

reas

onab

le,

wer

ede

emed

tobe

abi

tst

anda

rd.

The

reis

lack

ofcl

arity

rega

rdin

gth

eal

loca

tion

ofw

ork

and

inte

rvie

wer

sac

ross

the

four

hubs

inte

rms

ofco

untr

yco

vera

ge.

The

reis

also

som

eco

nfus

ion

conc

erni

ngth

ela

ngua

geco

vera

gepe

rco

untr

y,w

hich

does

not

com

ply

with

the

requ

irem

ents

set

inth

ete

chni

cal

spec

ific

atio

ns.

Str

engt

hs

The

use

ofa

sing

lesc

ript

and

cent

ral

data

colle

ctio

npl

atfo

rmis

posi

tivel

yva

lued

byth

eev

alua

tion

com

mit

tee

asal

lin

terv

iew

data

are

acce

ssib

lein

stan

tly.

Con

cern

ing

data

vali

dati

on,

the

use

ofa

cent

ral

scri

ptre

duce

spa

rtof

data

clea

ning

but

the

offe

rpr

esen

tsad

diti

onal

chec

ks,

such

asin

terv

iew

dura

tion

(too

shor

t/lo

ng),

doub

lein

terv

iew

s,st

raig

htlin

ing,

cons

iste

ncy,

item

non-

resp

onse

(<40

%),

outl

iers

,en

gage

men

t,..,

whi

char

ede

emed

tobe

rele

vant

byth

eev

alua

tion

com

mit

tee.

The

appr

oach

tow

eigh

ting

isso

und

and

wel

lde

scri

bed,

incl

udin

gth

eim

pact

on

com

para

bili

ty.

Qua

lity

ofth

eap

proa

chto

data

proc

essi

ng,

deli

very

and

stor

age,

incl

udin

gw

eigh

ting

and

the

impa

cton

com

para

bili

tyw

ithE

SE

NE

R-2

,as

Dat

apr

oces

sing

proc

ess

incl

udes

asc

reen

ing

phas

e,di

agno

stic

phas

ean

dtr

eatm

ent

wel

las

the

repo

rtin

g(1

0po

ints

)B

phas

e,w

ithth

esu

bmis

sion

ofa

data

file

with

allca

ses

whi

chfa

iled

the

qual

itych

eck

proc

edur

es.

Ago

odpo

int

ism

ade

onth

eEU

Gen

eral

Dat

aP

rote

ctio

nR

egul

atio

n.

The

appr

oach

toth

ede

live

rabl

es,

the

repo

rtin

gan

dm

eeti

ngs

isde

tail

edan

dth

orou

gh,

inlin

ew

ithte

nder

requ

irem

ents

.

Wea

kn

esse

s

The

eval

uati

onco

mm

itte

eex

pres

sed

som

eco

ncer

nsas

toth

eap

proa

chof

the

codi

ngof

the

activ

ityse

ctor

,w

hich

isau

tom

ated

usin

gke

ywor

dsaf

ter

tran

slat

ion

into

Eng

lish

.

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

ede

tail

son

the

data

stor

age.

TO

TA

L67O

UT

OF

IOO

Page 19: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

-0-c

U)CC)

U)

0.

D

0

‘Ix—,

Page 20: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

,i

t.Luy

JcI

Ww

I

Open

ten

der

pro

ced

ure

EU

OS

HN

2OI7

10PI

FISE

IOO

48—

Tec

hn

ical

eval

uat

ion

:

CR

ITE

RIO

NC

OM

ME

NT

S

CR

ITE

RIO

N1

EF

FIC

IEN

CY

AN

DQ

UA

LIT

YO

FT

HE

PR

OJE

CT

MA

NA

GE

ME

NT

:-

SC

OR

E:

21O

UT

OF

30P

OIN

TS

Str

engt

hs

The

appro

ach

topr

ojec

tm

anag

emen

tan

dw

ork

orga

nisa

tion

,in

clud

ing

the

reso

urc

eal

loca

tion

,th

efe

asib

ilit

yof

the

prop

osed

tim

etab

lean

dth

eco

ntin

genc

ypl

anfo

run

fore

seen

even

tsor

com

plic

atio

ns,

incl

udin

gpro

cedure

sfo

rqu

alit

yco

ntro

lan

das

sura

nce

.(1

0po

ints

)7

The

reis

acl

ear

desc

ript

ion

ofth

eor

gani

sali

ons

invo

lved

inth

eof

fer

and

the

diff

eren

tte

ams

that

will

beta

king

care

ofth

eta

sks.

The

reis

ale

adpr

ojec

tm

anag

erap

poin

ted

toco

ntac

tw

ithE

U-O

SHA

and

coor

dina

teth

e

core

team

and

the

nati

onal

part

ners

.A

dditi

onal

lyth

ere

isa

pers

onin

char

geof

qual

ity

cont

rol

and

risk

asse

ssm

ent.

Mos

tof

the

units

are

loca

ted

inIt

aly

and

the

appr

oach

to

proj

ect

man

agem

ent

iscl

ear

and

quit

ece

ntra

lise

d,w

hich

isva

lued

posi

tive

lyby

the

eval

uati

onco

mm

itte

e.

The

prop

osed

tim

etab

leis

gene

rall

yin

line

with

the

requ

irem

ents

set

inth

ete

chni

cal

3spe

cifi

cati

ons.

Wea

knes

ses

Whi

leth

eap

proa

chto

proj

ect

man

agem

ent

iscl

ear,

mor

ede

tail

sw

ould

have

been

wel

com

edon

how

the

diff

eren

tun

its

will

liai

seam

ong

each

othe

r.

The

prop

osed

timel

ine

isfe

asib

lebu

tth

eev

alua

tion

com

mit

tee

exp

ress

edco

ncer

nsov

er

carr

ying

out

fiel

dwor

kov

erth

esu

mm

er.

The

eval

uati

onco

mm

itte

efo

und

the

appr

oach

toqu

ality

cont

rol

reas

on

able

but

itw

ould

have

expe

cted

itto

bem

ore

embe

dded

inth

epr

opos

al.

Mor

ede

tail

and

tailo

ring

toth

e

spec

ific

need

san

dch

alle

nges

ofE

SE

NE

R-3

wou

ldha

vebe

enw

elco

med

.

Fur

ther

toth

is,

the

appr

oach

tori

skas

sess

men

tan

dth

eco

ntin

genc

ypl

anco

uld

have

been

expa

nded

.

The

eval

uati

onco

mm

itte

ew

ould

have

appr

ecia

ted

deta

ils

onda

tapr

otec

tion

.

Str

engt

hs

Ext

ent

and

qual

ity

ofth

eco

rete

am,

incl

udin

gth

eco

mpo

siti

onin

term

sof

exper

tise

and

skil

ls,

allo

cati

onor

role

san

dre

spon

sibi

liti

esof

all

team

mem

ber

sas

wel

las

reso

urc

eba

ck-u

p(1

0po

ints

)7

The

eval

uati

onco

mm

itte

eva

lues

very

posi

tive

lyth

em

ixof

skill

san

dex

pert

ise

ofth

eco

re

team

.T

here

isve

ryhi

ghex

pert

ise

onsu

rvey

met

hodo

logy

,sa

mpl

ing

and

data

anal

ysis

,as

wel

las

ling

uist

ics,

with

anap

poin

ted

expe

rtto

read

thro

ugh

the

fina

lre

port

s.

The

reis

are

ason

able

allo

cati

onof

reso

urc

es(in

term

sof

days

)ac

ross

all

envi

sage

dta

sks,

whi

chis

inlin

ew

ithth

ete

chni

cal

requ

irem

ents

.

Page 21: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

tender

pro

ced

ure

EU

OS

HN

2OI7

IOPI

F1SE

IOO

48—

Tec

hnic

alev

alu

atio

n:

)I,,

Salsy

wIl

sthh

_I

,JW

k

Wea

knes

ses

The

eval

uati

onco

mm

itte

eex

pre

ssed

conc

erns

abou

t(h

ela

ckof

reso

urce

back

-ups

.

Whi

leth

eex

tent

and

qual

ityof

the

core

team

was

foun

dto

beve

rygo

od,

the

eval

uati

onco

mm

itte

ew

ould

have

wel

com

eda

grea

ter

OSH

expe

rtis

ean

din

volv

emen

t.

The

min

imum

Eng

lish

leve

lre

quir

edfo

rth

ose

liais

ing

with

EU

-OSH

Ais

Cl,

whi

chis

not

the

case

ofth

equ

ality

man

ager

and

the

sam

ple

desi

gnan

dw

eigh

ting

lead

ers.

Str

engt

hs

The

offe

rpr

esen

tsfo

urhu

bs,

cove

ring

all

lan

gu

ages

and

coun

trie

s.T

here

are

two

—__ca

llce

ntre

sin

and

plus

two

part

ner

hubsm

aan

d

Inea

chhu

ba

proj

ect

man

ager

and

afi

eldw

ork

supe

rvis

orm

anag

erar

eap

poin

ted.

The

latt

erre

port

toth

efo

rmer

and

lead

the

coun

try

supe

rvis

ors,

who

then

liai

sew

ithth

ein

terv

iew

ers.

The

sam

eso

ftw

are

(is

tobe

used

for

all

inte

rvie

ws

acro

ssal

lco

untr

ies.

The

tend

erer

offe

rsa

cent

rali

sed

appr

oach

,ex

plai

ning

clea

rly

itsbe

nefi

tsin

term

sof

cons

iste

ncy

and

harm

onis

atio

n,w

hich

isde

emed

tobe

reas

onab

leby

the

eval

uati

onco

mm

itte

e.Q

uali

tyof

the

prop

osed

netw

ork

offi

eldw

ork

rese

arch

cent

res,

the

The

tend

erer

can

offe

r40

0C

ATI

wor

ksta

tion

s,ap

prop

riat

efo

rth

ete

chni

cal

requ

irem

ents

ofna

tion

alpr

ojec

tm

anag

ers

and

the

fiel

dwor

ksu

perv

isor

s(1

0po

ints

)7

ES

EN

FR

-3.

Wea

knes

ses

The

eval

uati

onco

mm

itte

eex

pre

ssed

som

eco

ncer

nsab

out

the

conc

entr

atio

nof

fiel

dwor

kin

one

ofth

ehu

bs(

cove

ring

upto

18co

untr

ies.

Fur

ther

toth

is,

the

sam

ere

sou

rces

are

allo

cate

d—

inte

rms

oflo

cal

proj

ect

man

ager

and

fiel

dwor

ksu

perv

isor

man

ager

days

-ac

ross

all

four

hubs

,de

spit

eth

edi

ffer

ent

num

ber

ofco

untr

ies

cove

red.

Mor

ede

tail

son

the

coor

dina

tion

with

the

non

-hub

sw

ould

have

been

wel

com

ed.

The

eval

uati

onco

mm

itte

eex

pre

ssed

som

edo

ubts

abou

tth

eex

pert

ise

of—

—w

hich

--

-ap

pea

rsto

conc

entr

ate

mor

eon

tele

mar

keti

ng,

dire

ctm

arke

ting

and

mar

ket

inve

stig

atio

n.

CR

ITE

RIO

N2

WO

RK

PAC

KA

GE

A:

QU

EST

ION

NA

IRE

AN

DSA

MPL

ING

:-S

CO

RE

:29

OU

TO

F35

PO

INT

S

Qua

lity

ofth

epr

opos

alto

the

ques

tion

nair

ere

visi

on,

incl

udin

gth

eS

tren

gths

sug

ges

ted

appro

ach

tone

wto

pics

,th

est

rate

gyto

gath

erin

form

atio

nT

heap

proa

chto

the

ques

tion

nair

ere

visi

onis

very

thor

ough

and

wel

lde

scri

bed

inge

nera

l.It

from

non-r

esponden

tsan

dth

epr

e-te

st(1

0po

ints

)7

Icon

sist

sof

thre

eco

nsec

utiv

est

eps:

(1)

in-d

epth

lite

ratu

rere

view

toid

entif

ypo

tent

ial

new

I-

Page 22: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nder

pro

ced

ure

EU

OSH

AI2

OI7

IOPI

FISE

IOO

48—

Tec

hnic

alev

alua

tion

:‘D

lS

oeI

wid

H.o

hhtt

W,k

whe

ther

toad

dqu

esti

ons.

Whi

leth

ere

are

nopr

opos

als

mad

e,th

eof

fer

incl

udes

som

e

exam

ples

,w

hich

are

rele

vant

.

The

reis

ave

ryso

und

appr

oach

toga

ther

ing

info

rmat

ion

from

non-

resp

onde

nts,

argu

ing

the

need

for

ast

rong

com

mun

icat

ion

stra

tegy

,m

akin

gan

emph

asis

onth

ein

itial

cont

acL

The

re

isa

deta

iled

desc

ript

ion

ofth

ein

form

atio

nto

beob

tain

ed,

eith

erin

the

inte

rvie

wor

via

web

sear

ches

,an

dho

wto

use

duri

ngth

ees

tim

atio

nphas

eor

duri

ngth

eco

nstr

ucti

onof

the

sam

pte

wei

ghts

,in

view

ofco

rrec

ting

the

bias

inth

ede

sign

wei

ght

indu

ced

byth

em

issi

ng

resp

onse

s.T

heev

alua

tion

com

mit

tee

valu

edth

ispr

opos

alve

rypo

sitiv

ely.

The

pre-

test

rega

rds

16in

-dep

thfa

ce-t

o-fa

cein

terv

iew

s,ac

ross

four

diff

eren

tse

ctor

san

d

thre

esi

zecl

asse

s.T

here

isa

good

mix

ofco

untr

ies

prop

osed

:B

elgi

um,

Est

onia

,F

ranc

e

and

Ital

y.—

_—

——

——

——

Wea

knes

ses

Whi

leth

eap

proa

chto

aski

ngth

eou

tcom

equ

esti

ons

isex

plai

ned

inde

tail

,w

ithre

leva

ntco

nsid

erat

ions

mad

e,th

eev

alua

tion

com

mit

tee

exp

ress

edso

me

conc

erns

abou

tas

king

thes

equ

esti

ons

befo

reth

ein

terv

iew

and

the

disc

oura

ging

effe

ctit

may

have

onre

spon

dent

s.

The

eval

uati

onco

mm

itte

ew

asno

ten

tire

lyco

nvin

ced

abou

tth

ede

mog

raph

icin

form

atio

n

abou

tth

ees

tabl

ishm

ent

owne

r—

who

may

not

alw

ays

beth

ere

spon

dent

toth

esu

rvey

-an

d

the

use

tobe

mad

eof

ii

The

rew

asa

deta

iled

and

soun

ddi

scus

sion

mad

eon

com

para

bili

tybu

tit

focu

sed

onsa

mpl

ing

and

wei

ghti

ngis

sues

and

not

som

uch

onth

ech

alte

nge5

toth

eco

gnit

ive

unde

rsta

ndin

gof

the

ques

tion

nair

eac

ross

diff

eren

tse

ctor

san

dsi

zecl

asse

s.

Mor

ede

tail

sw

ould

have

been

wel

com

edab

out

the

met

hodo

logy

ofth

epr

e-te

st,

incl

udin

gth

eis

sues

toco

ver.

The

reis

ave

ryso

und

and

deta

iled

prop

osal

totr

ansl

atio

n(w

hich

will

beca

rrie

dou

tby

Cap

stan

),in

line

with

the

requ

irem

ents

set

inth

ete

chni

cal

spec

ific

atio

nsan

dfo

llow

ing

the

TR

APD

appr

oach

.

Ath

ree-

step

appr

oach

toth

ena

tion

alad

-hoc

mod

ules

ispr

opos

ed,

focu

sing

mos

tly

onth

e

man

agem

ent

ofth

ejx

oces

s.T

hepr

opos

alis

soun

d,ra

isin

gre

leva

ntpo

ints

.

The

pilo

tte

sts

envi

sage

s60

inte

rvie

ws

per

coun

try,

mai

nly

inC

AT

t,w

itha

larg

een

ough

num

ber

ofm

icro

esta

blis

hmen

tsin

terv

iew

ed.

CA

WI

tobe

test

edto

o.T

heai

ms

ofth

epi

lot

are

rele

vant

and

its_

orga

nisa

lion

isw

ell

desc

ribe

d.

Qua

lity

ofth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

—w

hile

ensu

ring

inte

rnat

iona

lco

mpa

rabi

lity

-,in

clud

ing

the

nati

onal

ad-h

ocm

odul

es,

asw

ell

asth

epr

opos

edap

proa

chto

tran

slat

ion

and

the

pilo

tte

st(1

0po

ints

)9

Page 23: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

tender

pro

ced

ure

EU

OS

HA

I2O

I7IO

PIF

/SE

1004

8—T

echnic

alev

alu

atio

n:

3iw

W,I

yoad

H.th

h.J

oIW

k

Wea

knes

ses

Whi

leth

eap

proa

chto

the

nati

onal

ad-h

ocm

odul

ew

asso

und,

mor

ede

tail

sw

ould

have

bee

nw

elco

med

abou

tth

eIn

clus

ion

ofth

equ

esti

ons.

Str

engt

hs

The

reis

ave

rygo

odan

dde

tail

edap

proa

chto

sam

plin

gde

sign

.

The

proc

ess

isto

bem

anag

edce

ntra

lly

and

pre

sen

tsa

prob

abili

tysa

mpl

est

rate

gyto

redu

cese

lf-s

elec

tion

bias

,ad

opti

ngsp

ecif

ices

tim

ator

sth

atex

ploi

tau

xilia

ryva

riab

les

(suc

has

gene

rali

sed

regr

essi

ones

tim

ator

sor

cali

brat

edes

tim

ator

s),

whi

leai

min

gto

incr

ease

resp

on

sera

te,

with

use

ofhi

ghqu

ality

sam

ple

fram

esan

dst

rong

com

mun

icat

ion

phas

e.

Str

atif

icat

ion

in48

stra

tais

prop

osed

(fou

rm

acro

sect

ors

xfo

ursi

zecl

asse

sx

thre

ere

gion

s)in

ever

yco

untr

y,ne

xtth

eal

loca

tion

met

hod

isde

fine

dw

ithsi

xpl

anne

ddo

mai

ns,

thre

eat

coun

try

leve

lan

dth

ree

atth

eE

Ule

vel.

Tar

gets

per

stra

tum

are

tobe

defi

ned

inor

der

toen

sure

the

desi

red

leve

lof

relia

bilit

yin

each

ofth

esi

xdo

mai

ns,

and

will

bech

ose

nin

adva

nce

fixi

nga

valu

efo

rth

eco

effi

cien

tof

vari

atio

n(C

V).

Thi

sis

very

rele

vant

and

inte

rest

ing.

Qua

lity

ofth

esa

mpl

ing

stra

tegy

,in

clud

ing

the

prop

osed

busi

ness

The

reis

the

defi

nitio

nof

incl

usio

npr

obab

ilit

yan

dsa

mpl

eex

trac

tion

,w

itha

rand

omre

gist

ers,

the

sam

plin

gde

sign

invi

ewof

min

imis

ing

sele

ctio

nbi

asan

dse

lect

ion

ofun

its

with

vari

able

prob

abil

ity

sam

plin

g(P

roba

bili

tyP

ropo

rtio

nal

toS

ize)

.th

epo

tent

ial

impa

cts

onco

mpa

rabi

lity

with

ES

EN

ER

-2(1

5po

Ints

)13

follo

wed

byth

ees

tim

atio

nph

ase,

whi

chis

very

wel

lde

scri

bed

and

expl

aine

d.E

stim

ates

will

bepr

oduc

edre

ferr

ing

toal

lco

untr

ies,

the

EU

and

each

coun

try,

both

for

esta

blis

hmen

tsan

dw

orke

rs.

Cal

cula

tion

ofth

esa

mpl

ing

erro

rfo

rea

ches

tim

ate

will

bede

fine

din

the

proj

ect,

linke

dto

cali

brat

edes

tim

ator

s.

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

edi

scus

sion

onco

mpa

rabi

lity

and

the

impa

ctth

epr

opos

edch

ang

esto

the

sam

plin

gst

rate

gyw

illha

ve.

Wea

knes

ses

The

size

ofth

ein

itial

sam

ple

rele

ase

was

dee

med

tobe

too

high

byth

eev

alua

tion

com

mit

tee.

Mor

ede

taU

sw

ould

have

bee

nw

elco

med

abou

tth

efe

asib

ility

and

chal

leng

esof

usin

gof

ficia

lre

gist

ers.

Fur

ther

toth

is,

the

alte

rnat

ive

prop

osal

toop

tfo

rth

esa

me

com

mer

cial

bu

sin

ess

regi

ster

acro

ssal

lco

untr

ies,

Bur

eau

Van

Dijk

.w

asno

tco

nvin

cing

.

CR

ITE

RIO

N3

WO

RK

PAC

KA

GE

B:

TR

AIN

ING

,IM

PLE

ME

NT

AT

ION

AN

DD

ELIV

ER

Y:

-S

CO

RE

:31

OU

TO

F35

PO

INT

S

Qua

lity

ofth

est

rate

gy

for

fiel

dwor

kpr

epar

atio

n,pa

rtic

ular

lyth

eS

tren

gths

appro

ach

totr

aini

ngan

dth

eco

ntac

tst

rate

gyfo

rre

duci

ngno

n-re

spon

se,

The

reis

anin

tegr

ated

CA

TII

CA

WI

syst

emJ

whi

chen

com

pas

ses

ques

tion

nair

ein

clud

ing

the

onli

nem

odul

e(1

0po

ints

)pr

ogra

mm

ing,

sam

ple

man

agem

ent

and

mon

itor

ing.

Thi

sis

valu

edpo

siti

vely

byth

e

Page 24: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Lw

cpnnA

Qcn

syJ

k,W

nyaq1H

,dh

JQ

IWI

Open

tender

pro

ced

ure

EU

OSH

AI2

OI7

IOPI

FISE

IOO

48—

Tec

hn

ical

eval

uat

ion

:

eval

uati

onco

mm

itte

e.

Qua

lity

ofth

epr

opos

edfi

eldw

ork

exec

utio

n,de

scri

bing

the

over

all

appro

ach

tog

uar

ante

ehi

gh-q

uali

tyin

terv

iew

san

dm

inim

ise

non-

resp

on

sebi

asin

clud

ing

the

mon

itor

ing,

cont

rol

and

repo

rtin

gof

the

fiel

dwor

kpro

cess

(15

poin

ts)

15

Atr

aini

ngse

min

arbe

fore

fiel

dwor

kst

arts

isen

visa

ged,

follo

wed

bysp

ecif

ictr

aini

ngto

inte

rvie

wer

sin

each

coun

try

—by

the

nati

onal

proj

ect

man

ager

and

fiel

dwor

ksu

perv

isor

man

ager

inea

chof

the

hubs

.F

urth

erm

ore,

brie

fing

sat

coun

try

leve

lw

illbe

held

too.

The

re

isal

soa

poin

tof

the

tran

slat

ion

offi

eldw

ork

mat

eria

ls,

whi

chgo

esa

bit

beyo

ndth

ete

chni

cal

spec

ific

atio

ns.

The

over

all

appr

oach

isde

emed

tobe

reas

on

able

byth

eev

alua

tion

com

mit

tee.

The

appr

oach

toth

ere

duct

ion

ofno

n-re

spon

sei5

rele

vant

and

thor

ough

,w

ithw

ell-

just

ifie

d-

mea

sure

spr

opos

ed,

such

asse

ndin

gea

rly

lett

ers

-joi

ntly

sign

edby

the

resp

ecti

vena

tion

alst

atis

tics

inst

itut

e-,

usin

gex

peri

ence

din

terv

iew

ers

toco

nver

tre

fusa

ls,

havi

nggo

odqu

ality

regi

ster

s,in

crea

sing

the

num

ber

ofpo

tent

ial

cont

act

atte

mpt

san

ddr

afti

ngan

appr

opri

ate

•co

mm

unic

atio

nst

rate

gyfo

rco

ntac

t,am

ong

othe

rs.

On

the

cont

act

stra

tegy

,th

ere

are

upto

:15

call

sre

gard

ed(a

tdi

ffer

ent

tim

es)

befo

redi

scar

ding

aco

mpa

ny.

The

offe

ris

criti

cal

ofm

ixed

mod

esan

dw

ould

only

offe

rth

eon

line

mod

ule

inth

ose

coun

trie

sw

ithve

rylo

wre

spo

nse

rate

spl

usto

thos

ees

tabl

ishm

ents

that

wer

eun

reac

habl

evi

aph

one.

The

eval

uati

onco

mm

itte

efo

und

the

prop

osal

wel

lju

stif

ied

and

inte

rest

ing.

Wea

knes

ses

Mor

ede

tail

son

the

trai

ning

wou

ldha

vebe

enw

elco

med

byth

eev

alua

tion

com

mit

tee.

Itvm

sun

clea

rto

the

eval

uati

onco

mm

itte

eho

wm

any

adva

nce

lett

ers

will

bese

nt.

Str

engt

hs

The

four

hubs

are

clea

rly

defi

ned

inte

rms

ofth

eir

coun

try

cove

rage

,w

ithna

tion

alte

ams

inth

esa

me

plac

e,w

hich

isva

lued

posi

tive

lyby

the

eval

uati

onco

mm

itte

e.C

ount

rysu

perv

isor

sar

ena

tive

spea

ker

sof

the

lang

uage

.

The

reis

flex

ibili

typr

opos

edin

term

sof

inte

rvie

win

gti

mes

and

good

asse

ssm

ent

ofth

em

inim

uman

dm

axim

umnu

mbe

rof

inte

rvie

ws

per

inte

rvie

wer

.The

eval

uati

onco

mm

itte

ew

elco

mes

the

stra

tegi

esto

redu

cein

terv

iew

turn

over

,w

itha

key

initi

alse

lect

ion

and

ade

fini

tion

ofa

type

ofco

ntra

ctgr

anti

ngco

ntin

uity

.

Ade

tail

edlis

tof

qual

itych

ecks

,pe

rfor

med

byth

epr

ojec

tm

anag

eran

dth

efi

eldw

ork

supe

rvis

orm

anag

er,

ispr

ovid

ed.

Fiel

dwor

kw

eekl

yre

port

s,w

ithda

tabr

oken

dow

nby

coun

try,

are

tobe

prod

uced

.E

U-O

SHA

isw

elco

me

tovi

sit

the

hubs

and

will

begr

ante

dac

cess

toth

efi

eldw

ork

plat

form

.

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee.

Page 25: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

J!a

,54(d

yc,

MHc

vjth

_I

IWL

TI

Ope

nte

nder

pro

ced

ure

EU

OS

HN

2OI7

IOPI

FISE

IOO

4B—

Tec

hnic

alev

alua

tion

:

Str

engt

hs

Qua

lity

ofth

eap

proa

chto

data

proc

essi

ng,

deli

very

and

stor

age,

incl

udin

gw

eigh

ting

and

the

impa

cton

com

para

bili

tyw

ithE

SE

NE

R-2

,as

wel

las

the

repo

rtin

g(1

0po

ints

)8

The

over

all

appr

oach

toda

tapr

oces

sing

isso

und.

Ago

odpo

int

ism

ade

onth

eim

port

ance

ofsc

ript

ing,

asa

key

stag

eof

data

vali

dati

on,

ensu

ring

rule

san

dco

ndit

ions

are

the

sam

eac

ross

all

fiel

dwor

kce

ntre

s.T

esti

ngof

the

ques

tion

nair

esc

ript

ing

isen

visa

ged,

with

exam

ples

ofth

ety

pes

ofte

sts

usua

lly

perf

orm

edby

-Dat

ava

lida

tion

isto

beco

mpl

eted

with

post

-che

ckin

gac

tivi

ties

.

The

only

open

ques

tion

isth

ede

scri

ptio

nof

the

activ

ityse

ctor

,fo

rw

hich

codi

ngag

ents

will

beus

ed.

The

reis

aso

und

appr

oach

tow

eigh

ting

—us

eof

cali

brat

ion

esti

mat

ors-

,w

hich

isve

rylin

ked

toth

esa

mpl

ing

stra

tegy

,w

itha

deta

iled

desc

ript

ion

ofth

epr

oced

ure

tode

fine

them

inth

ree

step

s.T

heev

alua

tion

com

mit

tee

valu

esit

very

posi

tivel

y.W

hen

itco

mes

toth

eco

mpa

rabi

lity

with

ES

EN

ER

-2,

the

offe

rar

gues

that

both

proc

edur

esle

adto

unbi

ased

resu

lts,

the

only

diff

eren

cebe

ing

the

relia

bilit

yof

esti

mat

es.

Wea

knes

ses

The

eval

uati

onco

mm

itte

ew

ould

have

appr

ecia

ted

mor

ein

form

atio

non

the

sto

rag

eof

the

data

.

Whi

leth

ede

live

rabl

esar

ein

line

with

the

requ

irem

ents

,m

ore

deta

ils

wou

ldha

vebe

enw

elco

med

byth

eev

atua

tion

com

mit

tee.

TO

TA

L81

OU

TO

F10

0

pa.

Sig

ned

by:

PR

U

Page 26: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nder

pro

ced

ure

EU

OSH

A)2

0171

0P/F

ISE

IOO

48—

Tec

hnic

alev

alua

tion

:I

irW

.yandK

,cH

_J

iUk

CR

ITE

RIO

NC

OM

ME

NT

S

CR

ITE

RIO

NI

EF

FIC

IEN

CY

AN

DQ

UA

L1Y

OF

TH

EP

RO

JEC

TM

AN

AG

EM

EN

T:

-S

CO

RE

:22

OU

TO

F30

PO

INT

S

Str

engt

hs

The

offe

rp

rese

nts

ade

tail

edan

dso

und

appr

oach

topr

ojec

tm

anag

emen

t,de

mon

stra

ted

to

beef

fect

ive

inca

rryi

ngou

tsi

mil

aras

sign

men

tsIn

the

past

.

The

prop

osed

team

isor

gani

sed

inta

sk-b

ased

sub-

team

s(p

roje

ctm

anag

emen

t;

ques

tion

nair

e,su

rvey

desi

gnan

dtr

ansl

atio

ns;

sam

plin

gan

dw

eigh

ting

;sc

ript

ing

and

dat

a

proc

essi

ng),

unde

rth

edi

rect

ion

ofa

Pro

ject

Lea

der

supp

orte

dby

aD

eput

yP

roje

ctL

eade

r,

who

are

both

supp

orte

dby

two

Pro

ject

Exe

cuti

ves.

Aqu

ality

dire

ctor

,su

ppor

ted

bya

Del

iver

yM

anag

erar

ein

clud

edin

the

team

toen

forc

equ

ality

arra

ngem

ents

.T

hepr

opos

alis

suff

icie

ntly

dea

ran

dro

les

and

resp

onsi

bili

ties

are

expl

aine

dsa

tisf

acto

rily

.

The

num

ber

ofda

ysal

loca

ted

toea

chte

amm

embe

rar

ege

nera

lly

cons

iste

ntw

ithth

ety

pe

and

num

ber

ofta

sks

tobe

carr

ied

out.

The

appr

oach

topr

ojec

tm

anag

emen

tan

dw

ork

orga

nisa

tion

,in

dudi

ngT

hepr

opos

edti

met

able

seem

sfe

asib

lean

din

tine

with

tend

erre

quir

emen

ts,

incl

udin

ga

the

reso

urc

eal

loca

tion

,lh

efe

asib

ilit

yof

the

prop

osed

tim

etab

lean

dth

ere

flec

tion

onth

etig

htsc

hedu

le,

whi

chth

eev

alua

tion

com

mit

tee

valu

espo

sitiv

ely.

cont

inge

ncy

plan

!or

unfo

rese

enev

ents

orco

mpl

icat

ions

,In

dudi

ngE

lem

ents

for

aqu

ality

and

assu

rance

plan

are

exte

nsiv

ely

desc

ribe

d,in

clud

ing

proc

edur

es(o

rqu

alit

yco

ntro

lan

assu

ran

ce.

(10

poin

ts)

8re

spon

sibi

liti

esfo

rdel

iver

able

san

dex

ampl

esof

qual

ity

indi

cato

rsan

dri

sks.

The

reis

ave

ry

rele

vant

poin

tm

ade

onth

ene

edCo

mee

tth

eob

liga

tion

sun

der

EG

DT

Ran

dho

wth

ete

nder

eris

curr

entl

yre

view

ing

thei

rco

mpa

nypro

cess

esan

dpo

lici

esac

cord

ingl

y.

Ris

kas

sess

men

tan

dco

ntin

genc

iesl

rem

edia

lac

tion

sar

ere

port

ed.

Wea

knes

ses

Whi

leth

epr

opos

edti

met

able

isre

ason

able

,th

eev

alua

tion

com

mit

tee

expre

ssed

som

edo

ubts

over

carr

ying

out

fiel

dwor

kin

July

.

Als

o,th

ere

sour

ces

and

time

devo

ted

totr

ansl

atio

nap

pea

red

tobe

inex

cess

ofth

ere

quir

emen

tsin

the

tech

nIca

lsp

ecif

icat

ions

,w

itha

five

-mon

thdu

rati

onof

the

tran

slat

ion

phas

ean

d14

5da

ysal

loca

ted

tolh

etr

ansl

atio

nm

anag

er(i

nad

diti

onof

the

wor

kca

rrie

dou

tby

the

subc

ontr

acte

dco

mpa

ny,

Str

engt

hs

Ext

ent

and

qual

ity

ofth

eco

rete

am,

incl

udin

gth

eco

mpo

siti

onin

term

sof

The

reis

ave

ryex

pert

cent

ral

coor

dina

tion

team

,pr

esen

ting

ago

odm

ixof

skill

san

dex

pert

ise

and

skill

s,al

loca

tion

orro

les

and

resp

onsi

bili

ties

ofal

lte

amex

peri

ence

,pa

rtic

ular

lyin

term

sof

man

agem

ent

ofsi

mila

rpr

ojec

ts,

surv

eyde

sign

,an

dm

embe

rsas

wel

las

reso

urc

eba

ck-u

p(1

0po

ints

)7

stat

isti

cs.

The

allo

cati

onof

days

isva

lued

posi

tive

lyby

the

eval

uati

onco

mm

itte

e,ge

nera

lly

cons

iste

nt

Page 27: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nd

erp

roce

dure

EU

OSH

N2O

I7IO

PIFI

SEIO

O4B

—T

echn

ical

eval

uati

on:

jlW

.iyfl

Ihth

b_.i

d1k

wfth

the

task

sto

beca

rrie

dou

t.

The

reis

ave

ryex

tens

ive

back

-up

plan

,w

ithal

loca

ted

back

-up

staf

f.

Wea

knes

ses

Whi

leth

ere

are

two

expe

rts

prop

osed

for

the

ques

tionn

aire

desi

gn,

the

eval

uatio

nco

mm

ittee

wou

ldha

vew

elco

med

addi

tiona

lex

pert

ise

onO

SH.

Mor

ecl

arity

wou

ldha

vebe

enap

prec

iate

dco

ncer

ning

the

defin

ition

ofm

anag

emen

tan

dex

ecut

ive

role

sw

ithin

the

cent

ral

coor

dina

tion

team

.

Inlin

ew

ithth

is,

the

com

mun

icat

ion

betw

een

(1)

the

sam

plin

gan

dw

eigh

ting

team

and

(2)

the

data

proc

essi

ngte

amco

uld

have

been

mor

ede

taile

d.

Stre

ngth

s

The

reIs

acl

ear

desc

ript

ion

ofth

ene

twor

kof

field

wor

kre

sear

chce

ntre

s,en

suri

nga

cove

rage

ofal

l cou

ntri

es,

asre

quir

edin

the

tech

nica

lsp

ecif

icat

ions

.Fo

reac

hlo

cal

part

ner

the

nam

eof

the

Proj

ect

man

ager

and

ofth

eFi

eldw

ork

supe

rvis

orar

ere

port

ed,

with

rele

vant

Qua

lity

cith

epr

opos

edne

twor

kof

field

wor

kre

sear

chce

ntre

s,th

eex

peri

ence

.na

tiona

lpr

ojec

tman

ager

san

dth

efie

ldw

ork

supe

rvis

ors

(10

poin

ts)

7_

__

_-

_.

Wea

knes

ses

The

eval

uatio

nco

mm

ittee

wou

ldha

vew

elco

med

deta

ilson

(he

num

ber

ofw

orks

tatio

ns.

Som

eco

ncer

nsw

ere

expr

esse

dab

out

the

proj

ect

man

ager

and

field

wor

ksu

perv

isor

[or

Lux

embo

urg

nse

thi

any

ofth

eof

ficia

lJanuagof

the

pyij

y,.

CR

ITE

RIO

N2

WO

RK

PAC

KA

GE

kQ

UES

TIO

NN

AIR

EA

ND

SAM

PLIN

G:-

SCO

RE

:25

OU

TO

F35

POIN

TS

Stre

ngth

s

The

offe

rse

tsth

ege

nera

lpr

inci

ples

for

the

ques

tionn

aire

revi

sion

,with

sugg

esti

ons

alon

gth

elin

esof

the

tech

nica

lre

quir

emen

ts.

The

reIs

apr

opos

alto

test

the

surv

eyin

trod

ucto

ryte

xt,w

hich

was

posi

tivel

yva

lued

byth

eev

alua

tion

com

mitt

ee.

The

eval

uatio

nco

mm

ittee

appr

ecia

tes

the

them

atic

area

spr

esen

ted

for

new

topi

csan

dQ

ualit

yof

the

prop

osal

toth

equ

estio

nnai

rere

visi

on,

incl

udin

gth

eso

me

sugg

estio

nspr

ovid

edon

Impr

oved

scal

esan

dth

eex

ampl

esof

whe

reit

wou

ldbe

sugg

este

dap

proa

chto

new

topi

cs,

the

stra

tegy

toga

ther

info

rmat

ion

rele

vant

tore

view

the

ques

tionn

aire

wor

ding

for

MSE

s.G

ood

prop

osal

toha

vea

glos

sary

from

non-

resp

onde

nts

and

the

pre-

test

(10

poin

ts)

7fo

rsm

alle

mpl

oyer

s.

The

reis

are

leva

ntpo

int m

ade

onth

e•la

nd

the

info

rmat

ion

notic

eon

the

cons

ent

topa

rtic

ipat

e.

The

reis

aso

und

appr

oach

toth

ein

clus

ion

ofou

tcom

equ

estio

ns,

com

bini

ngCA

TIan

dC

AW

I,w

hich

Isw

ell

desc

ribe

d,in

clud

ing

rele

vant

sugg

estio

ns.

Con

cern

ing

non-

resp

pnde

nts,

the

eval

uatio

nco

mm

ittee

valu

espo

siU

v&yt

hepr

opos

ed

Page 28: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

_I

ciw

.,k

Op

ente

nder

pro

cedure

EU

OSH

AJ2

OI7

/OP/

F1SE

JOO

48—

Tec

hnic

alev

alua

tion

:I

appr

oach

thro

ugh

the

mod

ific

atio

nof

the

scre

ener

,ai

min

gto

capt

ure

the

info

rmat

ion

imm

edia

tely

from

the

gat

ekee

per

sIn

case

they

are

not

part

icip

atin

gin

the

surv

ey.

The

pre-

test

isge

nera

lly

wel

lex

plai

ned,

cove

ring

alt

the

requ

ired

area

san

dca

rrie

dou

tby

rese

arch

ers

not

Inte

rvie

wer

s,In

clud

ing

tran

slat

ion

follo

win

gth

eT

RA

PD

appr

oach

.T

here

is

apr

oper

sam

plin

gpl

ansu

gg

este

dan

dde

brie

fing

sess

ions

with

rese

arch

ers

are

envi

sage

d

atco

untr

yle

vel

afte

ra

smal

lnu

mbe

rof

Inte

rvie

ws

have

been

com

plet

ed,

enab

ling

ongo

ing

and

iter

ativ

efe

edba

ck.

The

reIs

aso

und

prop

osal

toth

ere

port

ing

and

use

ofre

sult

sof

the

cogn

itiv

epr

e-te

sl.

Rel

evan

tco

nsid

ej’a

tion

sar

egi

ven

conc

erni

ngth

ead

apta

tion

ofth

equ

esti

onna

ire

for

the

onli

nem

ode,

Wea

knes

ses

Qua

lity

ofth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

—w

hile

ensu

ring

inte

rnat

iona

lco

mpa

rabi

lity

-,in

clud

ing

the

nati

onal

ad-h

ocm

odul

es,

asw

ell

asth

epr

opos

edap

proa

chto

tran

slat

ion

and

the

pilo

tte

st(1

0po

ints

)9

Whi

leth

eev

alua

tion

com

mit

tee

appr

ecia

tes

the

them

atic

area

spr

esen

ted

for

new

topi

cs,

spec

ific

exam

ples

orm

ore

deta

iled

ques

tion

prop

osal

sw

ould

have

bee

nw

elco

med

.

Fur

ther

toth

is,

eval

uati

onco

mm

itte

ew

asno

ten

tire

lyco

nvin

ced

abou

tth

epr

opos

alto

rand

omis

eor

phor

itis

eop

tion

alm

odul

eson

05F

fis

sues

.

The

rew

ere

som

edo

ubts

byth

eev

alua

tion

com

mit

tee

conc

emin

gth

eco

untr

yco

vera

geof

the

pre-

test

asw

ell

asth

euse

ofm

onet

ary

ince

ntiv

es.

Str

engt

hs

The

offe

rspr

esen

tsa

very

soun

dpr

opos

alfo

rth

equ

esti

onna

ire

tran

slat

ion,

follo

win

gth

eT

RA

POap

proa

chan

dca

rrie

dou

tby

asp

ecia

lise

dco

ntra

ctor

(CA

PST

AN

).

The

appr

oach

toth

ena

tion

alad

-hoc

mod

ules

isin

line

with

the

tech

nica

lsp

ecif

icat

ions

,th

eof

fer

mak

ing

apo

int

onco

mpa

rabi

lity

issu

esan

dsu

gges

ting

topl

ace

the

mod

ule

atth

een

dof

the

ques

tion

nair

e.

The

pilo

tte

stIs

wel

lex

plai

ned

and

inlin

ew

ithth

ete

chni

cal

requ

irem

ents

,ca

rryi

ngou

t30

quot

a-co

ntro

lled

inte

rvie

ws

per

coun

try

and

with

asp

ecif

icai

mto

test

the

scre

enin

g,in

clud

ing

info

rmat

ion

inca

seof

non-

resp

onse

,w

hich

isve

ryre

leva

nt.

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

eap

proa

chto

the

mon

itor

ing,

the

outl

ook

ofth

epi

lot

repo

rtan

dth

ede

scri

ptio

nof

how

the

pIlo

twill

beus

edto

info

rmth

ede

velo

pmen

tof

the

fina

lqu

esti

onna

ire.

Page 29: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nd

erpro

ced

ure

EU

OSH

N2O

I7IO

PIFI

SEIO

O48

—T

echn

ical

eval

uati

on:

1uS

d,I7

rHw

ith

.3cik

Wea

knes

ses

The

eval

uatio

nco

mm

ittee

expr

esse

dso

me

doub

tsab

out

the

use

ofa

ques

tion

bank

for

the

natio

nal

ad-h

ocm

odul

es,

whi

chco

uld

then

beta

ilore

dto

the

natio

nal

need

s.

Whi

leth

eap

proa

chto

the

pilo

tis

deem

edto

beve

ryst

rong

and

wel

lex

plai

ned,

lhe

eval

uatio

nco

mm

ittee

wou

ldha

veex

pect

edth

atit

wou

ldte

stth

eC

AW

Iqu

estio

nnai

reto

o.

Stre

ngth

s

The

reis

aso

und

roun

dup

ofsa

mpl

ing

issu

esin

asu

rvey

like

ESE

NE

R,

rais

ing

som

eco

ncer

nsab

outr

each

ing

the

max

imum

sam

ple

size

inth

egi

ven

timel

ine,

part

icul

arly

insm

allc

ount

ries

.

The

offe

rpr

esen

tsth

esa

mpl

efr

ames

byco

untr

yan

dty

peof

sam

plin

gun

it(e

stab

lishm

ent

vsco

mpa

ny)

that

have

been

iden

tifie

dto

beof

bette

rqu

ality

asco

mpa

red

toth

ose

used

for

Qua

lity

ofth

esa

mpl

ing

stra

tegy

,in

clud

ing

the

prop

osed

busi

ness

ESE

NE

R2.

regi

ster

s,th

esa

mpl

ing

desi

gnin

view

ofm

inim

isin

gse

lect

ion

bias

and

The

eval

uatio

nco

mm

ittee

valu

edpo

sitiv

ely

the

prop

osal

toin

clud

eca

ses

with

noph

one

the

pote

ntia

lim

pact

son

com

para

bilit

yw

ithE

SEN

CR

-2(1

5po

ints

)9

and

size

clas

sin

form

atio

n.

The

appr

oach

toth

ere

fere

nce

stat

istic

sth

atwi

llbe

used

Ispr

esen

ted

and

issi

mila

rto

the

one

adop

ted

inE

SEN

ER

-2

The

sam

plin

gst

rate

gyfo

llow

sth

eE

SEN

ER

-2ap

proa

chto

ala

rge

exte

ntan

dis

info

rmed

byth

ete

chni

cal

asse

ssm

ent

ofth

epr

evio

usro

und

ofth

esu

rvey

.R

elev

ant

poin

tsar

em

ade

onth

est

ratu

mju

mpe

rs,

and

how

the

sam

plin

gw

illbe

base

don

the

resp

onse

rate

sIn

ESE

NE

R-2

inor

der

tom

inim

ise

use

ofst

oppi

ngru

le.

Page 30: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nder

pro

cedure

EU

OSH

N2O

I7/O

PIFI

SE!0

048—

Tec

hnic

alev

alua

tion

:i

IiiW

iryti

Woik

Wea

knes

ses

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edad

diti

onal

deta

ils

onth

esa

mpl

ing

appr

oach

.It

issi

mil

arto

the

one

adop

ted

inE

SE

NE

R-2

,w

itha

focu

son

the

use

ofst

oppi

ng

rule

san

dth

est

ratu

mju

mpe

rs,

whi

char

eve

ryim

port

ant,

but

mor

epr

opos

als

addi

ngto

the

sam

plin

gde

sign

wou

ldha

vebe

enw

elco

med

.

Con

cern

ing

the

use

ofre

fere

nce

stat

isti

cs,

addi

tion

also

urce

sbe

yond

the

LFS

orex

ampl

es

wou

ldha

vebe

enw

elco

med

byth

eev

alua

tion

com

mit

tee.

The

eval

uati

onco

mm

itte

eex

pre

ssed

som

edo

ubts

abou

tno

ten

visa

ging

anop

enen

ded

ques

tion

for

the

righ

tal

loca

tion

ofth

eac

tivity

sect

or.

The

eval

uati

onco

mm

itte

ew

ould

have

expe

cted

mor

ede

tail

son

pote

ntia

lis

sues

affe

ctin

g

com

para

bili

ty.

CR

ITE

RIO

N3

WO

RK

PAC

KA

GE

B:T

RA

ININ

G•

IMPL

EM

EN

TA

TIO

NA

ND

DE

LIV

ER

Y:

-S

CO

RE

:27

OU

TO

F35

PO

INT

S

Str

engt

hs

The

appr

oach

tofi

eldw

ork

prep

arat

ion

isva

lued

posi

tivel

yby

the

eval

uati

onco

mm

itte

e.T

he

inte

grat

ion

betw

een

CA

TIan

dC

AW

Iis

wel

lex

plai

ned,

with

the

prop

osal

tous

ea

sing

le

plat

form

acro

ssal

lco

untr

ies,

SP

SS

Dim

ensi

ons

assc

ript

ing

soft

war

e.T

his

enab

les

appl

ying

cons

iste

ncy

chec

ksw

hich

late

rre

duce

the

need

for

clea

ning

and

edit

ing.

The

appr

oach

totr

aini

ngis

pres

ente

dw

ithsu

ffic

ient

deta

ilan

dfo

cuse

son

the

trai

ning

of

trai

ners

(tra

inin

gof

coun

try

team

lead

ers)

,w

how

illbe

inch

arge

citr

aini

ngth

ein

terv

iew

ers.

Qua

lrty

ofth

est

rate

gyfo

rfi

eldw

ork

prep

arat

ion,

part

icul

arly

the

The

mat

eria

lto

beus

ed(o

rth

efi

eldw

ork

and

for

the

trai

ning

will

bepr

ovid

edby

the

tend

erer

appr

oach

totr

aini

ngan

dth

eco

ntac

tst

rate

gyfo

rre

duci

ngno

n-re

spon

se,

inal

lth

ela

ngua

ges

ofth

elo

cal

part

ners

.in

clud

ing

the

onli

nem

odul

e(1

0po

ints

)9

The

eval

uati

onco

mm

itte

eva

lued

posi

tivel

yth

eco

ntac

tst

rate

gypr

opos

edto

redu

ceno

n-

resp

on

se,

incl

udin

gth

eof

feri

ngan

dfo

llow

upof

the

CA

WI

inte

rvie

w.

The

cont

act

prot

ocol

s

incl

ude

am

inim

umof

seve

nph

one

call

sin

diff

eren

tti

mes

and

days

over

ape

riod

of14

days

and

deal

ing

with

the

wro

ngnu

mbe

rs,

aim

ing

toIn

crea

seth

eco

ntac

tra

teup

to90

%.

Wea

knes

ses

The

eval

uati

onco

mm

itte

eex

pre

ssed

som

eco

ncer

nsre

gard

ing

the

use

ofw

eblin

kto

the

mai

nsu

rvey

plat

form

inab

out

14of

the

36C

ATI

loca

tion

s.

Qua

lity

ofth

epr

opos

edfi

eldw

ork

exec

utio

n,de

scri

bing

the

over

all

Str

engt

hs

appr

oach

toguar

ante

ehi

gh-q

uali

tyin

terv

iew

san

dm

inim

ise

non-

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

ein

volv

emen

tof

high

lysk

illed

and

wel

l-tr

aine

d

resp

on

sebi

asin

clud

ing

the

mon

itor

ing,

cont

rol

and

repo

rtin

gof

the

inte

rvie

wer

s.

fiel

dwor

kp

roce

ss(1

5po

ints

)11

The

mon

itori

ng,

cont

rol

and

repo

rtin

gar

ew

ell

pres

ente

d.W

eekl

yfi

eldw

ork

prog

ress

repo

rts

tobe

deli

vere

dea

chT

uesd

ay,

follo

wed

upby

are

view

mee

ting

with

EU

-OSH

Aon

Page 31: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Vk

,l,

Ope

nte

nder

proc

edur

eE

UO

SHN

2O1

7IO

PIFI

SEIO

O48

—T

echn

ical

eval

uati

onf

Wed

nesd

ayor

Thu

rsda

y.A

cces

sto

surv

eyan

dpr

ogre

ssda

tais

prop

osed

too

with

ada

shbo

ard

tom

onito

rin

terv

iew

san

dov

eral

l,th

ere

isa

soun

dap

proa

chto

chec

king

qual

ityis

sues

.T

heev

alua

tion

com

mitt

eeva

lued

posi

tivel

yth

epr

opos

alto

prov

ide

the

inte

rvie

wer

sw

ithfe

edba

dkon

thei

rpe

rfor

man

ce.

Wea

knes

ses

The

eval

uatio

nco

mm

ittee

wou

ldha

vew

elco

med

mor

ede

tails

toth

eap

proa

chon

field

wor

kex

ecut

ion.

Som

esu

btas

ksun

der

field

wor

kpr

epar

atio

nw

ere

pres

ente

dun

derf

ield

wor

kex

ecut

ion

and,

beyo

ndth

eap

proa

chto

mon

itori

ng,c

ontr

olan

dre

port

ing

ofth

efie

ldw

ork

proc

ess,

the

eval

uatio

nco

mm

ittee

felt

ther

ew

aslit

tlede

tail

give

n.

Stre

ngth

s

The

eval

uatio

nco

mm

ittee

appr

ecia

tes

the

appptm

ent

ofa

data

man

ager

asw

ell

asth

epo

intm

ade

onda

tase

curi

tyan

dth

eus

eof

The

appr

oach

toda

tava

lidat

ion,

proc

essi

ng,

deliv

ery

and

stor

age

are

wel

lpr

esen

ted

and

are

gene

rally

inlin

ew

ithth

ete

nder

requ

irem

ents

,as

the

repo

rtin

gan

dde

liver

able

sfo

rese

enin

the

prop

osal

.T

heev

alua

tion

com

mitt

eepa

rtic

ular

lyva

lued

the

appr

oach

toda

ta

Qua

lity

ofth

eap

proa

chto

data

proc

essi

ng,

deliv

ery

and

stor

age,

valid

atio

n,w

ithits

prop

osed

Inte

grat

ion

into

the

scri

ptan

dth

ein

volv

emen

tof

the

data

incl

udin

gw

eigh

ting

and

the

impa

cton

com

para

bilit

yw

ithE

SEN

ER

-2,

asm

anag

erth

roug

hout

the

diff

eren

tsur

vey

stag

es.

wel

las

the

repo

rtin

g(1

0po

ints

)7

The

appr

oach

tow

eigh

ting

issi

mila

rto

that

inE

SEN

ER

-2an

dth

ere

isa

rele

vant

poin

ton

the

com

pari

son

ofsa

mpl

esan

dun

iver

ses.

Wea

knes

ses

The

eval

uatio

nco

mm

ittee

wou

ldha

veap

prec

iate

dm

ore

clar

ityan

dde

tails

rega

rdin

gth

eap

proa

chto

wei

ghtin

g.T

heof

fer

prop

oses

calib

ratio

ndu

eto

the

incl

usio

nof

regi

onan

dho

wca

libra

tion

wei

ghts

wou

ldbe

equi

vale

ntto

one

ofth

ew

eigh

ting

fact

ors

used

inE

SEN

ER

-2(e

stex

)bu

tm

ore

deta

ilsw

ould

have

been

wel

com

ed.

TOTA

L74

OU

TO

F10

0

Page 32: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

ItUC

UwF-

4CC£130LI..

2I’rCC’d

0CDLu

•0U01aI0tC0+2

Ca,a0

D

a.

D

.0-o0C03U,La

Page 33: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

/Euiu

pec

nA

gem

yO

pen

ten

der

pro

cedu

reE

UO

SHN

2OI7

IOP/

FISE

/004

8—T

echn

ical

eval

uati

on:’

jin

,Sni

nty

and

Ihth

hJ

otW

k

CR

ITE

RIO

NC

OM

ME

NT

S

CR

ITE

RIO

N1

EF

FIC

IEN

CY

AN

DQ

UA

LIT

YO

FT

HE

PR

OJE

CT

MA

NA

GE

ME

NT

:-

SC

OR

E:

27O

UT

OF

30P

OIN

TS

Str

engt

hs

The

eval

uati

onco

mm

itte

eap

prec

iate

sth

ecl

ear

desc

ript

ion

ofth

eap

proa

chto

the

proj

ect

man

agem

ent

and

wor

kor

gani

sati

on,

whi

chis

foun

dto

beve

ryre

leva

ntan

dap

prop

riat

efo

rth

epu

rpos

eof

the

surv

ey.

The

reso

urce

allo

cati

onof

the

core

team

isde

emed

good

and

inlin

ew

ithth

ere

quir

emen

ts-

.of

the

tech

nica

lsp

ecif

icat

ions

.T

heap

pro

ach

topr

ojec

tm

anag

emen

tan

dw

ork

orga

nisa

tion

,in

clud

ing

the

reso

urc

eal

loca

tion

,th

efe

asib

ilit

yof

the

prop

osed

tim

etab

lean

dth

eT

heti

met

able

pres

ente

dis

reas

on

able

and

take

sac

coun

tof

the

tight

sche

dule

ofth

eco

ntin

genc

ypl

anfo

runfo

rese

enev

ents

orco

mpl

icat

ions

,In

clud

ing

{sur

vey,

whi

chis

very

rele

vant

pro

cedure

sfo

rqu

alit

yco

ntro

lan

assu

rance

.(1

0po

ints

)9

The

appr

oach

toth

equ

ality

cont

rol,

miti

gatio

nm

easu

res

and

risk

asse

ssm

ent

isex

plai

ned

inde

tail

and

very

thor

ough

ly.

The

eval

uati

onco

mm

iLte

eva

lues

itve

rypo

siti

vely

asit

ispr

oper

lyta

ilor

edto

the

nee

ds

ofE

SE

NE

R-3

.

Wea

knes

ses

The

eval

uati

onco

mm

itte

ew

ould

have

appr

ecia

ted

addi

tion

alin

form

atio

nre

gard

ing

the

reso

urce

allo

cati

onof

the

mem

bers

outs

ide

the

core

team

.

Str

engt

hs

-T

hero

les

and

resp

onsi

bili

ties

ofth

eco

rete

amm

embe

rsar

ecl

earl

yde

fine

d.

The

eval

uati

onco

mm

itte

eva

lues

posi

tive

lyth

esk

ills

and

expe

rtis

eof

the

prop

osed

core

team

mem

bers

,w

ithst

rong

expe

rtis

ein

nati

onal

and

inte

rnat

iona

lO

SH

surv

eys.

Fur

ther

toE

xten

tan

dqu

alit

yof

the

core

team

,in

clud

ing

the

com

posi

tion

inte

rms

ofth

is,

the

reso

urc

eal

loca

tion

toth

epr

opos

edta

sks

isd

eem

edto

beve

rygo

od.

exper

tise

and

skil

ls,

allo

cati

onor

role

san

dre

spon

sibi

liti

esof

all

team

..

..

.-

mem

ber

sas

wel

las

reso

urc

eba

ck-u

p(1

0po

ints

)8

Inlin

ew

ithth

is,

the

appo

intm

ent

ofan

inte

rnal

advi

sory

boar

dto

act

asa

qual

ity

cont

rol

team

isco

nsid

ered

tobe

very

rele

vant

byth

eev

alua

tion

com

mit

tee.

IW

eaknes

ses

The

eval

uati

onco

mm

itte

ew

ould

have

wel

com

edad

diti

onal

deta

ils

onth

epa

rtic

ipat

ion

ofso

me

core

team

mem

bers

,as

wel

las

thei

rre

spec

tive

back

-ups

.

Str

engt

hs

Qua

lity

ofth

epro

pose

dne

twor

kof

fiel

dwor

kre

sear

chce

ntre

s,th

eT

heev

alua

tion

com

mit

tee

appr

ecia

tes

the

deta

iled

info

rmat

ion

abou

tth

epr

ojec

tm

anag

ers

nati

onal

proj

ect

man

ager

san

dth

efi

eldw

ork

supe

rvis

ors

(10

poin

ts)

10an

dfi

eldw

ork

supe

rvis

ors

acro

ssal

lfi

eldw

ork

cent

res

and

wel

com

esth

eir

high

expe

rtis

e.

Fiel

dwor

kis

plan

ned

via

ane

twor

kof

cent

res

acro

ssal

lco

untr

ies

—ex

cept

for

Irel

and

and

Mon

tene

gro.

with

ace

ntra

lin

fras

truc

ture

for

sam

ple

man

em

ent

mo

nit

qp

gan

dst

eeri

ng

Page 34: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

tender

pro

cedure

EU

OSH

AI2

OIT

IOPI

F1SE

IOO

4S—

Tec

hn

ical

eval

uat

ion

:

JIc

,ScIM

yc4

Iltb

hci

Wwk

offi

eldw

ork.

Fur

ther

toth

is,

the

eval

uati

onco

mm

itte

eap

prec

iate

sth

ech

ang

esim

plem

ente

d

inth

ene

twor

kof

loca

lpa

rtne

rs,

bas

edon

the

expe

rien

ceof

ES

EN

ER

-2,

amon

got

her

fact

ors.

The

reis

ago

odnu

mbe

rof

CA

TIw

orks

tati

ons

for

the

prop

osed

task

s.

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee.

CR

ITE

RIO

N2

WO

RK

PAC

KA

GE

A:

QU

EST

ION

NA

IRE

AN

DSA

MPL

ING

:-S

CO

RE

:33

OU

TO

F35

PO

INT

S

Str

engt

hs

The

appr

oach

toth

equ

esti

onna

ire

revi

sion

isde

emed

tobe

very

stro

ngby

the

eval

uati

on

com

mit

tee,

Itis

info

rmed

byth

eT

otal

Sur

vey

Err

or(T

SE

)ap

proa

ch,

aim

ing

tofo

cus

on

accu

racy

and

relia

bilit

yam

ong

the

diff

eren

tqu

ality

dim

ensi

ons.

The

prop

osal

isto

limit

the

num

ber

ofch

ange

sto

the

ques

tion

nair

e,w

hich

isve

ryw

ellj

usti

fied

and

bear

ing

inm

ind

com

para

bili

ty.

Fur

ther

toth

is,

ther

eis

asi

gnif

ican

tpo

int

onco

mpa

rabi

lity

ofth

efi

ndin

gsby

coun

try,

activ

ityse

ctor

and

size

clas

sw

hich

isve

ryre

leva

nt.

The

reis

ago

odap

proa

chto

the

outc

ome

ques

tion

san

dth

eco

llec

tion

ofou

tcom

e

indi

cato

rs.

The

reis

aso

und

prop

osal

tom

odif

yth

esc

reen

ing

part

,fo

llow

ing

atw

o-st

epap

proa

chfo

rse

ctor

and

size

iden

tific

atio

n,w

ithau

tom

ated

codi

ngdu

ring

the

inte

rvie

w.

Qua

liLy

ofth

epr

opos

alto

the

ques

tion

nair

ere

visi

on,

incl

udin

gth

eT

here

isa

very

rele

vant

appr

oach

toga

ther

ing

info

rmat

ion

from

non-

resp

onde

nts,

sugg

este

dap

proa

chto

new

topi

cs,

the

stra

tegy

toga

ther

info

rmat

ion

ackn

owle

dgin

gth

ese

nsit

ivit

yof

the

issu

ean

dth

ene

edfo

rin

tens

ive

pre-

test

ing.

The

aim

is

from

non-r

esponden

tsan

dth

epr

e-te

st(1

0po

ints

)8

toas

kth

ree

ques

tion

s,on

eon

size

and

two

onO

SH

(sug

gest

edqu

esti

ons

incl

uded

inth

eof

fer)

plus

the

reas

ons

for

refu

sal,

The

reis

ave

rygo

odpo

int

onth

eus

eof

the

find

ings

and

the

impa

ctth

eyw

illha

veon

wei

ghti

ng.

The

pre-

test

envi

sage

s36

face

tofa

cein

terv

iew

sin

thre

eco

untr

ies

(Ger

man

y,L

atvi

aan

dth

eN

ethe

rlan

ds)

and

isto

cove

rth

ew

hole

ques

tion

nair

e(n

eed

ofco

ntex

t)bu

tfo

cus

only

onth

ene

w!m

odif

ied

item

s.T

here

isa

very

deta

iled

desc

ript

ion

ofth

epr

oces

s,w

hich

will

beca

rrie

dou

tby

rese

arch

ers

invo

lved

inth

equ

esti

onna

ire

deve

lopm

ent,

with

aha

ltha

lfw

ayth

roug

hto

impl

emen

tch

ang

esif

need

ed.

The

eval

uati

onco

mm

itte

eva

lues

the

prop

osal

posi

tivel

y.

Wea

knes

ses

Add

ition

alde

tail

sab

out

how

the

cogn

itiv

epr

e-te

stw

illfe

edth

equ

esti

onna

ire

wou

ldha

vebe

enw

elco

med

.

Page 35: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Open

ten

der

pro

ced

ure

EU

OS

HN

2OI7

IOPI

FISE

IOO

4B—

Tec

hnic

alev

aluat

ion:

JI,

5ohi

yaJ

HrJ

ihJ

ttWoâ

appr

oach

asin

ES

EN

ER

-2bu

tai

min

gto

inte

rvie

wth

ree

unit

sin

stea

dof

two.

The

reis

anes

tim

atio

nof

the

sam

ple

need

ed,

look

ing

atth

eco

oper

atio

nra

tefo

rth

edi

ffer

ent

stra

ta,

with

som

epr

opos

als

mad

eon

cell

targ

ets.

As

long

asth

esa

mpl

efr

ame

info

rmat

ion

allo

ws,

the

tend

erer

prop

oses

anev

enfi

ner

diff

eren

tiat

ion

ofth

esa

mpl

ing

mat

rix

for

the

nati

onal

sam

plin

gpl

ans

(85

NA

CE

Rev

2.2

x4

size

clas

ses=

340

stra

ta).

Thi

sm

atri

xis

tobe

used

for

sam

ple

orde

rsfr

omre

gist

ers

but

not

tost

eer

fiel

dwor

k.F

orth

is,

an8

x4

mat

rix

ispr

opos

ed,

sim

ilar

toE

SE

NE

R-2

,w

here

asfo

rre

achi

ngta

rget

s,a

3x

4cc

lim

atri

xis

prop

osed

.

Som

eve

ryso

und

poin

tsar

em

ade

onth

est

ratu

mju

mpe

rsan

dth

eus

eof

the

stop

ping

rule

,w

hich

isex

pect

edto

beus

edle

ssfr

eque

ntly

than

inE

SE

NE

R-2

with

the

prop

osed

chan

ges

.

The

reis

apo

int

mad

eon

the

com

para

bili

tyw

ithE

SE

NE

R-2

argu

ing

that

sinc

eth

esa

me

pro

cess

esw

illbe

appl

ied,

resu

ltsh

ould

beco

mpa

rabl

e.T

head

iust

men

tpr

opos

edsh

ould

incr

ease

the

prec

isio

nof

ES

EN

ER

-3(n

arro

wer

conf

iden

cein

terv

als)

.-

-.

-

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee.

CR

ITE

RIO

N3

WO

RK

PAC

KA

GE

B:

TR

AIN

ING

,A

ND

DE

LIV

ER

Y:

-S

CO

RE

:34

OU

TO

F35

PO

INT

S

Str

engt

hs

The

eval

uati

onco

mm

itte

eva

lues

the

effo

rts

mad

eto

chan

gena

tion

alpa

rtne

rsth

atth

eco

ntra

ctor

iden

tifie

dto

have

suff

ered

from

perf

orm

ance

issu

esin

the

pasL

The

expe

rien

ceof

the

inte

rvie

wer

sis

very

good

and

rele

vant

for

the

surv

eyan

dth

etr

aini

ngpr

opos

edis

soun

d.T

wo

sem

inar

sar

een

visa

ged

for

the

nati

onal

proj

ect

man

ager

san

dfi

eldw

ork

supe

rvis

ors

(at

the

laun

chof

the

proj

ect

and

fiel

dwor

k,re

spec

tive

ly)

inw

itha

real

isti

can

dre

leva

ntdu

rati

on.

Qua

lity

ofth

est

rate

gy

for

fiel

dwor

kpr

epar

atio

n,pa

rtic

ular

lyth

eB

ased

onth

ese

sem

inar

s,w

ritte

nin

stru

ctio

nsw

illbe

prep

ared

for

the

trai

ning

ofap

pro

ach

totr

aini

ngan

dth

eco

ntac

tst

rate

gy

for

redu

cing

non-

resp

onse

,in

terv

iew

ers

atth

eco

untr

yle

vel,

inth

eir

nati

onal

lang

uage

.in

clud

ing

the

onli

nem

odul

e(1

0po

ints

)9

Asi

ngle

plat

form

ispr

opos

edfo

rC

ATI

and

CA

WI

whi

chfa

cili

tate

sst

anda

rdis

edco

ding

and

the

inte

grat

ion

ofco

nsis

tenc

yan

dpl

ausi

bilit

ych

ecks

inth

eC

All

ques

tion

nair

e.T

here

isa

thor

ough

desc

ript

ion

ofth

esc

ript

ing

ofth

equ

esti

onna

ire

(inc

ludi

ngth

etr

ansl

ated

vers

ions

)us

ing

aso

ftw

are

appl

icat

ion

for

NIP

O.

The

reis

apo

int

mad

eon

the

cont

act

stra

tegy

and

asu

ppor

tfo

rth

esa

me

appr

oach

inge

nera

las

inE

SE

NE

R-2

,w

hich

isw

ell

just

ifie

d.

Wea

knes

ses

Whi

leth

eus

eofa

shig

çpla

tforr

nfo

rC

ATI

and

CA

WI

isd

eem

edto

bere

leva

nt,

the

Page 36: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Ope

nte

nd

erp

roce

du

reE

UO

SHA

I2O

17I

OPI

FISE

IOO

48—

Tec

hnic

alev

alua

tion

:Lw

SoLe

!jcn

dH

hh

Jci

Wo.

k

Str

engt

hs

The

over

all

appr

oach

toth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

isve

ryre

leva

nt.

Con

cern

ing

the

nati

onal

ad-h

ocm

odul

es,

ther

eis

are

ason

able

poin

tm

ade

plac

ing

them

at

the

end

ofth

equ

esti

onna

ire

but

bein

gaw

are

ofth

eim

pact

onth

ege

nera

lqu

esti

onna

ire

flow

.

The

prop

osal

for

the

tran

slat

ion

follo

ws

the

TR

APD

asre

ques

ted

inth

ete

chni

cal

spec

ific

atio

ns,

whi

chis

wel

lde

scri

bed,

incl

udin

gth

etr

aini

ngfo

rtr

ansl

ator

san

dth

equ

alit

y

cont

rol

plan

for

the

tran

slat

ion.

Itw

illin

clud

ea

tran

slat

abil

ity

asse

ssm

ent

afte

rth

eco

gnit

ive

test

ing

and

befo

rela

unch

ing

the

tran

slat

ions

for

the

pilo

tte

st.

The

reis

aso

und

appr

oach

toth

epi

lot

test

,in

terv

iew

ing

upto

40es

tabl

ishm

ents

inth

ose

coun

trie

sw

ithm

ore

than

one

lang

uage

and

aim

ing

tote

stth

eun

ders

tand

ing

ofth

esu

rvey

both

byth

ere

spon

dent

san

dth

ein

terv

iew

ers

—,

the

tech

nica

lfu

ncti

onin

g,th

edu

rati

on,

the

cont

act

ph

ase

and

the

outc

ome

ques

tion

sas

wel

las

the

ques

tion

sto

non-

resp

onde

nts.

Itw

asap

prec

iate

dby

the

eval

uati

onco

mm

itte

eth

atth

epi

lot

test

isco

nsid

ered

tobe

anin

stru

men

tfo

rth

em

ain

fiel

dwor

kin

itse

lfai

min

gto

have

min

imal

chan

ges

afte

rwar

ds.

The

orga

nisa

tion

ofth

epi

lot

ispl

anne

dto

besi

mil

arto

that

ofth

em

ain

fiel

dwor

kbu

tw

ithso

me

diff

eren

ces,

part

icul

arly

the

freq

uenc

yof

the

list

enin

g-in

s.

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee.

Str

engt

hs

The

sam

plin

gst

rate

gyis

wel

lpr

esen

ted,

follo

win

gth

esa

me

gene

ral

appr

oach

toth

atin

ES

EN

ER

-2bu

tw

ithso

me

impr

ovem

ents

.

The

reis

ago

odin

lrod

ucti

onto

the

sam

plin

gch

alle

nges

ofE

SE

NE

R,

focu

sing

onth

eun

itof

enqu

iry

and

the

avai

labi

lity

ofna

tion

alst

atis

tics

,as

wel

las

are

leva

ntde

scri

ptio

nof

the

bu

sin

ess

regi

ster

sto

beus

ed.

The

offe

rpr

opos

esto

use

anan

tici

pato

rysa

mpl

eal

loca

tion

Qua

lity

ofth

esa

mpl

ing

stra

tegy

,in

clud

ing

the

prop

osed

bu

sin

ess

desi

gnba

sed

onth

eir

expe

rien

cew

ithE

SE

NE

R-2

.

The

choi

ceof

the

best

avai

labl

esa

mpl

ing

fram

esis

wel

lex

plai

ned.

The

tend

erer

prop

oses

adi

spro

port

iona

test

rati

fied

sam

plin

gde

sign

,as

inE

SE

NE

R-2

,bu

tsh

iftin

gth

ew

eigh

tm

ore

toth

ees

tabl

ishm

ents

(vs.

the

empl

oyee

s).

Inor

der

toin

form

the

fina

lde

cisi

on—

tobe

agre

edw

ithE

U-O

SHA

—,

the

offe

rsu

gges

tsid

entif

ying

five

cent

ral

mea

sure

son

whi

chto

carr

you

tth

ean

aly

ses

and

use

the

resu

lts

asa

basi

sfo

rth

ede

cisi

on,

whi

chis

very

rele

vant

.

The

reis

ave

ryso

und

revi

ewof

the

scre

enin

gp

roce

ssfo

rm

ulti-

site

com

pani

es,

runn

ing

thro

ugh

the

app

roac

hes

inE

SE

NE

R-1

and

ES

EN

ER

-2an

dju

stif

yinq

tofo

llow

the

sam

e

Qua

lity

ofth

ead

apta

tion

ofth

ena

tion

alqu

esti

onna

ires

—w

hile

ensu

ring

inte

rnat

iona

lco

mpa

rabi

lity

-,in

clud

ing

the

nati

onal

ad-h

ocm

odul

es,

asw

elt

asth

epr

opos

edap

proa

chto

tran

slat

ion

and

the

pilo

tte

st(1

0po

ints

)10

Page 37: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Au

i,y

Jta

rSaf

etycd

H.@

hhJ

oiW

a4

Ope

nte

nd

erpro

cedure

EUO

SKA

I2O

I7IO

P1FI

SEIO

O48

—T

echn

ical

eval

uati

on:

eval

uati

onco

mm

itte

ew

ould

have

appr

ecia

ted

mor

ede

tail

son

the

defi

nitio

nof

CA

WI.

Str

engt

hs

The

reis

ave

ryde

tail

edap

proa

chto

mon

itor

ing

ofin

terv

iew

ers

and

fiel

dwor

k,w

hich

isIc

onsi

dere

dto

beve

rygo

odby

the

eval

uati

onco

mm

itte

e.A

tth

ela

unch

ofth

esu

rvey

mon

itor

ing

isex

pect

edto

bem

ore

inte

nse

than

Lhe

10%

offe

red

thro

ugho

utth

ere

stof

fiel

dwor

k,w

itha

high

ernu

mbe

rof

supe

rvis

ors

invo

lved

.

The

prop

osed

coun

try

repo

rts

are

very

deta

iled

and

accu

rate

.

Qua

lity

ofth

epro

pose

dfi

eldw

ork

exec

utio

n,de

scri

bing

the

over

all

appro

ach

tog

uar

ante

ehi

gh-q

uali

tyin

terv

iew

san

dm

inim

ise

non-

resp

onse

bias

incl

udin

gth

em

onit

orin

g,co

ntro

lan

dre

port

ing

ofth

efi

eldw

ork

pro

cess

(15

poin

ts)

15

The

reis

ave

ryso

und

appr

oach

tow

ards

redu

cing

non-

resp

onse

.Fo

llow

ing

ave

ryre

leva

ntth

eore

tica

tre

view

onw

here

tofo

cus

the

effo

rts,

the

offe

rpr

opos

esto

conc

entr

ate

oncl

ose

cent

ral

mon

itor

ing,

alim

ited

initi

alsa

mpl

ean

dst

agg

ered

addi

tion

alre

leas

es,

plus

ase

ries

ofad

diti

onal

mea

sure

sw

hich

are

all

very

wel

lex

plai

ned

and

that

the

eval

uati

onco

mm

itte

eva

lues

posi

tivel

y.

The

reis

ago

odap

proa

chto

the

offe

rof

the

CA

WI

opti

on,

only

toC

ATI

refu

sers

.S

peci

fic

trai

ning

will

bepr

ovid

edto

inte

rvie

wer

sin

orde

rto

avoi

dus

ing

CA

WI

asan

easy

esca

pe.

Fur

ther

toth

is,

upto

four

rem

inde

rsw

illbe

sent

,th

ela

stov

erth

eph

one,

whe

reC

ATI

will

beof

fere

dag

ain.

The

eval

uati

onco

mm

itte

eva

lues

posi

tive

lyth

eap

proa

chto

the

onli

nem

odul

e.

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee.

Str

engt

hs

Qua

lity

ofth

eap

pro

ach

toda

tapr

oces

sing

,de

live

ryan

dst

orag

e,in

clud

ing

wei

ghti

ngan

dth

eim

pact

onco

mpa

rabi

lity

with

ES

EN

ER

-2,

asw

ell

asth

ere

port

ing

(10

poin

ts)

10

The

eval

uati

onco

mm

itte

eva

lues

posi

tive

lyth

eap

proa

chto

data

proc

essi

ng,

deli

very

and

stor

age,

whi

chis

expl

aine

dve

rycl

earl

y.T

heda

tach

ecki

ngsy

ntax

will

bede

velo

ped

inS

PS

Sfo

rth

epi

lot

test

and

will

bere

vise

dfo

rth

em

ain

fiel

dwor

k.T

hech

ecki

ngan

dva

lida

tion

will

beca

rrie

dou

tat

vari

able

and

resp

onde

ntle

vel,

with

som

esh

ort

exam

ples

and

sugg

esti

ons

prov

ided

,w

hich

are

rele

vant

.

Inte

rim

data

chec

ksar

een

visa

ged

afte

r5%

ofin

terv

iew

sha

veb

een

carr

ied

out

toen

sure

data

are

bein

gco

llec

ted

asin

tend

ed,

wee

kly

chec

kth

erea

fter

.A

dditi

onal

ly,

cros

s-ta

bula

tion

sw

ithin

teri

mda

taar

eto

bepe

rfor

med

afte

rco

mpl

etio

nof

10%

and

50%

ofin

terv

iew

sto

chec

kfo

rre

sponse

rate

san

dco

nten

tre

late

dda

ta.

Thi

sis

cons

ider

edto

beve

ryin

tere

stin

gby

the

eval

uati

onco

mm

itte

e.

The

reis

aso

und

desc

ript

ion

ofth

ero

utin

ean

dpl

ausi

bilit

ych

ecks

alte

rfi

eldw

ork

com

plet

ion,

with

the

mea

sure

sre

gard

ed,

whi

chin

som

eca

ses

incl

ude

re-c

onta

ctin

ges

tabl

ishm

ents

conc

erne

d.

One

open

-end

edqu

estio

nis

cons

ider

ed,

onth

eac

tivity

sect

orof

the

esta

blls

hmen

t,an

d

Page 38: EVALUATION REPORT - Information, statistics, legislation ... · EVALUATION REPORT Title of contract ESENER 3 Reference: EUOSHN2O1 7IOPIFISEIOO48 Contracting authority: EUOSHA Type

Loaç

tcn

*Otf

lrf

JIra

Wtt

ydH

r&ib

2cl

wrt

k

Ope

nte

nder

pro

cedu

reE

UO

SHN

2OI7

IOPI

FISE

IOO

48—

Tec

hnic

alev

alua

tion

:

Sig

ned

by:

only

tobe

used

inca

seth

ein

form

atio

nfr

omth

eb

usi

nes

sre

gist

eris

wro

ng.

The

appr

oach

tow

eigh

ting

isve

ryso

und,

deta

iled

and

accu

rate

,w

ithan

inte

rest

ing

disc

ussi

onof

the

use

ofth

ege

nera

tise

dre

gres

sion

esti

mat

ion

(GR

EG

)ap

proa

ch,

whi

chth

eypr

opos

e,vs

.th

eit

erat

ive

prop

orti

onal

fitti

ngap

proa

ch.

Wea

knes

ses

No

evid

ent

wea

knes

ses

wer

eid

entif

ied

byth

eev

alua

tion

com

mit

tee. T

OT

AL

O4O

UT

OF

100

RU

PRU

NE

TP

RU


Recommended