ESSnet DataWareHousing

Stocktaking

Pieter Vlag, Viviana di Giorgi, Sonia Queresma

ESSnet DataWareHousing - questionnaire 2

Questionnaire

Starting Point

• Data model perspective• Proces model perspective

• Meta data very important• Position of Business Register in system ??

• Needs of other NSIs

ESSnet DataWareHousing: questionnaire 3

Data model perspective

MAIN MOTIVATION: FLEXIBLE + CONSISTENT OUTPUT

ESSnet DataWareHousing Questionnaire 4

Process model perspective

Production processes

Input 1

Input 2

Data warehouse

The ‘process model’ perspective

Output 1

Output 2

Output 3

Known production processes, exploiting synergies or experience

processes

Knowninputs

Knownoutputs

Metadataattached to production process

MAIN MOTIVATION: EFFICIENCY IN PROCESS

ESSnet DataWareHousing Questionnaire 5

Questionaire

21.3.2011:- 21 responses (one per country considered); non-respons homogeneously distributed over Europe

- Many countries mention that “DWH” work is in progress or their system is partially implemented

- Inconsistencies in the answers, possibly related due to current vs. future situationdifferent perception of used terminology

- Most countries sent in a ‘common view’, but….. In case of >1 answer per country: answers on questions might differ

ESSnet DataWareHousing Questionnaire 6

Questionaire

- Good news

ESSnet DataWareHousing Questionnaire 7

Motivations for DWH

CONCLUSION:- data-integration,datalinking and efficiency are main motivations

ESSnet DataWareHousing Questionnaire 8

Questionnaire on perspectives

This suggests:- “Data model: ~ 9 countries- “ Process model” ~ 6 countries- “ mix of approaches” ~ 6 countries

ESSnet DataWareHousing Questionnaire 9

Questionnaire: answers on perspectives II

Two remarks:1.“ Process model” : 6 countries > one-to-one correspondences : 4 countries

2.if main characteristic of “data model perspectives” = flexibility in output, then most countries (17) have some “data model perspectives”in their system

ESSnet DataWareHousing Questionnaire 10

Questionnaire: answers on perspectives II

GENERAL IMPRESSION: preferred situation= data model perspectiveIn practice most countries start with a process data approach (and expand it at a later stage)

-> “data model” / “process model” perspective description do NOT discriminate between the practices

ESSnet DataWareHousing Questionnaire 11

Questionnaire: terminology

“data model” & “process model” perspectives were introduced two express that two main motivations may lead to starting a DWH- process efficiency- more flexible + consistent output

However, -questionnaire does not reveal a clear dinstinction between these motivations- models are not exclusive- in practice a mix is used (maybe related to “live phase” of the DWH

Proposal: use “data model perspective as ideal situation

?

ESSnet DataWareHousing Questionnaire 12

Meta data

Conclusion:Meta data are considered as important, but are not always integrated in the systems (yet ?)

Question: Which metadata to be used in DWH ?

ESSnet DataWareHousing Questionnaire 13

Conceptual differences II

Apparently differences in approaches

ESSnet DataWareHousing Questionnaire 14

Conceptual differences: active versus passive

surveys

Admin data

census

Cleaned input

Processed data I

Processed data II

Processed data III

Output data

Aggregate data

micro data

Time series

Active DWH: processing and storage NOT separated

Active DWH: proccessing steps between storage

Business register – not integral part

ESSnet DataWareHousing Questionnaire 15

Conceptual differences III

DWH model/system

Active(10)

Passive(11)

Business Reg. integral part

(5)

Business Reg. not integral part

(5)

Business Reg. integral part

(5)

Business Reg. not integral part

(6)

No relationship between (considered) active/passive datawarehouse and position of business register(in answers on questionnaire)

ESSnet DataWareHousing Questionnaire 16

Conceptual differences

CONCLUSION:- implicitly, doubts about business case ??????- output side: different statistical requirements are considered as complex

ESSnet DataWareHousing Questionnaire 17

Conclusions

Flexibility in output; data linking and efficiency in process are the main motivations for DWH ?

“data model” and “process model” descriptions were helpful for the questionnaire, but should be used differenly in future.

Meta data are – indeed – considered as important, and an interesting topic for this ESSnet to work out

Two opinions about DWH:- active vs. passive- position of business registerboth are not linked to each other: but interesting topics for ESSnet to work out

Different statistical requirements (=output ?) are considered as complicated and a possibly interesting topic for this ESSnet to work out

Data confidentiality hardly mentioned

ESSnet DataWareHousing Questionnaire 18

Continuation (advice)

Visit countries to obtain the details .

Focuss on:- Meta data- active vs. passive- position of business register- different statistical requirements

From SGA-2011provide requirements about these topics.