Big data analyses of the labourmarket
Emilio Colombo
Università Cattolica del Sacro Cuore
CRISP
ESCO Conference, 09/10/20171
What lays ahead?
• The world economy has been invested by a technologicalrevolution which is reshaping the global economic system
• The primary factor in change is the shifting “from aneconomy centred on producing physical goods to onecentred on innovation and knowledge (...) For the first timein history, the factor that is scarce is not physical capital butcreativity.“ (The new geography of jobs, E. Moretti, 2012)
2
The challenges of the future
• Technology, population ageing, migration flows are affectingthe labour market with a strength and a speed which areunprecedented
• Frey and Osborne (2013) 47% of Jobs will disappear in thenext 25 years.
• World Economic Forum (2016): 65% of children enteringprimary school today will ultimately end up working incompletely new job types that don’t yet exist.
3
The challenges of the future
•What skills will be required by the labour market in the future?
•Will the labour force be up to the standards of the demand side?
•What sort of mismatches will arise and where?
•Skill mismatch has a lot to do with the difficulty in correctly define and classify skills (mainly soft skills).
4
LABOUR MARKET
PEOPLE
What are the most required skills
for my job? Where is the demand for my occupation (company,
sector, territory)?
P.A.
Which
training/educational
policies are needed?
COMPANIES
What are the evolution
trends in skills for the
occupations of interest and how can I improve the
training of my employees?
EMPLOYMENT
AGENCIES
Which profiles the
companies are
looking for in my area / sector...?
5
Work mobility
The change, with greater speed, of skills required within the same occupation
The polarization of occupations
The emerging of new occupations Global markets vs local markets
6
Features of today’s labour market
New policies, new tools
• We need new policies at local, national and internationallevel
• But before we need to assess and know better the size andmagnitude of these phenomena and their implications forthe labour market
• This calls for new data and new tools.
7
New tools
• We need forecasting tools to identify the most relevanttrends. This is what Cedefop does since 2007• But forecasting tools are necessarily imprecise about
the features and skill requirements of the jobs of thefuture
• We need tools to investigate firms’ skill needs. This is whatCedefop has done with the European Skills and Jobs survey.• However surveys are rigid and lengthy tools
• We need tools able to detect the change in the labourmarket as it occurs. We need real time tools. 8
Problems with skill surveys
1. They are costly, considering direct (implementation) and indirect (opportunity cost) costs.
2. Their implementation is not easy, thus they cannot have a high frequency.
3. They have a top-down approach, i.e. soft skills and occupation-specific skills are generally pre-defined.
4. What you have is what you designed. Not more sometimes less.
9
How web vacancies may help
1. Less costly to be implemented. High initial cost but low marginal cost.
2. No implementation lag. Almost real time data.
3. Bottom-up approach, richer classification especially useful for some soft skills and particularly for occupation-specific skills.
4. Information is always there. Unless there are storage problems you can also go back and retrieve what you missed.
10
Big Data: from data to knowledge
The value added of big data is the ability to answer the questions of different stakeholders with more and timely data!
http://www.wollybi.com/
11
How does it work?
Collect Web Job VacanciesScrape and collect data from several sources, previously ranked to guarantee high quality (egpresence of publication/update timestamp, territorial granularity, etc)
Data transformation and cleansing
Fills the data into our lossless data model performing data cleaning activities (eg, duplicate detection)
ClassificationUse supervised machine learning approaches to classify Web Job Vacancies according to ISCO/ESCO classification system
Skills extractionUse top AI algorithms (eg, semantic similarity measures, IE techniques, NLP) to extract, recognise and link skills to occupation profiles
Data VisualisationKnowledge representation of data evolution over time by means of data visualisation tools, having in mind the stakeholder features
12
Some useful insights
• Insights: big data let the data speak and allow us to grasp what ishappening now, and understand how these changes are manifestingthemselves.
• In the next slides some insights will be presented to give an idea ofthe informative potential:• Insight 1: how widespread are ICT skills in occupations?• Insight 2: what are the new emerging occupations?• Insight 3: how the demand for skills has changed over recent years?
• These question are difficult to answer particularly with a top downapproach
• So let’s ask the data
13
The Skill Digital Rate (SDR)
measures the incidence of
digital skills in a given
occupation.
For ICT occupations, the
average SDR is 68%, higher
values are recorded in new
occupations.
Fonte WollyBI – Italian Labour Market Digital Monitor
The Skill Digital Rate in ICT occupations
Skill Digital Rate percentage 14
Skill Digital Rate pct
For non-ICT occupations, SDRs is
growing for administrative,
management activities and
market development activities
(e.g. human resources,
accounting and marketing).
Source WollyBI – Italian Labour Market Digital Monitor
The Skill Digital Rate in non-ICT occupations
15
Freight handlers
Human resource managers
Accounting associate professionals
Management and organisation analysts
Advertising and marketing professionals
Digital skills in non-ICT occupations
Prevalence of Digital Basic Skills and of computer
intermediation
LOW SKILL DIGITAL RATE
Uso PC, Word Proc. , Social Network, Wordpress
Prevalence of digital skills Applications and
Techniques
HIGH SKILL DIGITAL RATE
Database
ERP CAD Java HTML5 CMS
PHP
Uso PC, Word Proc. , Social Network, Wordpress
The composition of digital skills
in non-ICT professions changes
according to the Skill Digital Rate:
the lower the SDR occupations
the more basic the digital skills
(i.e. Basic and Information
Intermediation). The higher the
SDR, the more technical the skills
(i.e. application and techniques).
Source WollyBI – Italian Labour Market Digital Monitor 16
The «new» emerging occupations
Data Scientist Cloud Computing Cyber Security Expert
Business Intelligence
Analyst Big Data Analyst Social Media Marketing
Trend changes:February-April 2017 compared to the
year 2013: + 278%
More than 7 thousands vacancies
from February 2013 to April 2017
172013 2014 2015 2016 2017
Skills - Data Scientist Skills - Cloud Computing
Relational Abilities Team Working
Java Machine Learning Reporting
Data Visualization Organizational Abilities
DataWarehouse DataBase
Statistical Models Phyton
Professionalism Sql Knowledge ERP Applications
Data Mining Data Analysis Office Package
SAS Programming Management Knowledge
Business Intelligence & Analytics Pc Use
Sql Pc Use Linux Telecommunication Knowledge
Team Working Management
Knowledge Professionalism
Java Html
Relational Abilities Vmwar DataBase
Windows Server Scripting
Skills - Cyber Security ExpertSkills - Business Intelligence
Analyst
Relational Abilities Team Working
Responsibility Sense Security Certification
Professionalism Linux Organizational Abilities
Telecommunication Knowledge
Unix Security Systems Java
Firewall Problem Solving
Malware Analysis Pc Use
Team Working SAS Programming
Office Package SqlProfessionalism Data Modeling
Java Dbms
Relational Abilities DataBaseData Analysis ETL
Business Object QlikView
Business Intelligence
Reporting
Skills - Big Data Analyst Skills - Social Media Marketing
Reporting SAS & R Jboss
Relational Abilities DataBase Data
Analysis NoSQL Java DataWarehouseBusiness Intelligence SQL Server
Team Working Cloudera ClouderaHadoop Professionalism
Python Sql
Team Working Indesign
Photoshop SEOOffice Package Professionalism
Relational Abilities Web Edit
Graphic Programs Html5Management/OrganizationTeam Working Marketing Knowledge
Google Adwords
Social Network Wordpress
Changes skill demand 2013 vs. 2017
Soft skills are growing in importance due to the relevance of collaboration and relational abilities even in highly ICT-specialized jobs.
The occupation description by ESCO: Database designers and administrators design, develop, control, maintain and support the optimal performance and
security of databases.21
2521 - Database Designers and Administrators
• DATA WHAREHOUSE: -11%.
• ORACLE: -7 %
• LINUX: -4 %
• MANAGEMENT & ORGANIZATION:
+21%.
• MANAGE THE RELATIONS WITH
CLIENTS: +18 %
• ATTITUDES & VALUES: +12 %
Changes skill demand 2013 vs. 2017
22
2431 – Advertising and marketing professionals
• ADOBE PHOTOSHOP: +3 %
• GRAPHIC EDITING SOFTWARES: +3 %
• ATTITUDES & VALUES: -7 %
• MS OFFICE: -3 %
• THINK CREATIVELY: -2 %
Conversely, the need for hard skills is growing as they convert traditional jobs towards digital.
The occupation description by ESCO: Advertising and marketing professionals develop and coordinate advertising strategies and campaigns, determine the
market for new goods and services, and identify and develop market opportunities for new and existing goods and services.
Big data allows big KNOWLEDGE
Pros of Web Data:
1. Data-driven approach that allows:
• The analysis of occupations and skills dynamics
• The identification of new trends and new emerging occupations
2. No implementation lag. Near real-time analyses.
3. Less costly to be implemented vs surveys. High initial cost but low marginal cost (maintenance and evolution).
The main featureof a Big Data approach is the real-time collectionof all relevant data and “let the data speak";
23