Open data on the job market is important for policy and decision-making purposes to ensure Europe’s competitiveness and that the skills of the workforce meet demand. When working on data-driven solutions, the European Commission’s big data test infrastructure (BDTI) helps public administrations to experiment with real data before starting system development.
Extracting data from online job advertisements requires converting the digital footprint left by companies in the online labour market into relevant statistical variables. The two types of data collected are:
- Structured data: Job location and publication date (quickly collected by data scrapers).
- Natural language found in free text: Here, Artificial Intelligence (AI), combining Natural Language Processing (NLP) and machine learning algorithms, are used to clean the free text and extract data on occupation, education, skills, salary and so on.
Skills-OVATE (Skills Online Vacancy Analysis Tool for Europe) collects online job advertisement data from all EU Member States and more than 94,000 webpages, including public employment services, private job boards, recruitment agencies, company websites, and online newspapers. Skills-OVATE is expected to be fully operational later in 2020, providing a pan-European system for stakeholders on all levels: EU, national and regional.
The next steps include data quality monitoring procedures, defining a governance model, and expanding the system for large-scale production of statistics in labour market intelligence.
Looking for more open data related news? Visit the EDP news archive and follow us on Twitter, Facebook or LinkedIn. Interested in learning more about COVID-19 and open data? Visit our EDP for COVID-19 page.