Skills Pulse – Technology Industry Skills forecasts based on AI-mined public data

From Collecting Data to Skills Forecasts

Over half of all jobs will be transformed by technology over the next 10 years, according to the OECD. This will require one billion people to upskill by 2030. Ultimately, everyone will need upskilling in some form. Continuous learning will take an increasingly important role for everyone. But which skills will be in demand?

Let’s think about forecasts for a second. Organized acquisition of observational weather data started in the middle of the 19th century.  Weather forecasts came in the 40s, nearly a hundred years later. In the labor market, the need for the international standard classification of occupations (ISCO) was discussed as early as 1921 at the International Conference of Labour Statisticians. Work to develop ISCO was initiated in 1947, and the first ISCO was published in 1958. Since then, many ontologies have been built to enable making the labor market data more categorized for statistical use.

Ontologies made by humans are static, where the occupations and skills demand change constantly due to disruptions in the labor markets. To predict the changes in skills demand, we have to apply Natual language processing (NLP) and Self-organized maps (SOM) to build a dynamic and self-learning system. These technologies are no magic but can help to futureproof us. More about Headai tech can be read on the Science page.

“Continuous learning plays a key role in our economic competitiveness. More than half of all SMEs feel that the availability of skilled labor is limiting growth” – Technology Industries of Finland’s report “There is Work – Where are the Experts” (In Finnish)

Skills Pulse

Skills Pulse gives Technology Industries of Finland and its member companies analyzed information on skills demand and trends in the different industry sub-sectors, based on public data sources. It is a Headai-made web service that operates as its own page but it can also be embedded easily into any existing system. It allows studying the results with different parameters, like:

• Sub-sectors: Electronics & electrotechnical industry, IT, Metals Industry, Consulting Engineering, Mechanical Engineering
• Regional level: Choose, which country’s labor market data to use
• Time frame: Build visualizations with a time filter: e.g. one, two, or three years back with 6 months prediction for future trends
• Limiting the data with a text search for digging into the desired concepts and their clusters

The service is developed by Headai together with Technology Industries of Finland and the member companies. The quality of the first results of the AI calculations will be validated with the help of the members. This procedure offers transparency and enables finding any possible misinterpretations from the AI calculations.

“The cooperation with Headai opens up interesting opportunities on data. It enables a broad analysis on skills demand, but also to go deeper to the level of individual concepts.”
–Touko Apajalahti, Advisor, Higher Education Policy, Growth and Skills, Technology Industries of Finland

What Data Sources Are in Action?

There are numerous textual data sources available, but how to choose the best ones? We always need to think, which data would give answers to the questions at hand. Which data would describe the past, and which one gazes into the future. Is the data easily available through API, or does it need parsing, cleaning with NLP operations,  translation, or other actions? In the Skills Pulse, there have been so far four different data sets utilized:

• Job ads  categorized into five sub-sectors of Finnish Technology Industry. Gives the information about skills demand in the labor market
• Theseus – The common database of the Finnish UAS (Universities of Applied Sciences). Can help to predict future R&D trends.
• Crunchbase – data about investments and startups.
• Open access research articles – helps to reveal the possible technologies that are rising next.

From Data to Visualization

The backend operations of Headai’s product and the coordination of the delivery have been done in Finland, but the development of the front end and the visualization components has mostly been done in Colombia, by Headai’s Junior Data Scientist, Andrés Felipe Zapata Palacio. He describes that visualization plays a big role in the Skills Pulse:

“Transforming predictions and trendings into an insightful visualization was one of the most important priorities during the development process.”
– Andrés Felipe Zapata Palacio, Junior Data Scientist at Headai

In the following images, examples of the visualizations are being presented.

Hexagon map of skills demand. The deeper the color, the more important the skills is.  The blue strokes separate the words that don’t have a strong connection in the studied data set.

The user can compare the results with different ontologies. This could be described as taking a different point of view or looking at the data through different classes.

Machinery & metal industry: TOP10 skills from the previous 3 years.

Machinery & metal industry: Communication skills trendline and forecast.

Technology Industries of Finland’s expert Antti Poikola, Headai’s founder Harri Ketamo, Head of Business Development Anu Passi-Rauste, and Technology Industries of Finland’s Director Leena Pöntynen together design the data flows and AI operations of the Skills Pulse solution using Headai’s Data to Action cards.

Mindmap is a Challenge for the Algorithm, but it’s a Challenge for Anyone

It may seem easy to be able to draw a mindmap, a map of concepts, based on e.g. a text document of few pages. However, it requires considerable cognitive abilities. The first step is to identify the key concepts in the text and the relationships between them. After this, you need to be aware of the missing concepts and be able to connect them to the previous ones.

“Creating a concept map is such a cognitively multi-threaded operation that no two experts can hardly end up with a similar concept map, no matter how familiar the topic is”, explains Harri Ketamo, Founder & Chairman of Headai.

What is the Most Important Skill in 2030?

It is critical thinking. We need to develop our abilities to interpret data. To discover and understand the background process behind the results. How to study the results in relation to the process of machine-based data analysis. To keep in mind the importance of data quality. There is no magic to it. Big data can be interpreted with using a variety of ontologies. Some of them are human-made, others can be dynamic, made by a machine using an algorithm set. It’s vital to understand what choices were made. And finally, who made the algorithm.