Mineração de texto aplicada à análise de vagas de desenvolvimento de software

Abstract

Technological advancement has significantly increased the volume of data processed, transformed the job market, and made skills such as programming and data analysis essential. The COVID-19 pandemic accelerated the demand for remote work and exposed discrepancies between candidates’ skills and employers’ requirements. This study analyzes job openings for software developers through text mining, aiming to identify trends and propose strategies to align expectations between employers and candidates. Following the CRISP-DM methodology, the study went through the stages of business understanding, data collection, data preparation, modeling, evaluation, and deployment. The data was obtained through APIs and web scraping with Python, and the analysis was carried out on the Google Colab platform using natural language processing (NLP) techniques, such as tokenization, stopword removal, and lemmatization. In the modeling phase, the bag-of-words technique and the k-means algorithm were applied for exploratory analysis and data segmentation. The results indicate a high demand for Java, PHP, and Python, with a strong emphasis on database management and unit testing. It is concluded that training strategies are essential to bridge the gap between companies’ demands and professionals’ qualifications. Updating academic curricula, investing in training programs and certifications can enhance employability and meet industry demands. Additionally, clearer job descriptions and continuous analysis of market trends, supported by advanced NLP techniques, can assist professionals and companies in making strategic decisions.