Articuli: plataforma de tradução de datilologia com base em visão computacional para formação de palavras

Abstract

This work presents the development of a platform proposal called ArticuLi, a responsive web application that performs the formation of words from images of signs from the dactilological alphabet of the Brazilian Sign Language (LIBRAS), translated in sequence and with the option of reproduction in audio. ArticuLi uses a pre-trained Convolutional Neural Network model based on a dataset, the Common Objects in Context (COCO). Through the model, key points are identified, that is, key points that represent the hand articulations and that are necessary to estimate your pose. Using the OpenCV library, the algorithm analyzes the coordinates of the mapped points to discover the distance, height and proximity between them and translate the signals sequentially to return a word. The Minimum Viable Product (MVP) tests were performed with images photographed by 10 volunteers using their smartphones as a capture device. Thus, a balanced set with 5 samples for each signal was obtained, totaling 130 images. Considering the total number of images and the amount of correctly translated signals, including static and dynamic signals, the proposal achieved an accuracy of 94.6%, that is, of the 130 images analyzed, 123 were correctly translated where the letters A, B, C , D, E, G, I, K, L, M, N, O, P, Q, R, S, T, U, V, W and Y obtained 100% accuracy, the letters F, H, J and Z 80% and X 40% accuracy, making it able to form words