SisTIRA: plataforma web de tutoria inteligente para avaliação automática de respostas discursivas

Abstract

This paper introduces the Intelligent Automatic Response Tutoring System (SisTIRA), a platform based on Natural Language Processing (NLP) and Machine Learning (ML) techniques for automated assessment of short answer responses in Portuguese. SisTIRA integrates a language model as an automatic evaluator in the LLM-as-judge paradigm, providing accurate and scalable grading. The research explores how NLP technology, combined with advanced language models like Gemini-1.5-flash-8b, can be used to assess responses in a more dynamic and fair manner, enhancing the possibilities for personalization and feedback. The study includes a preliminary analysis with 20 rounds of evaluation, demonstrating the system's consistency and the feasibility of automating the grading of discursive answers in educational environments like Virtual Learning Environments (VLEs).