Lists
Lists
Abrar, Umar and Homayounvala, Elaheh (2026) Multi-trait writing quality assessment system using transformer-based models with explanatory feedback and robustness analysis. In: 14th International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA-2026), 8-9 June 2026, London. (In Press), 8-9 June 2026, London. (In Press)
Automated Essay Scoring has become highly important in educational technology and natural language processing due to the growing demand for scalable, cost-effective, and time-efficient writing assessment systems. Traditional manual text evaluation is labour-intensive and time-consuming, and it often generates inconsistencies between human evaluators. Earlier traditional AES systems relied on handcrafted linguistic features, which often struggled to capture semantic coherence and contextual relationships within the text. Recent advancements in transformer-based architectures have significantly improved contextual representation learning for writing assessment tasks.
This paper presents a transformer-based multi-trait writing quality assessment system using a DistilBert-based multi-output regression architecture. Unlike many existing AES studies that focus on holistic scoring, the proposed system simultaneously predicts multiple writing traits, including content, organisation, word choice, sentence fluency, conventions and overall textual quality. In addition, the study also introduces robustness evaluation under noisy textual inputs along with cross-dataset generalisation analysis to investigate the model’s reliability and transferability across different datasets. Experiments conducted on ASAP++ and LEAF++ datasets which demonstrated strong performance, obtaining the average QWK of 0.704 and an overall score agreement of 0.778 on ASAP++. Robustness evaluation demonstrated stability under punctuation removal and sentence duplication, but typographical noise caused substantial performance degradation. Furthermore, a Streamlit-based deployment prototype was developed to provide real-time textual assessment together with the trait-level explanatory feedback and writing improvement suggestions. Overall, the findings in this study demonstrate that transformer-based models are highly effective in modelling semantic and contextual dimensions of writing quality.
![[thumbnail of 286_CRC.pdf]](https://repository.londonmet.ac.uk/style/images/fileicons/text.png "286_CRC.pdf")
Restricted to Repository staff only until 25 May 2027.
Download (625kB) | Request a copy
 |
View Item |