Human-in-the-Loop Correction of Paraphrase-Sensitive Responses in Medical Vision Language Models: A Methodological Framework for Scalable Empirical Analysis and Optimization

• Kanjana Sriwiset

  Department of Biomedical Engineering, Rajamangala University of Technology Thanyaburi, Pathum Thani, Thailand

  Author

• Phichai Thongdee

  Faculty of Informatics and Artificial Intelligence, Nakhon Si Thammarat Rajabhat University, Nakhon Si Thammarat, Thailand

  Author

• Siriporn Kaewmanee

  Department of Computer Science, Ubon Ratchathani University, Ubon Ratchathani, Thailand

  Author

• Somchai Chaiyasit

  Department of Electrical Engineering, Maejo University, Chiang Mai, Thailand

  Author

The deployment of medical vision language models (VLMs) in clinical settings demands robustness to linguistic variability, yet these models often exhibit high sensitivity to paraphrase, generating semantically inconsistent responses that threaten patient safety. This manuscript introduces a comprehensive methodological framework for scalable empirical analysis and optimization of paraphrase-sensitive responses in medical VLMs through human-in-the-loop (HITL) correction. We formalize paraphrase sensitivity as a measurable property and propose a novel correction pipeline that integrates uncertainty quantification, targeted paraphrase generation, and clinician-annotated feedback to iteratively refine model behavior. Our framework introduces a multi-objective optimization objective balancing fidelity, safety, and human effort, solved via a proximal policy optimization variant with human preference rewards. We implement a modular architecture combining a paraphrase generation module, an uncertainty estimator, a correction policy network, and a human feedback interface. Extensive experiments on medical visual question answering datasets (e.g., SLAKE, VQA-RAD, PathVQA) and a newly curated paraphrase challenge set demonstrate that our method reduces harmful paraphrase-induced errors by up to 47\% while maintaining diagnostic accuracy. Ablation studies isolate the contributions of each component, and mechanistic interpretability analyses reveal that corrections primarily adjust attention over clinically salient image regions. This work establishes a theoretical and practical foundation for safe deployment of medical VLMs, emphasizing the indispensable role of human oversight in achieving reliable performance under linguistic variability.

• PDF

Copyright (c) 2026 International Journal of Scientific and Applied Computing Research

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.