TA-RAG: Confidence-Calibrated Statement-Level Verification for Hallucination Resilient Regulatory Knowledge Systems

Abstract

Despite the heavily deployment of large language model (LLM) integrated with Retrieval Augmented Generation (RAG) for enterprise’s regulatory systems, they still generate hallucinated responses unsupported by the retrieved information. In highstakes areas such as banking, legal compliances, this can present potentially real liability and operational risks. In this paper, we propose a Trust Aware RAG Framework and demonstrate that it can be used to transparently incorporate multistage statement level verification and confidence scoring in the standard RAG pipeline. Generated answers are effectively decomposed into atomic claims and determined to be correct or not with respect to retrieved documents according to a support-based confidence score, achieving calibrated answer rejection with poor evidence coverage. Tested on a real-world RBI regulatory document corpus with factual and outofscope queries, the system shows significant reduction in hallucinations and increased reliability over RAG baselines. The proposed methodology provides a practical way to a more secure and reliable enterprise knowledge system.