This research paper introduces a new task called hallucination reasoning, which aims to identify the underlying causes of hallucinations generated by large language models (LLMs). The authors propose a novel zero-shot method called Model Knowledge Test (MKT) to assess whether an LLM has sufficient knowledge to generate a response. The MKT perturbs the subject of the prompt and analyzes the impact on the generated text, distinguishing between fabricated text (lack of knowledge) and misaligned text (sampling randomness or dependencies). This approach significantly enhances existing hallucination detection methods, demonstrating the importance of understanding hallucination causes for improving LLM reliability.

https://ar ...