![Phantom](/sites/default/files/styles/whats_news/public/2023-10/Screenshot%202023-10-29%20at%2012.47.08%20PM.png?h=7621c87f)
Source https://hyunw.kim/fantom/
Theory of mind (ToM) evaluations currently focus on testing models using passive narratives that inherently lack interactivity. This team introduces FANToM 👻, a new benchmark designed to stress-test ToM within information-asymmetric conversational contexts via question answering. The authors show how the benchmark draws upon important theoretical requisites from psychology and necessary empirical considerations when evaluating large language models (LLMs).
Log in or register to join the discussion