Independent AI research
We test
the intent.
When you push back on an AI, does it stick to the truth or tell you what you want to hear? We measure it, model by model, and publish everything.
First study: the pushback test
We asked 8 widely used AI models hundreds of questions that have a correct answer. Then we pushed back, once and then five times in a row, and recorded when they gave up a correct answer.
Two pre-registered studies. Full results, method, prompts and data will be published here on 15 October.
How we work
Pre-registered
Hypotheses and analysis are fixed on OSF before every run.
Open
Method, prompts, code and data are public.
Fair
Every lab gets the results and a chance to reply before we publish.
Independent
Not affiliated with or funded by any AI company.
Who
aiforensic.io is run by Omri Sigelman, an independent researcher. Contact me on LinkedIn.
Disclosure: Claude, an AI model by Anthropic, helps build and analyse our studies. Anthropic models are also among the models we test.