install inspect-ai with openai/anthropic extensions installed in the same venv
uv tool install inspect-ai --with openai --with anthropic
--display [full|conversation|rich|plain|log|none] --log-level [debug|trace|http|info|warning|error|critical|notset] --log-level http # useful for debugging
--limit 2 # only run the first 2 evals --timeout 60 --model --effort [low|medium|high|xhigh|max] --reasoning-effort [none|minimal|low|medium|high|xhigh|max]
--model MODEL_ID --model-role grader=MODEL_ID use "list-all-models" script to see available model ids, for example:
anthropic/claude-sonnet-5 anthropic/claude-fable-5 anthropic/claude-opus-4-8 anthropic/claude-opus-4-7 anthropic/claude-sonnet-4-6 anthropic/claude-opus-4-6 anthropic/claude-opus-4-5-20251101 anthropic/claude-haiku-4-5-20251001 anthropic/claude-sonnet-4-5-20250929 anthropic/claude-opus-4-1-20250805 openai/gpt-5.6-sol openai/gpt-5.6-terra openai/gpt-5.6-luna openai/gpt-5.5 openai/gpt-5.4 openai/gpt-5.4-mini openai/gpt-5.2 openai/codex-auto-review ollama/gemma3:4bwith-anthropic with-openai inspect eval simpleqa.py --model ollama/gemma3:4b --model-role grader=anthropic/claude-haiku-4-5 --log-level http