MENU
  

install inspect-ai with openai/anthropic extensions installed in the same venv

uv tool install inspect-ai --with openai --with anthropic

--display [full|conversation|rich|plain|log|none] --log-level [debug|trace|http|info|warning|error|critical|notset] --log-level http # useful for debugging

--limit 2 # only run the first 2 evals --timeout 60 --model --effort [low|medium|high|xhigh|max] --reasoning-effort [none|minimal|low|medium|high|xhigh|max]

--model MODEL_ID --model-role grader=MODEL_ID use "list-all-models" script to see available model ids, for example:

    anthropic/claude-sonnet-5
    anthropic/claude-fable-5
    anthropic/claude-opus-4-8
    anthropic/claude-opus-4-7
    anthropic/claude-sonnet-4-6
    anthropic/claude-opus-4-6
    anthropic/claude-opus-4-5-20251101
    anthropic/claude-haiku-4-5-20251001
    anthropic/claude-sonnet-4-5-20250929
    anthropic/claude-opus-4-1-20250805
    openai/gpt-5.6-sol
    openai/gpt-5.6-terra
    openai/gpt-5.6-luna
    openai/gpt-5.5
    openai/gpt-5.4
    openai/gpt-5.4-mini
    openai/gpt-5.2
    openai/codex-auto-review
    ollama/gemma3:4b    

with-anthropic with-openai inspect eval simpleqa.py --model ollama/gemma3:4b --model-role grader=anthropic/claude-haiku-4-5 --log-level http