LLM Evaluation

← Back to main page

Run of 2026-02-11 15:01:31

15
Models Tested
13
Total Tasks
52.3%
Overall Success Rate
102/195
Passed Combinations

Details

Score Model task10_multiple_tests task11_relationship_classifier task12_need_reply task13_meeting_action_items task14_graph_money_distribution task1_file_list task2_fix_python_syntax task4_web_fetch task5_dedup_contact task6_config_merger task7_log_parser task8_regex_extraction task9_cpp_footguns
84.6% openrouter/google/gemini-3-pro-preview ✅ ✅ ✅ ❌ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅
84.6% openrouter/anthropic/claude-sonnet-4.5 ✅ ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅
76.9% openrouter/anthropic/claude-opus-4.5 ✅ ✅ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
76.9% openrouter/anthropic/claude-haiku-4.5 ✅ ✅ ❌ ✅ ❌ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅
76.9% openrouter/x-ai/grok-code-fast-1 ✅ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅
69.2% openrouter/google/gemini-2.5-pro ✅ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ✅
53.8% openrouter/deepseek/deepseek-v3.1-terminus ✅ ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ❌
46.2% openrouter/qwen/qwen3-coder ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ✅ ✅
46.2% openrouter/x-ai/grok-3-mini ✅ ✅ ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌
46.2% openrouter/google/gemini-2.5-flash-lite-preview-09-2025 ✅ ✅ ✅ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ❌
38.5% openrouter/openai/gpt-oss-120b ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ✅ ✅ ✅ ❌
38.5% openrouter/openai/gpt-oss-20b ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌
23.1% openrouter/google/gemini-2.5-flash-preview-09-2025 ❌ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ✅ ✅ ❌
23.1% openrouter/openai/gpt-4o-mini ✅ ✅ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ❌ ❌
0.0% openrouter/deepseek/deepseek-chat-v3-0324 ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌