SvelteBench Visualization

Note: OpenAI thinking models (o3, o4) do not support temperature adjustments. o1-pro models use "medium" reasoning effort setting.

← Back to All Results

OpenRouter

▼

mistralai/devstral-small

▼

Test	Status	pass@1	pass@10	Passing Samples	Errors	Actions
counter	⚠️ PARTIAL	0.7000	1.0000	7/10	6	View Prompt View Tests
derived	⚠️ PARTIAL	0.1000	1.0000	1/10	13	View Prompt View Tests
derived-by	⚠️ PARTIAL	0.1000	1.0000	1/10	13	View Prompt View Tests
each	❌ FAIL	0.0000	0.0000	0/10	10	View Prompt View Tests
effect	❌ FAIL	0.0000	0.0000	0/10	10	View Prompt View Tests
hello-world	⚠️ PARTIAL	0.3000	1.0000	3/10	7	View Prompt View Tests
inspect	❌ FAIL	0.0000	0.0000	0/10	13	View Prompt View Tests
props	❌ FAIL	0.0000	0.0000	0/10	10	View Prompt View Tests
snippets	❌ FAIL	0.0000	0.0000	0/10	10	View Prompt View Tests