SvelteBench Visualization

GitHub View Raw Results → v1 Results

Top Models Leaderboard

Average pass@1 scores
Rank Model Score
1 claude-opus-4-6 (Anthropic)
100.0%
2 claude-opus-4-7 (Anthropic)
100.0%
3 claude-opus-4-8 (Anthropic)
100.0%
4 claude-sonnet-4-6 (Anthropic)
100.0%
5 claude-sonnet-5 (Anthropic)
100.0%
6 gemini-3.1-pro-preview (Google)
100.0%
7 gemini-3.5-flash-lite (Google)
100.0%
8 gemini-3.7-flash (Google)
100.0%
9 gemma-4-31b-it (Google)
100.0%
10 anthropic/claude-opus-4.5 (OpenRouter)
100.0%
11 grok-4.6 (xAI)
100.0%
12 gemini-3.6-flash (Google)
98.9%
13 gpt-5.6-luna (OpenAI)
98.9%
14 nex-agi/nex-n2-pro:free (OpenRouter)
98.9%
15 qwen/qwen3.5-plus-20260420 (OpenRouter)
98.9%
16 qwen/qwen3.6-max-preview (OpenRouter)
98.9%
17 qwen/qwen3.6-plus-preview:free (OpenRouter)
98.9%
18 grok-4.5 (xAI)
98.9%
19 gpt-5.2 (OpenAI)
97.8%
20 gpt-5.5 (OpenAI)
97.8%
21 qwen/qwen3.7-max (OpenRouter)
97.8%
22 qwen/qwen3.7-plus (OpenRouter)
97.8%
23 xiaomi/mimo-v2.5-pro (OpenRouter)
97.8%
24 gpt-5.3-codex (OpenAI)
96.7%
25 gpt-5.6-terra (OpenAI)
96.7%
26 minimax/minimax-m3 (OpenRouter)
96.7%
27 qwen/qwen3.8-2.4t-a95b (OpenRouter)
96.7%
28 glm-5.2 (Z.ai)
96.7%
29 kimi-k3 (Moonshot)
95.6%
30 gpt-5.4 (OpenAI)
95.6%
31 gpt-5.6-sol (OpenAI)
95.6%
32 minimax/minimax-m2.5 (OpenRouter)
95.6%
33 deepseek/deepseek-v4-flash-0731 (OpenRouter)
95.1%
34 gemini-3-flash-preview (Google)
94.4%
35 gpt-5.3-chat-latest (OpenAI)
94.4%
36 openrouter/pony-alpha (OpenRouter)
94.4%
37 qwen/qwen3.8-max (OpenRouter)
94.4%
38 x-ai/grok-build-0.1 (OpenRouter)
94.4%
39 z-ai/glm-5.3 (OpenRouter)
94.4%
40 grok-4.3 (xAI)
94.4%
41 claude-sonnet-4-5 (Anthropic)
93.3%
42 composer-2 (Cursor)
93.3%
43 openrouter/healer-alpha (OpenRouter)
93.3%
44 xiaomi/mimo-v2-flash:free (OpenRouter)
93.3%
45 xiaomi/mimo-v2-omni (OpenRouter)
93.3%
46 xiaomi/mimo-v2-pro (OpenRouter)
93.3%
47 thinkingmachines/inkling-small (OpenRouter)
92.8%
48 gemma-4-26b-a4b-it (Google)
92.2%
49 muse-spark-1.1 (Meta)
92.2%
50 kimi-k2-thinking-turbo (Moonshot)
92.2%
51 openrouter/sherlock-dash-alpha (OpenRouter)
92.2%
52 poolside/laguna-xs.2:free (OpenRouter)
92.2%
53 qwen/qwen3.5-27b (OpenRouter)
92.2%
54 openrouter/owl-alpha (OpenRouter)
92.2%
55 z-ai/glm-5-turbo (OpenRouter)
92.2%
56 stepfun/step-3.7-flash (OpenRouter)
91.1%
57 deepseek/deepseek-v3.2 (OpenRouter)
91.1%
58 deepseek/deepseek-v4-pro (OpenRouter)
91.1%
59 kwaipilot/kat-coder-pro-v2 (OpenRouter)
91.1%
60 openrouter/hunter-alpha (OpenRouter)
91.1%
61 qwen/qwen3.5-122b-a10b (OpenRouter)
91.1%
62 glm-5.1 (Z.ai)
91.1%
63 moonshotai/kimi-k2-0905 (OpenRouter)
90.0%
64 moonshotai/kimi-k2.7-code (OpenRouter)
90.0%
65 claude-sonnet-4-20250514 (Anthropic)
90.0%
66 gemini-3-pro-preview (Google)
90.0%
67 meta/muse-spark-1.2 (OpenRouter)
90.0%
68 moonshotai/kimi-k2.5 (OpenRouter)
90.0%
69 stealth/ox-alpha (OpenRouter)
90.0%
70 kimi-k2-thinking (Moonshot)
88.9%
71 gpt-5-chat-latest (OpenAI)
88.9%
72 bytedance-seed/seed-2-1-turbo (OpenRouter)
88.9%
73 deepseek/deepseek-v4-flash-vision-exp (OpenRouter)
88.9%
74 deepseek/deepseek-v4-pro-0813 (OpenRouter)
88.9%
75 kwaipilot/kat-coder-pro:free (OpenRouter)
88.9%
76 meituan/longcat-2.0 (OpenRouter)
88.9%
77 meta/muse-glimmer-30b (OpenRouter)
88.9%
78 minimax/minimax-m2.1 (OpenRouter)
88.9%
79 minimax/minimax-m2.7 (OpenRouter)
88.9%
80 mistralai/mistral-large-2512 (OpenRouter)
88.9%
81 qwen/qwen3-max (OpenRouter)
88.9%
82 qwen/qwen3.5-397b-a17b (OpenRouter)
88.9%
83 qwen/qwen3.5-plus-02-15 (OpenRouter)
88.9%
84 qwen/qwen3.6-27b (OpenRouter)
88.9%
85 tencent/hy3:free (OpenRouter)
88.9%
86 grok-4.20-experimental-beta-0304-non-reasoning (xAI)
88.9%
87 grok-code-fast-1 (xAI)
88.9%
88 glm-5 (Z.ai)
88.9%
89 gpt-5.2-codex (OpenAI)
87.8%
90 deepseek/deepseek-v3.2-speciale (OpenRouter)
87.8%
91 deepseek/deepseek-v4-flash (OpenRouter)
87.8%
92 thinkingmachines/inkling (OpenRouter)
87.8%
93 xiaomi/mimo-v2.5 (OpenRouter)
87.8%
94 z-ai/glm-4.7 (OpenRouter)
87.8%
95 grok-4.20-experimental-beta-0304-reasoning (xAI)
87.8%
96 gemini-3.1-flash-lite (Google)
86.7%
97 deepseek/deepseek-v3.2-exp (OpenRouter)
86.7%
98 qwen/qwen3.6-flash (OpenRouter)
86.7%
99 gemini-2.5-pro (Google)
85.6%
100 gemini-3.5-flash (Google)
85.6%
101 gpt-5 (OpenAI)
85.6%
102 kwaipilot/kat-coder-air-v2.5 (OpenRouter)
85.6%
103 qwen/qwen3.8-27b (OpenRouter)
85.6%
104 nvidia/nemotron-3-ultra-550b-a55b:free (OpenRouter)
84.4%
105 qwen/qwen3.6-35b-a3b (OpenRouter)
84.4%
106 tencent/hy3-preview:free (OpenRouter)
84.4%
107 z-ai/glm-4.6 (OpenRouter)
84.4%
108 z-ai/glm-5v-turbo (OpenRouter)
84.4%
109 claude-haiku-4-5-20251001 (Anthropic)
84.4%
110 gemini-3.1-flash-lite-preview (Google)
84.4%
111 kimi-k2.6 (Moonshot)
84.4%
112 openai/gpt-5.4-mini (OpenRouter)
83.3%
113 gpt-5.1-chat-latest (OpenAI)
83.3%
114 bytedance-seed/seed-2.0-code (OpenRouter)
83.3%
115 openai/gpt-5.1-chat (OpenRouter)
83.3%
116 qwen/qwen3.7-flash (OpenRouter)
82.5%
117 kwaipilot/kat-coder-pro-v2.5 (OpenRouter)
82.2%
118 poolside/laguna-s-2.1 (OpenRouter)
81.1%
119 upstage/solar-pro4 (OpenRouter)
81.1%
120 poolside/laguna-xs-2.1 (OpenRouter)
81.1%
121 bytedance-seed/seed-2.0-lite (OpenRouter)
80.0%
122 openai/gpt-5.1 (OpenRouter)
80.0%
123 openrouter/sherlock-think-alpha (OpenRouter)
80.0%
124 poolside/laguna-m.1:free (OpenRouter)
80.0%
125 stepfun/step-3.5-flash:free (OpenRouter)
80.0%
126 x-ai/grok-4.1-fast (OpenRouter)
80.0%
127 bytedance-seed/seed-2.0-mini (OpenRouter)
78.9%
128 qwen/qwen3-max-thinking (OpenRouter)
78.9%
129 qwen/qwen3.5-35b-a3b (OpenRouter)
77.8%
130 bytedance-seed/seed-1.6 (OpenRouter)
76.7%
131 minimax/minimax-m2 (OpenRouter)
75.6%
132 openrouter/polaris-alpha (OpenRouter)
73.3%
133 qwen/qwen3-coder-next (OpenRouter)
71.1%
134 nvidia/nemotron-3-super-120b-a12b:free (OpenRouter)
70.0%
135 inclusionai/ling-2.6-1t (OpenRouter)
68.9%
136 mistralai/mistral-small-2603 (OpenRouter)
68.9%
137 gpt-5.1-codex-max (OpenAI)
67.8%
138 moonshotai/kimi-linear-48b-a3b-instruct (OpenRouter)
66.7%
139 cohere/north-mini-code:free (OpenRouter)
64.4%
140 openai/gpt-5.4-nano (OpenRouter)
63.3%
141 z-ai/glm-4.7-flash (OpenRouter)
60.0%
142 amazon/nova-2-lite-v1:free (OpenRouter)
56.7%
143 prime-intellect/intellect-3 (OpenRouter)
55.6%
144 nvidia/nemotron-3.5-lightning (OpenRouter)
55.6%
145 arcee-ai/trinity-large-thinking (OpenRouter)
50.0%
146 mistralai/devstral-2512:free (OpenRouter)
50.0%
147 gpt-5-codex (OpenAI)
47.8%
148 openai/gpt-5.1-codex (OpenRouter)
45.6%
149 inception/mercury-2 (OpenRouter)
44.4%
150 qwen/qwen3.5-9b (OpenRouter)
42.2%
151 bytedance-seed/seed-1.6-flash (OpenRouter)
35.6%
152 openrouter/aurora-alpha (OpenRouter)
34.4%
153 essentialai/rnj-1-instruct (OpenRouter)
30.0%
154 gpt-5-mini (OpenAI)
23.3%
155 arcee-ai/trinity-large-preview:free (OpenRouter)
23.3%
156 mistralai/ministral-3b-2512 (OpenRouter)
22.2%
157 upstage/solar-pro-3:free (OpenRouter)
20.0%
158 nvidia/nemotron-3-nano-30b-a3b:free (OpenRouter)
18.9%
159 ibm-granite/granite-4.1-8b (OpenRouter)
16.7%
160 openai/gpt-5.1-codex-mini (OpenRouter)
16.7%
161 openrouter/elephant-alpha (OpenRouter)
15.6%
162 gpt-5-nano (OpenAI)
13.3%
163 liquid/lfm-2-24b-a2b (OpenRouter)
13.3%
164 liquid/lfm-2.5-2.6b:free (OpenRouter)
11.1%
165 qwen/qwen3-vl-8b-instruct (OpenRouter)
11.1%
166 deepcogito/cogito-v2-preview-llama-405b (OpenRouter)
10.0%
167 afm-3-core (Apple)
8.9%
168 mistralai/ministral-8b-2512 (OpenRouter)
6.7%
169 mistralai/ministral-14b-2512 (OpenRouter)
3.3%
170 allenai/olmo-3.1-32b-instruct (OpenRouter)
2.2%
171 allenai/olmo-3.1-32b-think:free (OpenRouter)
0.0%
172 qwen/qwen3.5-flash-02-23 (OpenRouter)
0.0%

Note: Certain OpenAI thinking models (o3, o4) and gpt-5 do not support temperature adjustments (only default value of 1 is supported). Models with "-reasoning-" suffix (e.g., gpt-5-2025-08-07-reasoning-medium) will use the specified reasoning effort setting.

Errata: The "inspect" test has known correctness issues but is retained in the benchmark suite to maintain consistency and fairness in scoring across all evaluated models.

Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 16
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 3/3 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 23
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 12
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 60% 100% 6/10 8
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 9/9 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 13
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 11
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 4
Prompt Tests
derived-by 70% 100% 7/10 9
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 12
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 18
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 10
Prompt Tests
derived 40% 100% 4/10 6
Prompt Tests
derived-by 30% 100% 3/10 7
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 20% 100% 2/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 40% 100% 4/10 6
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 50% 100% 5/10 5
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 50% 100% 5/10 5
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 20% 100% 2/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 20% 100% 2/10 8
Prompt Tests
derived 0% 0% 0/10 13
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 12
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 18
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 70% 100% 7/10 6
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 70% 100% 7/10 3
Prompt Tests
derived-by 40% 100% 4/10 8
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 16
Prompt Tests
props 60% 100% 6/10 4
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 30% 100% 3/10 13
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 12
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 11
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 10% 100% 1/10 9
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 70% 100% 7/10 3
Prompt Tests
each 70% 100% 7/10 4
Prompt Tests
effect 60% 100% 6/10 4
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 30% 100% 3/10 7
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 11
Prompt Tests
effect 60% 100% 6/10 4
Prompt Tests
hello-world 80% 100% 8/10 4
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 15
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 80% 100% 8/10 2
Prompt Tests
effect 30% 100% 3/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 3
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 20% 100% 2/10 8
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 60% 100% 6/10 4
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 30% 100% 3/10 7
Prompt Tests
effect 0% 0% 0/10 16
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 10% 100% 1/10 9
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 16
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 40% 100% 4/10 10
Prompt Tests
each 80% 100% 8/10 3
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 6/6 0
Prompt Tests
inspect 78% 100% 7/9 4
Prompt Tests
props 100% 100% 5/5 0
Prompt Tests
snippets 78% 100% 7/9 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 12
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 9
Prompt Tests
derived 50% 100% 5/10 7
Prompt Tests
derived-by 40% 100% 4/10 8
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 10% 100% 1/10 10
Prompt Tests
hello-world 60% 100% 6/10 6
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 0% 0% 0/10 13
Prompt Tests
snippets 20% 100% 2/10 14
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 30
Prompt Tests
derived 30% 100% 3/10 10
Prompt Tests
derived-by 10% 100% 1/10 13
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 15
Prompt Tests
derived-by 50% 100% 5/10 9
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 50% 100% 5/10 6
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 24
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 80% 100% 8/10 3
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 40% 100% 4/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 25
Prompt Tests
derived 20% 100% 2/10 11
Prompt Tests
derived-by 0% 0% 0/10 24
Prompt Tests
each 0% 0% 0/10 11
Prompt Tests
effect 0% 0% 0/10 12
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 24
Prompt Tests
derived 0% 0% 0/7 7
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 12
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 19
Prompt Tests
props 10% 100% 1/10 14
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 15
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 8
Prompt Tests
derived 80% 100% 8/10 4
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 80% 100% 8/10 3
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 9/9 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 70% 100% 7/10 6
Prompt Tests
snippets 40% 100% 4/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 17
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 90% 100% 9/10 4
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 40% 100% 4/10 6
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 15
Prompt Tests
props 0% 0% 0/10 22
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 60% 100% 6/10 5
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 60% 100% 6/10 4
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 0% 0% 0/10 19
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 80% 100% 8/10 3
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 70% 100% 7/10 3
Prompt Tests
each 30% 100% 3/10 7
Prompt Tests
effect 50% 100% 5/10 6
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 60% 100% 6/10 5
Prompt Tests
snippets 40% 100% 4/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 4
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 20
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 0% 0% 0/10 20
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 10% 100% 1/10 20
Prompt Tests
props 0% 0% 0/10 11
Prompt Tests
snippets 0% 0% 0/10 22
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 70% 100% 7/10 7
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 19
Prompt Tests
props 30% 100% 3/10 7
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 14
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 30% 100% 3/10 14
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 10% 100% 1/10 32
Prompt Tests
props 30% 100% 3/10 11
Prompt Tests
snippets 10% 100% 1/10 13
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 30% 100% 3/10 15
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 40% 100% 4/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 4
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 30% 100% 3/10 7
Prompt Tests
derived-by 10% 100% 1/10 9
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 20% 100% 2/10 8
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 50% 100% 5/10 10
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 50% 100% 5/10 6
Prompt Tests
effect 20% 100% 2/10 12
Prompt Tests
hello-world 50% 100% 5/10 5
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 0% 0% 0/10 13
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 16
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 16
Prompt Tests
derived-by 30% 100% 3/10 9
Prompt Tests
each 60% 100% 6/10 4
Prompt Tests
effect 20% 100% 2/10 9
Prompt Tests
hello-world 60% 100% 6/10 4
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 12
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 30% 100% 3/10 7
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 20% 100% 2/10 8
Prompt Tests
snippets 0% 0% 0/10 28
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 80% 100% 8/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 80% 100% 8/10 5
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 70% 100% 7/10 7
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 34
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 10% 100% 1/10 17
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 40% 100% 4/10 6
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 40% 100% 4/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 30
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 4
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 6
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 40% 100% 4/10 10
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 20% 100% 2/10 13
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 27
Prompt Tests
props 50% 100% 5/10 5
Prompt Tests
snippets 20% 100% 2/10 14
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 10% 100% 1/10 9
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 9
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 19
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 3
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 6
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 70% 100% 7/10 6
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 14
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 8
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 60% 100% 6/10 4
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 7
Prompt Tests
derived 50% 100% 5/10 6
Prompt Tests
derived-by 60% 100% 6/10 9
Prompt Tests
each 50% 100% 5/10 9
Prompt Tests
effect 40% 100% 4/10 8
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 10% 100% 1/10 15
Prompt Tests
props 30% 100% 3/10 13
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 6/6 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 9
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 8
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 15
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 8/8 0
Prompt Tests
hello-world 100% 100% 5/5 0
Prompt Tests
inspect 62% 100% 5/8 3
Prompt Tests
props 100% 100% 5/5 0
Prompt Tests
snippets 0% 0% 0/2 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 4
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 21
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 3
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 50% 100% 5/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 5
Prompt Tests
props 90% 100% 9/10 4
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 8/8 0
Prompt Tests
derived 100% 100% 8/8 0
Prompt Tests
derived-by 100% 100% 8/8 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 100% 100% 7/7 0
Prompt Tests
hello-world 100% 100% 8/8 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 9/9 0
Prompt Tests
snippets 75% 100% 6/8 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 7
Prompt Tests
derived 0% 0% 0/10 12
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 13
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 17
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 60% 100% 6/10 7
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 31
Prompt Tests
props 90% 100% 9/10 2
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 40% 100% 4/10 12
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 50% 100% 5/10 5
Prompt Tests
each 60% 100% 6/10 4
Prompt Tests
effect 40% 100% 4/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 10
Prompt Tests
props 40% 100% 4/10 10
Prompt Tests
snippets 10% 100% 1/10 13
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 90% 100% 9/10 2
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests