SvelteBench Visualization

GitHub View Raw Results → v1 Results

Top Models Leaderboard

Average pass@1 scores
Rank Model
1 claude-opus-4-6 (Anthropic)
100.0%
— —
2 claude-opus-4-7 (Anthropic)
100.0%
— —
3 claude-opus-4-8 (Anthropic)
100.0%
— —
4 claude-sonnet-4-6 (Anthropic)
100.0%
— —
5 claude-sonnet-5 (Anthropic)
100.0%
— —
6 claude-sonnet-5-5 (Anthropic)
100.0%
— —
7 gemini-3.1-pro-preview (Google)
100.0%
— —
8 gemini-3.5-flash-lite (Google)
100.0%
— —
9 gemini-3.7-flash (Google)
100.0%
— —
10 gemini-4-argon (Google)
100.0%
— —
11 gemini-4-hydrogen (Google)
100.0%
— —
12 gemma-4-31b-it (Google)
100.0%
— —
13 gpt-5.3-codex (OpenAI)
100.0%
— —
14 gpt-5.4 (OpenAI)
100.0%
— —
15 gpt-5.5 (OpenAI)
100.0%
— —
16 gpt-5.6-luna (OpenAI)
100.0%
— —
17 gpt-6-astra (OpenAI)
100.0%
— —
18 gpt-6-luna (OpenAI)
100.0%
— —
19 gpt-6-sol (OpenAI)
100.0%
— —
20 gpt-6.1-sol (OpenAI)
100.0%
— —
21 anthropic/claude-opus-4.5 (OpenRouter)
100.0%
— —
22 deepseek/deepseek-v4.1-flash (OpenRouter)
100.0%
— —
23 deepseek/deepseek-v4.1-flash-1017 (OpenRouter)
100.0%
— —
24 google/gemini-3.8-flash (OpenRouter)
100.0%
— —
25 openai/gpt-6.1-terra (OpenRouter)
100.0%
— —
26 openrouter/pareto-code (OpenRouter)
100.0%
— —
27 particle-ai/particle-1 (OpenRouter)
100.0%
— —
28 sakana/fugu-max (OpenRouter)
100.0%
— —
29 sakana/fugu-ultra-v2 (OpenRouter)
100.0%
— —
30 unbiased/pareto (OpenRouter)
100.0%
— —
31 z-ai/glm-5.3-flashx (OpenRouter)
100.0%
— —
32 Beam-1.0-501B-A23B (Reflection)
100.0%
— —
33 grok-4.5 (xAI)
100.0%
— —
34 grok-4.6 (xAI)
100.0%
— —
35 gemini-3.6-flash (Google)
98.9%
— —
36 nex-agi/nex-n2-pro:free (OpenRouter)
98.9%
— —
37 nex-agi/nex-n2.5-pro:free (OpenRouter)
98.9%
— —
38 qwen/qwen3.5-plus-20260420 (OpenRouter)
98.9%
— —
39 qwen/qwen3.6-max-preview (OpenRouter)
98.9%
— —
40 qwen/qwen3.6-plus-preview:free (OpenRouter)
98.9%
— —
41 z-ai/glm-5.3-flash (OpenRouter)
98.9%
— —
42 grok-4.7 (xAI)
98.9%
— —
43 muse-ember-1.0 (Meta)
97.8%
— —
44 kimi-k3 (Moonshot)
97.8%
— —
45 kimi-k3.1-pro (Moonshot)
97.8%
— —
46 gpt-5.2 (OpenAI)
97.8%
— —
47 baidu/ernie-light-dt-1.0-intl (OpenRouter)
97.8%
— —
48 qwen/qwen3.7-max (OpenRouter)
97.8%
— —
49 qwen/qwen3.7-plus (OpenRouter)
97.8%
— —
50 qwen/qwen3.8-2.4t-a95b (OpenRouter)
97.8%
— —
51 stealth/ox-alpha (OpenRouter)
97.8%
— —
52 xiaomi/mimo-v2.5-pro (OpenRouter)
97.8%
— —
53 gpt-5.6-terra (OpenAI)
96.7%
— —
54 inclusionai/ling-3.0-flash-sante:free (OpenRouter)
96.7%
— —
55 minimax/minimax-m3 (OpenRouter)
96.7%
— —
56 unbiased/pareto-26.9 (OpenRouter)
96.7%
— —
57 thinkingmachines/Inkling-2-Small (Thinking Machines Tinker)
96.7%
— —
58 glm-5.2 (Z.ai)
96.7%
— —
59 deepseek/deepseek-v4-flash-0731 (OpenRouter)
96.3%
— —
60 claude-haiku-5-5 (Anthropic)
95.6%
— —
61 composer-2 (Cursor)
95.6%
— —
62 gpt-5.3-chat-latest (OpenAI)
95.6%
— —
63 gpt-5.6-sol (OpenAI)
95.6%
— —
64 minimax/minimax-m2.5 (OpenRouter)
95.6%
— —
65 poolside/laguna-xs.2:free (OpenRouter)
95.6%
— —
66 qwen/qwen3.8-max (OpenRouter)
95.6%
— —
67 stealth/crouton-alpha (OpenRouter)
95.6%
— —
68 unbiased/pareto-26.10-preview (OpenRouter)
95.6%
— —
69 xiaomi/mimo-v2-omni (OpenRouter)
95.6%
— —
70 gemini-3-flash-preview (Google)
94.4%
— —
71 google/gemini-omni-1.2-flash (OpenRouter)
94.4%
— —
72 meta/muse-spark-1.3 (OpenRouter)
94.4%
— —
73 meta/muse-spark-1.3-contributor (OpenRouter)
94.4%
— —
74 openrouter/healer-alpha (OpenRouter)
94.4%
— —
75 openrouter/pony-alpha (OpenRouter)
94.4%
— —
76 x-ai/grok-build-0.1 (OpenRouter)
94.4%
— —
77 xiaomi/mimo-v2-pro (OpenRouter)
94.4%
— —
78 z-ai/glm-5.3 (OpenRouter)
94.4%
— —
79 grok-4.3 (xAI)
94.4%
— —
80 claude-opus-5-5 (Anthropic)
93.3%
— —
81 claude-sonnet-4-5 (Anthropic)
93.3%
— —
82 hf.co/webAI-Official/Air-26B-A4B-gguf:latest (Ollama) Model page ↗
93.3%
18.1 tok/s
~1m 59s* 2,166 tok
83 qwen3.6:27b-coding-mtp-q4_K_M (Ollama) Model page ↗
93.3%
9.1 tok/s
~9m 27s* 5,175 tok
84 gemma4:31b-it-q4_K_M (Ollama) Model page ↗
93.3%
3.6 tok/s
~2m 48s* 603 tok
85 qwen/qwen3.5-27b (OpenRouter)
93.3%
— —
86 xiaomi/mimo-v2-flash:free (OpenRouter)
93.3%
— —
87 thinkingmachines/inkling-small (OpenRouter)
92.8%
— —
88 gemma-4-26b-a4b-it (Google)
92.2%
— —
89 muse-spark-1.1 (Meta)
92.2%
— —
90 kimi-k2-thinking-turbo (Moonshot)
92.2%
— —
91 flint:40b-a2b-gguf-nvfp4 (Ollama) Model page ↗
92.2%
27.6 tok/s
~1m 3s* 1,726 tok
92 kwaipilot/kat-coder-pro-v2 (OpenRouter)
92.2%
— —
93 openrouter/owl-alpha (OpenRouter)
92.2%
— —
94 openrouter/sherlock-dash-alpha (OpenRouter)
92.2%
— —
95 stepfun/step-3.7-flash (OpenRouter)
92.2%
— —
96 stepfun/step-5-preview (OpenRouter)
92.2%
— —
97 xiaomi/mimo-v2.6-flash (OpenRouter)
92.2%
— —
98 xiaomi/mimo-v2.6-pro (OpenRouter)
92.2%
— —
99 z-ai/glm-5-turbo (OpenRouter)
92.2%
— —
100 mistralai/mistral-large-4-0 (OpenRouter)
91.7%
— —
101 deepseek/deepseek-v3.2 (OpenRouter)
91.1%
— —
102 deepseek/deepseek-v4-pro (OpenRouter)
91.1%
— —
103 meta/muse-spark-open (OpenRouter)
91.1%
— —
104 openrouter/hunter-alpha (OpenRouter)
91.1%
— —
105 qwen/qwen3.5-122b-a10b (OpenRouter)
91.1%
— —
106 glm-5.1 (Z.ai)
91.1%
— —
107 claude-sonnet-4-20250514 (Anthropic)
90.0%
— —
108 gemini-3-pro-preview (Google)
90.0%
— —
109 nemotron-3.5-bolt:74b-a9b-iq2 (Ollama) Model page ↗
90.0%
18.2 tok/s
~41s* 750 tok
110 muse-glimmer:30b-q4_K_M (Ollama) Model page ↗
90.0%
4.3 tok/s
~3m 44s* 970 tok
111 deepseek/deepseek-v4-flash (OpenRouter)
90.0%
— —
112 deepseek/deepseek-v4-pro-0813 (OpenRouter)
90.0%
— —
113 meituan/longcat-2.0 (OpenRouter)
90.0%
— —
114 meta/muse-spark-1.2 (OpenRouter)
90.0%
— —
115 meta/muse-spark-1.2-contributor (OpenRouter)
90.0%
— —
116 minimax/minimax-m2.7 (OpenRouter)
90.0%
— —
117 minimax/minimax-m3.1-flash:free (OpenRouter)
90.0%
— —
118 moonshotai/kimi-k2-0905 (OpenRouter)
90.0%
— —
119 moonshotai/kimi-k2.5 (OpenRouter)
90.0%
— —
120 moonshotai/kimi-k2.7-code (OpenRouter)
90.0%
— —
121 qwen/qwen3.5-plus-02-15 (OpenRouter)
90.0%
— —
122 qwen/qwen3.6-27b (OpenRouter)
90.0%
— —
123 qwen/qwen3.8-max-0902 (OpenRouter)
90.0%
— —
124 gemini-3.1-flash-lite (Google)
88.9%
— —
125 kimi-k2-thinking (Moonshot)
88.9%
— —
126 hf.co/mrgnw/gemma-4-e2b-svelte5:Q4_K_M (Ollama) Model page ↗
88.9%
38.6 tok/s
~3.8s* 146 tok
127 qwen3.6:35b-a3b-q4_K_M (Ollama) Model page ↗
88.9%
24.4 tok/s
~2m 56s* 4,298 tok
128 gemma4:26b-a4b-it-q4_K_M (Ollama) Model page ↗
88.9%
20.6 tok/s
~1m 54s* 2,336 tok
129 gpt-5-chat-latest (OpenAI)
88.9%
— —
130 bytedance-seed/seed-2-1-turbo (OpenRouter)
88.9%
— —
131 deepseek/deepseek-v4-flash-vision-exp (OpenRouter)
88.9%
— —
132 inception/mercury-2.5-preview (OpenRouter)
88.9%
— —
133 kwaipilot/kat-coder-pro:free (OpenRouter)
88.9%
— —
134 meta/muse-glimmer-30b (OpenRouter)
88.9%
— —
135 minimax/minimax-m2.1 (OpenRouter)
88.9%
— —
136 mistralai/mistral-large-2512 (OpenRouter)
88.9%
— —
137 qwen/qwen3-max (OpenRouter)
88.9%
— —
138 qwen/qwen3.5-397b-a17b (OpenRouter)
88.9%
— —
139 qwen/qwen3.8-flash (OpenRouter)
88.9%
— —
140 tencent/hy3:free (OpenRouter)
88.9%
— —
141 grok-4.20-experimental-beta-0304-non-reasoning (xAI)
88.9%
— —
142 grok-code-fast-1 (xAI)
88.9%
— —
143 glm-5 (Z.ai)
88.9%
— —
144 qwen3.8:27b-mtp-q4_K_M (Ollama) Model page ↗
87.8%
10.2 tok/s
~50s* 509 tok
145 gpt-5.2-codex (OpenAI)
87.8%
— —
146 deepseek/deepseek-v3.2-speciale (OpenRouter)
87.8%
— —
147 nex-agi/nex-n2.5-mini:free (OpenRouter)
87.8%
— —
148 thinkingmachines/inkling (OpenRouter)
87.8%
— —
149 xiaomi/mimo-v2.5 (OpenRouter)
87.8%
— —
150 z-ai/glm-4.7 (OpenRouter)
87.8%
— —
151 grok-4.20-experimental-beta-0304-reasoning (xAI)
87.8%
— —
152 gpt-5 (OpenAI)
86.7%
— —
153 deepseek/deepseek-v3.2-exp (OpenRouter)
86.7%
— —
154 nvidia/nemotron-3-ultra-550b-a55b:free (OpenRouter)
86.7%
— —
155 qwen/qwen3.6-flash (OpenRouter)
86.7%
— —
156 gemini-2.5-pro (Google)
85.6%
— —
157 gemini-3.1-flash-lite-preview (Google)
85.6%
— —
158 gemini-3.5-flash (Google)
85.6%
— —
159 kimi-k2.6 (Moonshot)
85.6%
— —
160 kwaipilot/kat-coder-air-v2.5 (OpenRouter)
85.6%
— —
161 qwen/qwen3.6-35b-a3b (OpenRouter)
85.6%
— —
162 qwen/qwen3.8-27b (OpenRouter)
85.6%
— —
163 stealth/space-bunny-alpha (OpenRouter)
85.6%
— —
164 claude-haiku-4-5-20251001 (Anthropic)
84.4%
— —
165 hf.co/bartowski/nex-agi_Nex-N2.5-mini-GGUF:Q4_K_M (Ollama) Model page ↗
84.4%
20.1 tok/s
~23s* 469 tok
166 hf.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M (Ollama) Model page ↗
84.4%
3.5 tok/s
~3m 24s* 718 tok
167 qwen/qwen3-max-thinking (OpenRouter)
84.4%
— —
168 tencent/hy3-preview:free (OpenRouter)
84.4%
— —
169 z-ai/glm-4.6 (OpenRouter)
84.4%
— —
170 z-ai/glm-5v-turbo (OpenRouter)
84.4%
— —
171 gpt-5.1-chat-latest (OpenAI)
83.3%
— —
172 bytedance-seed/seed-2.0-code (OpenRouter)
83.3%
— —
173 openai/gpt-5.1-chat (OpenRouter)
83.3%
— —
174 openai/gpt-5.4-mini (OpenRouter)
83.3%
— —
175 upstage/solar-pro4 (OpenRouter)
83.3%
— —
176 qwen/qwen3.7-flash (OpenRouter)
82.5%
— —
177 kwaipilot/kat-coder-pro-v2.5 (OpenRouter)
82.2%
— —
178 stepfun/step-3.5-flash:free (OpenRouter)
82.2%
— —
179 ornith-1.5:35b-q4_K_M (Ollama) Model page ↗
81.1%
25.3 tok/s
~9.4s* 238 tok
180 openai/gpt-5.1 (OpenRouter)
81.1%
— —
181 poolside/laguna-s-2.1 (OpenRouter)
81.1%
— —
182 poolside/laguna-xs-2.1 (OpenRouter)
81.1%
— —
183 bytedance-seed/seed-2.0-lite (OpenRouter)
80.0%
— —
184 openrouter/sherlock-think-alpha (OpenRouter)
80.0%
— —
185 poolside/laguna-m.1:free (OpenRouter)
80.0%
— —
186 ricursiveai/ripple-65b-preview (OpenRouter)
80.0%
— —
187 x-ai/grok-4.1-fast (OpenRouter)
80.0%
— —
188 bytedance-seed/seed-2.0-mini (OpenRouter)
78.9%
— —
189 hf.co/rockypod/svelte-coder:Q4_K_M (Ollama) Model page ↗
77.8%
8.1 tok/s
~33s* 270 tok
190 inclusionai/ling-2.6-1t (OpenRouter)
77.8%
— —
191 qwen/qwen3.5-35b-a3b (OpenRouter)
77.8%
— —
192 bytedance-seed/seed-1.6 (OpenRouter)
76.7%
— —
193 minimax/minimax-m2 (OpenRouter)
75.6%
— —
194 hf.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M (Ollama) Model page ↗
73.3%
38.6 tok/s
~33s* 1,271 tok
195 openrouter/polaris-alpha (OpenRouter)
73.3%
— —
196 hf.co/rockypod/svelte-coder-4b:Q4_K_M (Ollama) Model page ↗
72.2%
26.3 tok/s
~9.9s* 260 tok
197 nvidia/nemotron-3-super-120b-a12b:free (OpenRouter)
71.1%
— —
198 qwen/qwen3-coder-next (OpenRouter)
71.1%
— —
199 laguna-xs-2.1:q4_K_M (Ollama) Model page ↗
70.0%
32.5 tok/s
~3.9s* 127 tok
200 hf.co/rockypod/svelte-coder-8b:Q4_K_M (Ollama) Model page ↗
70.0%
14.8 tok/s
~15s* 226 tok
201 mistralai/mistral-small-2603 (OpenRouter)
68.9%
— —
202 gpt-5.1-codex-max (OpenAI)
67.8%
— —
203 north-mini-code-1.0:q4_K_M (Ollama) Model page ↗
66.7%
28.0 tok/s
~50s* 1,391 tok
204 moonshotai/kimi-linear-48b-a3b-instruct (OpenRouter)
66.7%
— —
205 inception/mercury-2.5 (OpenRouter)
65.6%
— —
206 cohere/north-mini-code:free (OpenRouter)
64.4%
— —
207 openai/gpt-5.4-nano (OpenRouter)
64.4%
— —
208 lfm2.5:24b-a2b-q4_K_M (Ollama) Model page ↗
63.3%
39.5 tok/s
~1m 26s* 3,402 tok
209 z-ai/glm-4.7-flash (OpenRouter)
61.1%
— —
210 JetBrains/mellum2-instruct-q4_k_m (Ollama) Model page ↗
58.9%
39.9 tok/s
~3.7s* 148 tok
211 nemotron-3.5-lightning:30b-a3b-q4_K_M (Ollama) Model page ↗
58.9%
26.5 tok/s
~1m 20s* 2,122 tok
212 hf.co/bartowski/Spark-X2.6-4B-GGUF:Q4_K_M (Ollama) Model page ↗
56.7%
23.4 tok/s
~2m 33s* 3,577 tok
213 amazon/nova-2-lite-v1:free (OpenRouter)
56.7%
— —
214 nvidia/nemotron-3.5-lightning (OpenRouter)
56.7%
— —
215 prism-ml/ternary-bonsai-2-27b (OpenRouter)
56.0%
— —
216 prime-intellect/intellect-3 (OpenRouter)
55.6%
— —
217 nemotron-3.5-lightning:30b-8k (Ollama) Model page ↗
51.1%
26.5 tok/s
—
218 arcee-ai/trinity-large-thinking (OpenRouter)
50.0%
— —
219 mistralai/devstral-2512:free (OpenRouter)
50.0%
— —
220 gpt-5-codex (OpenAI)
47.8%
— —
221 glm-4.7-flash:q4_K_M (Ollama) Model page ↗
45.6%
22.3 tok/s
~1m 2s* 1,382 tok
222 openai/gpt-5.1-codex (OpenRouter)
45.6%
— —
223 qwen/qwen3.5-9b (OpenRouter)
45.6%
— —
224 gemma4:e2b-it-qat (Ollama) Model page ↗
44.4%
38.1 tok/s
~22s* 854 tok
225 inception/mercury-2 (OpenRouter)
44.4%
— —
226 rnj-1:8b-instruct-q8_0 (Ollama) Model page ↗
37.8%
7.1 tok/s
~21s* 146 tok
227 qwen3.5:9b-q4_K_M (Ollama) Model page ↗
36.7%
13.0 tok/s
~1m 20s* 1,040 tok
228 bytedance-seed/seed-1.6-flash (OpenRouter)
35.6%
— —
229 openrouter/aurora-alpha (OpenRouter)
34.4%
— —
230 essentialai/rnj-1-instruct (OpenRouter)
30.0%
— —
231 ornith-1.5:9b-q4_K_M (Ollama) Model page ↗
25.6%
13.0 tok/s
~39s* 511 tok
232 gpt-oss:20b (Ollama) Model page ↗
24.4%
18.6 tok/s
~44s* 821 tok
233 gpt-5-mini (OpenAI)
24.4%
— —
234 hf.co/naren-1219/SvelteMind:F16 (Ollama) Model page ↗
23.3%
11.8 tok/s
~9.5s* 113 tok
235 arcee-ai/trinity-large-preview:free (OpenRouter)
23.3%
— —
236 lfm3:1.7b-q8_0 (Ollama)
22.2%
35.7 tok/s
~1m 49s* 3,904 tok
237 mistralai/ministral-3b-2512 (OpenRouter)
22.2%
— —
238 upstage/solar-pro-3:free (OpenRouter)
20.0%
— —
239 nvidia/nemotron-3-nano-30b-a3b:free (OpenRouter)
18.9%
— —
240 gemma4:e2b-it-q4_K_M (Ollama) Model page ↗
17.8%
36.7 tok/s
~22s* 814 tok
241 hf.co/naren-1219/SvelteMind:Q4_K_M (Ollama) Model page ↗
17.8%
34.9 tok/s
~2.4s* 82 tok
242 ibm-granite/granite-4.1-8b (OpenRouter)
16.7%
— —
243 openai/gpt-5.1-codex-mini (OpenRouter)
16.7%
— —
244 openrouter/elephant-alpha (OpenRouter)
15.6%
— —
245 granite4.2:30b-q4_K_M (Ollama) Model page ↗
14.4%
3.8 tok/s
~16m 15s* 3,677 tok
246 gpt-5-nano (OpenAI)
14.4%
— —
247 liquid/lfm-2-24b-a2b (OpenRouter)
13.3%
— —
248 twil-8b:q4_k_m-balanced (Ollama)
12.2%
12.8 tok/s
~2m 44s* 2,096 tok
249 twil-8b:q8_0-verbose (Ollama)
12.2%
7.5 tok/s
~10m 18s* 4,638 tok
250 lfm2:24b (Ollama) Model page ↗
11.1%
43.3 tok/s
~3.8s* 165 tok
251 liquid/lfm-2.5-2.6b:free (OpenRouter)
11.1%
— —
252 qwen/qwen3-vl-8b-instruct (OpenRouter)
11.1%
— —
253 deepcogito/cogito-v2-preview-llama-405b (OpenRouter)
10.0%
— —
254 afm-3-core (Apple)
8.9%
— —
255 barvhaim/granite-switch-4.1-3b-preview:latest (Ollama) Model page ↗
8.9%
4.8 tok/s
~30s* 147 tok
256 mistralai/ministral-8b-2512 (OpenRouter)
6.7%
— —
257 llama3.3:70b-instruct-q2_K (Ollama) Model page ↗
5.6%
2.7 tok/s
~57s* 152 tok
258 mistral-nemo:12b-instruct-2407-q4_K_M (Ollama) Model page ↗
4.4%
9.9 tok/s
~13s* 128 tok
259 mistralai/ministral-14b-2512 (OpenRouter)
3.3%
— —
260 allenai/olmo-3.1-32b-instruct (OpenRouter)
2.2%
— —
261 llama3.1:8b-instruct-q4_K_M (Ollama) Model page ↗
1.1%
15.1 tok/s
~8.1s* 122 tok
262 mimo-v2.6:9b-q4_K_M (Ollama)
0.0%
13.1 tok/s
~4m 51s* 3,810 tok
263 allenai/olmo-3.1-32b-think:free (OpenRouter)
0.0%
— —
264 qwen/qwen3.5-flash-02-23 (OpenRouter)
0.0%
— —

* Local models: Models in the local models tab were run through Ollama on an AMD Ryzen™ 5 7640HS processor (50W TDP) with Radeon™ 760M RDNA3 graphics. Speed is the generation rate measured on that hardware. The average response time is an average value based on one response sample per test.

Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 16
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 4/4 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 8/8 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 1/1 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 40% 100% 4/10 24
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 3/3 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 23
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 12
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 60% 100% 6/10 8
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 9/9 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 4
Prompt Tests
derived-by 70% 100% 7/10 9
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 80% 100% 8/10 4
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 80% 100% 8/10 2
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 13
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 4
Prompt Tests
derived 0% 0% 0/10 11
Prompt Tests
derived-by 0% 0% 0/10 13
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 10% 100% 1/10 11
Prompt Tests
hello-world 60% 100% 6/10 8
Prompt Tests
inspect 10% 100% 1/10 18
Prompt Tests
props 20% 100% 2/10 8
Prompt Tests
snippets 0% 0% 0/10 14
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 40% 100% 4/10 9
Prompt Tests
each 50% 100% 5/10 5
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 70% 100% 7/10 3
Prompt Tests
inspect 10% 100% 1/10 19
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 20
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 7
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 20% 100% 2/10 10
Prompt Tests
each 30% 100% 3/10 7
Prompt Tests
effect 40% 100% 4/10 7
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 50% 100% 5/10 11
Prompt Tests
props 40% 100% 4/10 8
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 9
Prompt Tests
derived 0% 0% 0/10 12
Prompt Tests
derived-by 0% 0% 0/10 12
Prompt Tests
each 50% 100% 5/10 6
Prompt Tests
effect 20% 100% 2/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 7
Prompt Tests
derived 0% 0% 0/10 13
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 10% 100% 1/10 9
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 70% 100% 7/10 6
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 90% 100% 9/10 2
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 7
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 4
Prompt Tests
derived 70% 100% 7/10 5
Prompt Tests
derived-by 50% 100% 5/10 13
Prompt Tests
each 80% 100% 8/10 2
Prompt Tests
effect 50% 100% 5/10 8
Prompt Tests
hello-world 70% 100% 7/10 5
Prompt Tests
inspect 70% 100% 7/10 9
Prompt Tests
props 40% 100% 4/10 9
Prompt Tests
snippets 0% 0% 0/10 14
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 50% 100% 5/10 9
Prompt Tests
derived-by 90% 100% 9/10 3
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 15
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 5
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 2
Prompt Tests
inspect 90% 100% 9/10 4
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 30
Prompt Tests
derived 60% 100% 6/10 6
Prompt Tests
derived-by 0% 0% 0/10 13
Prompt Tests
each 70% 100% 7/10 4
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 30% 100% 3/10 12
Prompt Tests
inspect 0% 0% 0/10 17
Prompt Tests
props 10% 100% 1/10 19
Prompt Tests
snippets 30% 100% 3/10 11
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 16
Prompt Tests
derived 20% 100% 2/10 10
Prompt Tests
derived-by 0% 0% 0/10 12
Prompt Tests
each 10% 100% 1/10 12
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 50% 100% 5/10 8
Prompt Tests
inspect 0% 0% 0/10 19
Prompt Tests
props 10% 100% 1/10 15
Prompt Tests
snippets 30% 100% 3/10 11
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 5
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 3
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 50% 100% 5/10 5
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 0% 0% 0/10 24
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 70% 100% 7/10 4
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 80% 100% 8/10 5
Prompt Tests
snippets 10% 100% 1/10 15
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 90% 100% 9/10 2
Prompt Tests
inspect 10% 100% 1/10 21
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 50% 100% 5/10 9
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 0% 0% 0/10 26
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 2
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 50% 100% 5/10 5
Prompt Tests
snippets 20% 100% 2/10 20
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 20
Prompt Tests
derived-by 0% 0% 0/10 30
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 17
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 25
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 14
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 70% 100% 7/10 3
Prompt Tests
derived-by 60% 100% 6/10 10
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 60% 100% 6/10 4
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 40% 100% 4/10 24
Prompt Tests
props 60% 100% 6/10 4
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 10
Prompt Tests
derived 20% 100% 2/10 12
Prompt Tests
derived-by 0% 0% 0/10 13
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 10% 100% 1/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 10% 100% 1/10 9
Prompt Tests
inspect 0% 0% 0/10 12
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 50% 100% 5/10 5
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 22
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 16
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 27
Prompt Tests
derived 0% 0% 0/10 11
Prompt Tests
derived-by 0% 0% 0/10 12
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 40% 100% 4/10 6
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 22
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 3
Prompt Tests
derived 60% 100% 6/10 4
Prompt Tests
derived-by 50% 100% 5/10 7
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 20% 100% 2/10 14
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 21
Prompt Tests
props 40% 100% 4/10 7
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 40% 100% 4/10 9
Prompt Tests
each 60% 100% 6/10 4
Prompt Tests
effect 40% 100% 4/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 20% 100% 2/10 25
Prompt Tests
props 80% 100% 8/10 5
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 3
Prompt Tests
derived-by 60% 100% 6/10 10
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 60% 100% 6/10 5
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 25
Prompt Tests
props 60% 100% 6/10 4
Prompt Tests
snippets 40% 100% 4/10 16
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 2
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 21
Prompt Tests
derived 10% 100% 1/10 9
Prompt Tests
derived-by 0% 0% 0/10 18
Prompt Tests
each 40% 100% 4/10 6
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 9
Prompt Tests
derived 50% 100% 5/10 6
Prompt Tests
derived-by 40% 100% 4/10 10
Prompt Tests
each 30% 100% 3/10 11
Prompt Tests
effect 20% 100% 2/10 13
Prompt Tests
hello-world 90% 100% 9/10 2
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 30% 100% 3/10 13
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 2
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 8
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 12
Prompt Tests
props 90% 100% 9/10 4
Prompt Tests
snippets 70% 100% 7/10 5
Prompt Tests
View model page ↗
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 30% 100% 3/10 7
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 10% 100% 1/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 10% 100% 1/10 9
Prompt Tests
snippets 0% 0% 0/10 30
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 12
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 17
Prompt Tests
effect 10% 100% 1/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 26
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 15
Prompt Tests
effect 0% 0% 0/10 12
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 15
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 10
Prompt Tests
derived 40% 100% 4/10 6
Prompt Tests
derived-by 30% 100% 3/10 7
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 20% 100% 2/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 40% 100% 4/10 6
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 50% 100% 5/10 5
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 50% 100% 5/10 5
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 20% 100% 2/10 8
Prompt Tests
derived 0% 0% 0/10 13
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 18
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 70% 100% 7/10 6
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 70% 100% 7/10 3
Prompt Tests
derived-by 40% 100% 4/10 8
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 16
Prompt Tests
props 60% 100% 6/10 4
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 30% 100% 3/10 13
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 11
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 10% 100% 1/10 9
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 70% 100% 7/10 3
Prompt Tests
each 70% 100% 7/10 4
Prompt Tests
effect 60% 100% 6/10 4
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 30% 100% 3/10 7
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 11
Prompt Tests
effect 60% 100% 6/10 4
Prompt Tests
hello-world 80% 100% 8/10 4
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 15
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 80% 100% 8/10 2
Prompt Tests
effect 30% 100% 3/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 3
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 20% 100% 2/10 8
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 60% 100% 6/10 4
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 30% 100% 3/10 7
Prompt Tests
effect 0% 0% 0/10 16
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 10% 100% 1/10 9
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 16
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 40% 100% 4/10 10
Prompt Tests
each 80% 100% 8/10 3
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 6/6 0
Prompt Tests
inspect 89% 100% 8/9 1
Prompt Tests
props 100% 100% 5/5 0
Prompt Tests
snippets 78% 100% 7/9 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 9
Prompt Tests
derived 50% 100% 5/10 7
Prompt Tests
derived-by 40% 100% 4/10 8
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 10% 100% 1/10 10
Prompt Tests
hello-world 60% 100% 6/10 6
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 0% 0% 0/10 13
Prompt Tests
snippets 20% 100% 2/10 14
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 30
Prompt Tests
derived 30% 100% 3/10 10
Prompt Tests
derived-by 10% 100% 1/10 13
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 15
Prompt Tests
derived-by 50% 100% 5/10 9
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 50% 100% 5/10 6
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 3
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 60% 100% 6/10 7
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 80% 100% 8/10 3
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 40% 100% 4/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 10% 100% 1/10 25
Prompt Tests
derived 20% 100% 2/10 11
Prompt Tests
derived-by 0% 0% 0/10 24
Prompt Tests
each 0% 0% 0/10 11
Prompt Tests
effect 0% 0% 0/10 12
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 24
Prompt Tests
derived 0% 0% 0/7 7
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 12
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 0% 0% 0/10 19
Prompt Tests
props 10% 100% 1/10 14
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 12
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 8
Prompt Tests
derived 80% 100% 8/10 4
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 80% 100% 8/10 3
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 9/9 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 70% 100% 7/10 6
Prompt Tests
snippets 40% 100% 4/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 14
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 90% 100% 9/10 4
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 40% 100% 4/10 6
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 15
Prompt Tests
props 0% 0% 0/10 22
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 60% 100% 6/10 5
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 25% 100% 1/4 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 60% 100% 6/10 4
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 0% 0% 0/10 19
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 80% 100% 8/10 3
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 70% 100% 7/10 3
Prompt Tests
each 30% 100% 3/10 7
Prompt Tests
effect 50% 100% 5/10 6
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 60% 100% 6/10 5
Prompt Tests
snippets 40% 100% 4/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 3
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 9/9 0
Prompt Tests
derived 100% 100% 9/9 0
Prompt Tests
derived-by 100% 100% 9/9 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 6/6 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 4
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 20
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 0% 0% 0/10 20
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 10% 100% 1/10 20
Prompt Tests
props 0% 0% 0/10 11
Prompt Tests
snippets 0% 0% 0/10 22
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 70% 100% 7/10 7
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 16
Prompt Tests
props 30% 100% 3/10 7
Prompt Tests
snippets 10% 100% 1/10 11
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 30% 100% 3/10 14
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 20% 100% 2/10 29
Prompt Tests
props 30% 100% 3/10 11
Prompt Tests
snippets 10% 100% 1/10 13
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 40% 100% 4/10 12
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 40% 100% 4/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 4
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 30% 100% 3/10 7
Prompt Tests
derived-by 10% 100% 1/10 9
Prompt Tests
each 70% 100% 7/10 3
Prompt Tests
effect 20% 100% 2/10 8
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 50% 100% 5/10 10
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 30% 100% 3/10 7
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 14
Prompt Tests
each 50% 100% 5/10 6
Prompt Tests
effect 20% 100% 2/10 12
Prompt Tests
hello-world 50% 100% 5/10 5
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 2
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 80% 100% 8/10 2
Prompt Tests
derived-by 80% 100% 8/10 6
Prompt Tests
each 0% 0% 0/10 13
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 13
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 16
Prompt Tests
derived-by 30% 100% 3/10 9
Prompt Tests
each 60% 100% 6/10 4
Prompt Tests
effect 20% 100% 2/10 9
Prompt Tests
hello-world 60% 100% 6/10 4
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 2
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 12
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 30% 100% 3/10 7
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 20% 100% 2/10 8
Prompt Tests
snippets 0% 0% 0/10 28
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 80% 100% 8/10 2
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 100% 100% 9/9 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 9/9 0
Prompt Tests
props 100% 100% 6/6 0
Prompt Tests
snippets 100% 100% 8/8 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 13
Prompt Tests
props 80% 100% 8/10 5
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 70% 100% 7/10 7
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 34
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 10% 100% 1/10 17
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 40% 100% 4/10 6
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 40% 100% 4/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 30
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 80% 100% 8/10 4
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 6
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 40% 100% 4/10 10
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 20% 100% 2/10 13
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 27
Prompt Tests
props 50% 100% 5/10 5
Prompt Tests
snippets 20% 100% 2/10 14
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 13
Prompt Tests
derived 60% 100% 6/10 4
Prompt Tests
derived-by 30% 100% 3/10 9
Prompt Tests
each 80% 100% 8/10 3
Prompt Tests
effect 80% 100% 8/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 44% 100% 4/9 14
Prompt Tests
props 50% 100% 5/10 5
Prompt Tests
snippets 0% 0% 0/9 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 10% 100% 1/10 9
Prompt Tests
effect 70% 100% 7/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 9
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 20% 100% 2/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 3
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 70% 100% 7/10 6
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 14
Prompt Tests
props 80% 100% 8/10 2
Prompt Tests
snippets 10% 100% 1/10 9
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 80% 100% 8/10 8
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 60% 100% 6/10 4
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 60% 100% 6/10 7
Prompt Tests
derived 50% 100% 5/10 6
Prompt Tests
derived-by 60% 100% 6/10 9
Prompt Tests
each 50% 100% 5/10 9
Prompt Tests
effect 40% 100% 4/10 8
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 30% 100% 3/10 13
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 0% 0% 0/10 10
Prompt Tests
derived 0% 0% 0/10 10
Prompt Tests
derived-by 0% 0% 0/10 10
Prompt Tests
each 0% 0% 0/10 10
Prompt Tests
effect 0% 0% 0/10 10
Prompt Tests
hello-world 0% 0% 0/10 10
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 10
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 6/6 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 70% 100% 7/10 9
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 70% 100% 7/10 3
Prompt Tests
snippets 30% 100% 3/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 30% 100% 3/10 15
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 4
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 90% 100% 9/10 1
Prompt Tests
effect 100% 100% 8/8 0
Prompt Tests
hello-world 100% 100% 5/5 0
Prompt Tests
inspect 62% 100% 5/8 3
Prompt Tests
props 100% 100% 5/5 0
Prompt Tests
snippets 0% 0% 0/2 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 88% 100% 7/8 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 13% 100% 1/8 13
Prompt Tests
props 100% 100% 8/8 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 4
Prompt Tests
hello-world 90% 100% 9/10 2
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 7/7 0
Prompt Tests
effect 100% 100% 5/5 0
Prompt Tests
hello-world 100% 100% 4/4 0
Prompt Tests
inspect 100% 100% 6/6 0
Prompt Tests
props 100% 100% 8/8 0
Prompt Tests
snippets 100% 100% 8/8 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 3
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 50% 100% 5/10 9
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 90% 100% 9/10 4
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 90% 100% 9/10 1
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 50% 100% 5/10 5
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 8/8 0
Prompt Tests
derived 100% 100% 8/8 0
Prompt Tests
derived-by 100% 100% 8/8 0
Prompt Tests
each 100% 100% 9/9 0
Prompt Tests
effect 100% 100% 7/7 0
Prompt Tests
hello-world 100% 100% 8/8 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 9/9 0
Prompt Tests
snippets 75% 100% 6/8 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 9/9 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 2/2 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 2
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 5/5 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 4/4 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 40% 100% 4/10 7
Prompt Tests
derived 0% 0% 0/10 12
Prompt Tests
derived-by 40% 100% 4/10 6
Prompt Tests
each 20% 100% 2/10 8
Prompt Tests
effect 0% 0% 0/10 11
Prompt Tests
hello-world 80% 100% 8/10 2
Prompt Tests
inspect 0% 0% 0/10 10
Prompt Tests
props 0% 0% 0/10 10
Prompt Tests
snippets 0% 0% 0/10 12
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 7
Prompt Tests
props 90% 100% 9/10 1
Prompt Tests
snippets 10% 100% 1/10 17
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 60% 100% 6/10 7
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 0% 0% 0/10 31
Prompt Tests
props 90% 100% 9/10 2
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 50% 100% 5/10 5
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 7
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 70% 100% 7/10 3
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 60% 100% 6/10 6
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 90% 100% 9/10 1
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 40% 100% 4/10 12
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 50% 100% 5/10 5
Prompt Tests
each 60% 100% 6/10 4
Prompt Tests
effect 40% 100% 4/10 10
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 7
Prompt Tests
props 40% 100% 4/10 10
Prompt Tests
snippets 10% 100% 1/10 13
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 70% 100% 7/10 3
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 40% 100% 4/10 6
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 20% 100% 2/10 8
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 90% 100% 9/10 1
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 0% 0% 0/10 16
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 10% 100% 1/10 9
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 4
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 90% 100% 9/10 1
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 60% 100% 6/10 4
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 100% 100% 10/10 0
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 90% 100% 9/10 1
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 80% 100% 8/10 3
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 10
Prompt Tests
props 90% 100% 9/10 2
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 20% 100% 2/10 8
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 80% 100% 8/10 2
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 90% 100% 9/10 1
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 30% 100% 3/10 7
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 100% 100% 10/10 0
Prompt Tests
Test pass@1 pass@10 Passing Samples Errors Actions
counter 100% 100% 10/10 0
Prompt Tests
derived 100% 100% 10/10 0
Prompt Tests
derived-by 100% 100% 10/10 0
Prompt Tests
each 100% 100% 10/10 0
Prompt Tests
effect 100% 100% 10/10 0
Prompt Tests
hello-world 100% 100% 10/10 0
Prompt Tests
inspect 80% 100% 8/10 2
Prompt Tests
props 100% 100% 10/10 0
Prompt Tests
snippets 90% 100% 9/10 1
Prompt Tests