Repository navigation
Commit 22ce5a8
Report the per-trial diagnostics a rank sweep needs
Closes #548.
`bicv` returned `Rank / Repeat / Metric / R2X`, which is enough to plot a curve
but not enough to audit the rank it implies. BiCV rows now also carry:
Train Block R2X in-sample R2X on the block the model was fit to
NTrainGenes, NTestGenes, NTrainCells, NTestCells
Seed the seed determining that trial's splits and init
`Train Block R2X` is the one that changes what the output can be used for. The
useful reading of a BiCV curve is the held-out score turning over while the
in-sample score keeps climbing, and the existing "Fit R2X" metric cannot supply
that comparison: it is a separate fit, on the full dataset rather than on the
training block, so it answers a different question. The value was already being
computed and discarded.
`_bicv_trial` now takes an explicit `seed` and builds its own generator instead
of drawing from a shared one, which is what makes `Seed` meaningful -- a single
trial can be replayed from the value in the table. A test asserts that replay
reproduces the recorded scores exactly.
The block sizes are reported rather than assumed because the cell split is
stratified per condition, so the realised fraction is only approximately the
requested one, and the spread across repeats has to be read against the block
each repeat actually used.
These columns are NaN on "Fit R2X" rows, which come from an unsplit fit.
The additions are columns rather than a new `Metric` value, so
`plot_bicv_r2x` -- which colours by `Metric` -- is unchanged. The
`test_bicv_shape_and_range` column assertion is relaxed from an exact set to a
subset, since the return shape is intentionally wider.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 0de3096 commit 22ce5a8
3 files changed
Lines changed: 170 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
113 | 131 | | |
114 | 132 | | |
115 | 133 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
| 109 | + | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
115 | | - | |
116 | | - | |
| 115 | + | |
| 116 | + | |
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
131 | 141 | | |
| 142 | + | |
132 | 143 | | |
133 | 144 | | |
134 | 145 | | |
| |||
138 | 149 | | |
139 | 150 | | |
140 | 151 | | |
141 | | - | |
| 152 | + | |
142 | 153 | | |
143 | 154 | | |
144 | 155 | | |
| |||
181 | 192 | | |
182 | 193 | | |
183 | 194 | | |
184 | | - | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
185 | 204 | | |
186 | 205 | | |
187 | 206 | | |
| |||
250 | 269 | | |
251 | 270 | | |
252 | 271 | | |
253 | | - | |
254 | | - | |
| 272 | + | |
| 273 | + | |
255 | 274 | | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
256 | 292 | | |
257 | 293 | | |
258 | 294 | | |
| |||
301 | 337 | | |
302 | 338 | | |
303 | 339 | | |
| 340 | + | |
| 341 | + | |
304 | 342 | | |
305 | 343 | | |
306 | 344 | | |
307 | | - | |
| 345 | + | |
| 346 | + | |
308 | 347 | | |
309 | 348 | | |
310 | 349 | | |
311 | 350 | | |
312 | | - | |
| 351 | + | |
313 | 352 | | |
314 | 353 | | |
315 | 354 | | |
316 | 355 | | |
317 | 356 | | |
318 | 357 | | |
319 | 358 | | |
320 | | - | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
321 | 366 | | |
322 | 367 | | |
323 | 368 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
| 48 | + | |
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| |||
268 | 268 | | |
269 | 269 | | |
270 | 270 | | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
0 commit comments