Repository navigation
Commit 5eb310c
Correct the held-out slice scale, and hold out half of each mode
PARAFAC2 writes slice k as `P_k (B diag(a_k)) C^T` with `P_k`
orthonormal, so the reconstruction's norm is `||(B diag(a_k)) C^T||`
regardless of how many cells the slice holds: all of a slice's energy
sits in `a_k`. BiCV fit `a_k` against the training cells and then reused
it unchanged for the held-out cells, and since a slice's energy grows
like sqrt(n_k) that overstated the predicted block by
sqrt(n_train_k / n_test_k) -- a factor of two at the old default split.
The symptom was a metric that got worse as the model got better. On
noiseless, exactly-rank-4 data fit at the true rank, the training block
scored 0.9996 while the held-out block scored -0.034: the fit had
recovered the structure exactly and the score still called it worse than
predicting the mean. On the IBD cohort the curve was negative at every
rank and fell monotonically as rank rose, putting its argmax at rank 1
and firing the "rank at the edge of tested ranks" warning. The error was
purely one of magnitude -- a single global rescale of the prediction
recovered R^2 = 0.970.
`_holdout_scale` corrects each condition by sqrt(n_test_k / n_train_k),
which needs only the cell counts and is exact when the two halves of a
condition have comparable per-cell energy -- true here, since the split
is random within each condition. Three candidates were measured
(this one, re-fitting `a_k` per held-out slice from the train genes, and
unconstrained least-squares cell loadings); all three fixed the bug and
none scored above zero on pure noise, so the narrowest was taken.
Both held-out fractions now default to one half, following Owen and
Perry (https://arxiv.org/abs/0908.2062), who report that "in simulated
examples we find that a method leaving out half the rows and half the
columns performs well". That is also the one split at which the old
score happened to be unbiased, so it is the setting under which older
numbers remain comparable, and it recovers the true rank in every
simulated scenario tested -- which the correction alone did not at 0.2.
Measured after the change: noiseless rank-4 data at its own rank scores
0.97-0.99 at every fraction (was -0.034); the spread across fractions
0.1-0.7 falls from 3.75 to 0.02; argmax lands on the true rank for
(4, noise 0.3), (6, noise 0.3) and (4, noise 1.0); pure noise still
scores negative. Row-order invariance from the alignment fix underneath
is undisturbed, agreeing to 8.9e-16 between contiguous and fully
interleaved conditions.
The dense reference in the tests derives the same factor independently
from the counts, so it still pins the streaming algebra rather than the
old behaviour. The tutorial's "peaks" claim is softened to describe the
plateau the curve actually forms, since past the plateau neighbouring
ranks differ only in the fourth decimal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016VJRB1RMUAb1id11XZ6KWS1 parent 9c9108b commit 5eb310c
3 files changed
Lines changed: 174 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
| 95 | + | |
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| |||
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | | - | |
| 113 | + | |
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
11 | 19 | | |
12 | 20 | | |
13 | 21 | | |
| |||
88 | 96 | | |
89 | 97 | | |
90 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
91 | 127 | | |
92 | 128 | | |
93 | 129 | | |
| |||
231 | 267 | | |
232 | 268 | | |
233 | 269 | | |
234 | | - | |
| 270 | + | |
| 271 | + | |
235 | 272 | | |
| 273 | + | |
236 | 274 | | |
237 | 275 | | |
238 | 276 | | |
| |||
266 | 304 | | |
267 | 305 | | |
268 | 306 | | |
269 | | - | |
270 | | - | |
| 307 | + | |
| 308 | + | |
271 | 309 | | |
272 | 310 | | |
273 | 311 | | |
| |||
298 | 336 | | |
299 | 337 | | |
300 | 338 | | |
301 | | - | |
| 339 | + | |
302 | 340 | | |
303 | | - | |
304 | | - | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
305 | 347 | | |
306 | 348 | | |
307 | 349 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
325 | 326 | | |
326 | 327 | | |
327 | 328 | | |
328 | | - | |
329 | | - | |
| 329 | + | |
| 330 | + | |
330 | 331 | | |
331 | 332 | | |
332 | 333 | | |
| |||
388 | 389 | | |
389 | 390 | | |
390 | 391 | | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
391 | 396 | | |
392 | | - | |
| 397 | + | |
393 | 398 | | |
394 | 399 | | |
395 | 400 | | |
| |||
500 | 505 | | |
501 | 506 | | |
502 | 507 | | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
0 commit comments