Repository navigation
fix(dataset): Malay filenames byte-identical (#51): mv rename block, baru.txt slot, probe exact section - #67
Merged
Merged
Conversation
…probe Shipped v0.9.0 answers `nak delete lama.txt` with `rm old.txt` (issue #51): 2/8 exact on Malay-named files. The unshipped run 10/11 models score 7/8, so the head rows' Malay slot names already carry the fix; the one miss is the two-file rename, which basics_head.txt had no block for. - basics_head.txt: `mv {t} {t2}` block, `{t2}` slot (text list shifted by 7) - head.py: `{t2}` alongside `{f2}`/`{d2}`; test in test_pipeline.py - probe.py: EXACT section, printed as `#51 exact n/8`, leak-checked - RESULTS.md: numbers, falsified pool hypothesis, pool_v10 as next start pool_v10 built locally (pool_v9 with mv resampled, 236 rename rows in); no retrain, per the run 11 decision. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 12 (pool_v10, mv rename block) left #51 exact at 7/8: rename works on every name pair except baru.txt -> lama.txt, which falls into basics.txt's mv old.txt new.txt template because baru.txt appears in zero pool rows. pool_v11 swaps notes.txt for baru.txt in the text-file list, 220 rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ship Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #51 on the data side; not a model ship.
What
dataset/basics_head.txt:mv {t} {t2}file-rename block;{t2}slot inhead.py(text list shifted by 7, like{f2});baru.txtreplacesnotes.txtin the{t}list.training/probe.py:EXACTsection — eight whole-line prompts (nak delete lama.txt→rm lama.txt), printed as#51 exact n/8, covered by the basics leak check.RESULTS.md: "Issue Malay filenames in the question come back translated (lama.txt -> old.txt) #51" section, runs 12 and 13.Numbers
The issue's hypothesis (translated
lama.txt/old.txtpairs in the pool) is falsified — zero such rows; the mapping is the base prior, and head rows with Malay names on both sides beat it. The one prompt that survived wasbaru.txt, a name in zero pool rows.Why no ship
v11-dpo BM is below v0.9.0 with the CI clear of zero — the head-row family drifted 0.497 → 0.423 one inside-CI step at a time. Details in RESULTS.md. The data fix ships with the next model that clears ALFA; README workaround stays until then.
pool_v10 / pool_v11 rebuilt from pool_v7 with
head.py+lists.py; every non-mvrow byte-identical to pool_v9.🤖 Generated with Claude Code