Skip to content

fix: decode 8-bit string fields as latin-1 instead of ascii - #456

Open
johansolve wants to merge 1 commit into
canboat:masterfrom
johansolve:fix/latin1-string-fields
Open

fix: decode 8-bit string fields as latin-1 instead of ascii#456
johansolve wants to merge 1 commit into
canboat:masterfrom
johansolve:fix/latin1-string-fields

Conversation

@johansolve

@johansolve johansolve commented Aug 3, 2026

Copy link
Copy Markdown

The problem

Node's 'ascii' encoding masks off bit 7 rather than validating its input, so any byte at 0x80 or above silently decodes to a different character.

A B&G Vulcan 7 sends PGN 129285 route names as STRING_LAU with control byte 1 and latin-1 characters. The Swedish route name Hättan-Askim arrives on the bus as:

48 e4 74 74 61 6e 2d 41 73 6b 69 6d 00
H  ä  t  t  a  n  -  A  s  k  i  m

and canboatjs decodes it as Hdttan-Askim0xe4 becomes 0x64. The plotter itself displays the name correctly, so the corruption happens on our side. Downstream this reaches Signal K as navigation.courseRhumbline.activeRoute.name, which then no longer matches the same route stored in the resources API, where the name is intact because it took an HTTP path instead of an N2K one.

Buffer.from([0x48, 0xe4, 0x74, 0x74, 0x61, 0x6e]).toString('ascii')   // 'Hdttan'
Buffer.from([0x48, 0xe4, 0x74, 0x74, 0x61, 0x6e]).toString('latin1')  // 'Hättan'

The change

'ascii''latin1' in the three readers that decode 8-bit string fields: STRING_LAU, STRING_FIX, and String with start/stop byte.

This is backwards compatible where it counts. For genuine 7-bit ASCII the two encodings produce byte-identical output, so no currently correct decode can break. Above 0x7f, latin-1 at least preserves the byte value — a consumer that knows better can reinterpret with Buffer.from(s, 'latin1') — whereas 'ascii' destroys it irrecoverably.

It follows the same reasoning as 82d12b2, which moved STRING_LZ off 'ascii' for this class of problem.

toPgn already writes these fields with charCodeAt, i.e. latin-1, so this removes a round-trip asymmetry for high bytes rather than introducing one.

Visible behaviour change: padding or junk bytes at 0x80 and above in STRING_FIX now render as U+0080–U+00FF instead of their bit-7-stripped equivalents. Both are garbage; the output differs.

I checked test/pgns/ for existing expectations containing non-ASCII characters before and after — there are none besides the one added here.

Test case

A real 129285 captured off the boat, including the full fast-packet reassembly, so the regression is pinned to a packet a shipping plotter actually sends.

It carries skipEncoderTest, and the reason is worth stating because it is not what it first looks like: the Vulcan NUL-terminates its STRING_LAU fields and counts the terminator in the length byte, while the encoder omits it. Re-encoding therefore yields three bytes fewer (0x0f vs 0x0e for the route name, and the same for each waypoint name). The reader handles both forms; changing the encoder would move the bytes under every other STRING_LAU encode test, so that is deliberately left alone. The 0x7fffffff lat/lon in the first waypoint, which is what I first assumed was the cause, round-trips exactly.

Known follow-up, deliberately not in this PR

STRING_LZ is now the one remaining inconsistency: it is read as 'utf-8' but written with charCodeAt, so 'ä' encodes to 0xe4 and reads back as U+FFFD. That affects 21 fields across the PGN definitions. Since the UTF-8 read was a deliberate choice in #296, it seemed wrong to reverse it as a side effect of this fix — but the read and write sides do disagree, and it may be worth a look.

CI

The suite is red on master before this change: four failures around Simnet: Command AP NoDrift and Simnet: Command AP Change Course, which reproduce on an untouched c584d70 with the @canboat/ts-pgns version that ^1.11.9 currently resolves to. This branch goes from 188 passing / 4 failing to 189 passing / 4 failing — the same four. prettier --check lib test and eslint are both clean.

The new test was mutation-checked with clean builds: reverting the STRING_LAU line to 'ascii' makes it fail, restoring 'latin1' makes it pass. (Worth noting for anyone verifying locally: tsc -b is incremental and will not rebuild the file on a revert, which makes the fix look like it has no effect.)

Co-Authored by Claude Code Opus 5.

Summary by CodeRabbit

  • Bug Fixes

    • Improved decoding of Latin-1 text in navigation data, including route and waypoint names.
    • Preserved correct handling of null-terminated and fixed-length strings.
  • Tests

    • Added regression coverage for packets containing accented characters and navigation metadata.

Node's 'ascii' encoding masks off bit 7 rather than validating the input, so any
byte at 0x80 or above silently becomes a different character. A B&G Vulcan 7 sends
PGN 129285 route names as STRING_LAU with control byte 1 and latin-1 characters:
"Hättan-Askim" arrives as 48 e4 74 74 ... and decoded as 'Hdttan-Askim'. The
plotter shows the name correctly, so the corruption is ours.

latin-1 is backwards compatible for the range that matters: for genuine 7-bit
ASCII the two encodings produce identical output, so no correct decode can break.
Above 0x7f it at least preserves the byte value, which a consumer can reinterpret;
'ascii' destroys it. This follows the same reasoning as 82d12b2, which moved
STRING_LZ off 'ascii' for the same class of problem.

Applies to STRING_LAU, STRING_FIX and 'String with start/stop byte'. toPgn already
writes these fields with charCodeAt, i.e. latin-1, so this also makes the round
trip symmetric for high bytes rather than introducing an asymmetry.

Visible behaviour change: padding or junk bytes at 0x80 and above in STRING_FIX
now render as U+0080-U+00FF instead of their bit-7-stripped equivalents.

The test case is a real 129285 captured off the boat, including the fast-packet
reassembly, so the regression is pinned to a packet a shipping plotter actually
sends.
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c0c41984-bea8-475f-a1e9-ba539891f15c

📥 Commits

Reviewing files that changed from the base of the PR and between c584d70 and 3cd7961.

📒 Files selected for processing (2)
  • lib/fromPgn.ts
  • test/pgns/129285.js

📝 Walkthrough

Walkthrough

PGN string decoding now uses Latin-1 for four string formats. A PGN 129285 fixture verifies decoding of Latin-1 route and waypoint names, including Hättan-Askim.

Changes

Latin-1 decoding

Layer / File(s) Summary
Update PGN string decoding
lib/fromPgn.ts
STRING_LAU, start/stop-byte, length-prefixed, and fixed-length strings now decode with Latin-1.
Add PGN 129285 regression coverage
test/pgns/129285.js
The captured B&G Vulcan 7 message verifies Latin-1 names, NUL-terminated strings, navigation values, nullable fields, and waypoint coordinates.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: fix, test

Suggested reviewers: sbender9, dirkwa

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change from ASCII to Latin-1 decoding for 8-bit string fields.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@johansolve
johansolve marked this pull request as ready for review August 3, 2026 14:57
@johansolve

Copy link
Copy Markdown
Author

Please label as Fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant