Skip to content

feat: improve Unicode width calculation for emoji and CJK (v2)#576

Closed
kolkov wants to merge 1 commit intocharmbracelet:proposal-v2-expfrom
kolkov:fix/v2-emoji-unicode-width
Closed

feat: improve Unicode width calculation for emoji and CJK (v2)#576
kolkov wants to merge 1 commit intocharmbracelet:proposal-v2-expfrom
kolkov:fix/v2-emoji-unicode-width

Conversation

@kolkov
Copy link
Copy Markdown

@kolkov kolkov commented Oct 8, 2025

Port of the Unicode width improvements to v2 branch, addressing Korean character rendering issues reported in opencode project (anomalyco/opencode#2013).

Changes:

  • Add comprehensive Korean/Japanese character detection via checkAsianCharacter()

    • Korean Hangul (unicode.Hangul) + Jamo ranges
    • Japanese Hiragana & Katakana
    • Enclosed CJK Letters (0x3200-0x32FF)
  • Implement emoji-specific width calculation fallback using go-runewidth

    • Detect emoji ranges (Emoticons, Symbols, Dingbats, etc.)
    • Use runewidth for accurate emoji width when detected
    • ansi.StringWidth already handles CJK correctly
  • Add comprehensive Unicode width tests

    • Test emoji width calculation
    • Test CJK character detection
    • Test Korean/Japanese character identification

This should help resolve Korean character disappearing issues in terminal emulators like WezTerm and Ghostty.

Related: #563, anomalyco/opencode#2013

  • I have read CONTRIBUTING.md.
  • I have created a discussion that was approved by a maintainer (for new features).

Port of the Unicode width improvements to v2 branch, addressing Korean
character rendering issues reported in opencode project (anomalyco/opencode#2013).

Changes:
- Add comprehensive Korean/Japanese character detection via checkAsianCharacter()
  - Korean Hangul (unicode.Hangul) + Jamo ranges
  - Japanese Hiragana & Katakana
  - Enclosed CJK Letters (0x3200-0x32FF)

- Implement emoji-specific width calculation fallback using go-runewidth
  - Detect emoji ranges (Emoticons, Symbols, Dingbats, etc.)
  - Use runewidth for accurate emoji width when detected
  - ansi.StringWidth already handles CJK correctly

- Add comprehensive Unicode width tests
  - Test emoji width calculation
  - Test CJK character detection
  - Test Korean/Japanese character identification

This should help resolve Korean character disappearing issues in
terminal emulators like WezTerm and Ghostty.

Related: charmbracelet#563, anomalyco/opencode#2013

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kolkov kolkov requested a review from meowgorithm as a code owner October 8, 2025 23:17
@rluisr
Copy link
Copy Markdown

rluisr commented Oct 20, 2025

@meowgorithm could you review this PR?

@meowgorithm
Copy link
Copy Markdown
Member

Hi. Thanks for the contribution. All the tests in this PR already pass on v2-exp, and we found a lot code here to be incorrect (for example, the unicode ranges).

I'm going to close this one, but if there's something we're missing please let us know (ideally, with some reproducible code that illustrates the issue).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants