Skip to content

fix: include mvec for compilation - #120

Open
botamochi0x12 wants to merge 1 commit into
westonplatter:masterfrom
botamochi0x12:bugfix/including-mvec-for-compilation
Open

fix: include mvec for compilation#120
botamochi0x12 wants to merge 1 commit into
westonplatter:masterfrom
botamochi0x12:bugfix/including-mvec-for-compilation

Conversation

@botamochi0x12

@botamochi0x12 botamochi0x12 commented Aug 5, 2026

Copy link
Copy Markdown

Abstract

This PR fixes a symbol lookup error: ... undefined symbol: _ZGVbN2v_cos
(see #90 and #101 ) that occurs at
runtime on Linux systems whose GCC/glibc auto-vectorize cos() into a libmvec
SIMD variant. extconf.rb only linked -lm, not -lmvec, so the symbol was
left unresolved until the code path that calls it actually ran.

Background

The bundled pHash-0.9.6/configure.ac always builds with -O3 -ffast-math
unless --enable-debug is passed (ref: pHash-0.9.6/configure.ac
), which lets GCC substitute cos() calls with glibc's SIMD variant
_ZGVbN2v_cos from libmvec (ref: vector math ABI
). extconf.rb's $LIBS only listed -lm, so that symbol stayed unresolved
until code that calls it actually ran.

Reproducing the crash turned out to be inconsistent by build path:
Debian/Ubuntu's libm.so is itself a linker script that sometimes pulls in
libmvec automatically via AS_NEEDED, which is why local
bundle exec rake compile builds don't always show the crash. Installing
the actual published gem (gem install phashion, i.e. what every real user
does) reproduces it reliably. See the Appendix for the experiments that
pinned this down.

Contribution

Add -lmvec to $LIBS in ext/phashion_ext/extconf.rb so the built
extension carries an explicit NEEDED entry for libmvec.so.1 and the
symbol resolves at load time regardless of build path or distro linker-script
behavior.

Before

$ gem build phashion.gemspec -o /tmp/phashion-broken.gem
[[...Trimmed...]]
  Version: 1.2.0
  File: phashion-broken.gem
$ gem install /tmp/phashion-broken.gem
Building native extensions. This could take a while...
Successfully installed phashion-1.2.0
[[...Trimmed...]]
irb(main):001> require "phashion"
=> true
irb(main):002> broccoli = Phashion::Image.new("test/gif/Broccoli_Super_Food.gif")
=> #<Phashion::Image:0x00007dab40c20060 @filename="test/gif/Broccoli_Super_Food.gif">
irb(main):003> broccoli.duplicate? broccoli
irb: symbol lookup error: /usr/local/rvm/gems/ruby-3.4.7/gems/phashion-1.2.0/lib/phashion_ext.so: undefined symbol: _ZGVbN2v_cos
$ gem uninstall phashion
Successfully uninstalled phashion-1.2.0

After

$ gem build phashion.gemspec -o /tmp/phashion-fixed.gem
[[...Trimmed...]]
  Version: 1.2.0
  File: phashion-fixed.gem
$ gem install /tmp/phashion-fixed.gem
Building native extensions. This could take a while...
Successfully installed phashion-1.2.0
[[...Trimmed...]]
irb(main):001> require "phashion"
=> true
irb(main):002> broccoli = Phashion::Image.new("test/gif/Broccoli_Super_Food.gif")
=> #<Phashion::Image:0x00007417740d7410 @filename="test/gif/Broccoli_Super_Food.gif">
irb(main):003> broccoli.duplicate? broccoli
=> true
$ gem uninstall phashion
Successfully uninstalled phashion-1.2.0

Caveats

This PR may conflict with #105.

Environment & Appendix

Environment

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.4 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
$ gcc --version | head -1
gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
$ ld --version | head -1
GNU ld (GNU Binutils for Ubuntu) 2.42
$ ruby -v
ruby 3.4.7 (2025-10-08 revision 7a5688e2a2) +PRISM [x86_64-linux]

Appendix

macOS (Darwin) should not trigger this error

_ZGVbN2v_cos is glibc's vector-math ABI: glibc's <math.h> annotates
functions like cos() so that GCC's auto-vectorizer, under -O3 -ffast-math, can substitute calls to the SIMD variant implemented in
libmvec. This mechanism is specific to GNU/Linux:

  • macOS ships Apple's libSystem/libm, not glibc, and has no libmvec or
    glibc-style bits/math-vector.h declarations for cos() to hook into.
  • The default macOS compiler is Apple Clang (even the gcc binary on PATH
    is normally a Clang shim), which doesn't perform this substitution.
    extconf.rb doesn't override CC/CXX on Darwin, so the default Clang
    toolchain is what actually builds the extension.
  • Even a real Homebrew-installed GNU GCC on macOS still targets Darwin's
    system libm, not glibc, so it has nothing to vectorize into.

We have not reproduced this on real macOS hardware to confirm empirically —
this conclusion is based on the above reasoning about the toolchain, not a
direct test. Flagging as an assumption rather than a verified fact.

Experiments for completeness (Docker)

Verified in disposable containers, discarded after each run:

  1. Debian 12 (ruby:3.4-bookworm, GCC 12.2.0, glibc 2.36, binutils 2.40).
    Built from a pristine git archive HEAD checkout (pre-fix, -lm only)
    via both bundle exec rake compile and a bare ruby extconf.rb && make.
    Neither crashed — readelf -d showed libmvec.so.1 pulled in anyway via
    the AS_NEEDED clause in /usr/lib/x86_64-linux-gnu/libm.so.

  2. Ubuntu 24.04 (ubuntu:24.04 + distro-packaged ruby-full 3.2.3, GCC
    13.3.0, binutils 2.42).
    Same pristine pre-fix checkout, same two build
    paths. Again did not crash for the same AS_NEEDED reason.

  3. Ubuntu 24.04 + RVM Ruby 3.4.7 (matches the reporter's environment
    exactly, GCC 13.3.0, binutils 2.42).
    Pristine pre-fix checkout built via
    bundle exec rake compile or ruby extconf.rb && make — still did not
    crash. But installing the published gem the normal way:

    $ gem install phashion
    $ ruby -e 'require "phashion"; i = Phashion::Image.new("test/gif/Broccoli_Super_Food.gif"); i.duplicate?(i)'
    ruby: symbol lookup error: .../phashion-1.2.0/lib/phashion_ext.so: undefined symbol: _ZGVbN2v_cos

    reproduced the crash reliably, every time. readelf -d on that
    gem install-built .so shows no libmvec.so.1 NEEDED entry at all —
    unlike the rake-compiler-built copies in the same container.

    Rebuilding as a .gem from the fixed source and installing that
    (gem build phashion.gemspec && gem install ./phashion-1.2.0.gem) adds an
    explicit libmvec.so.1 NEEDED entry and the same script now returns
    true without crashing.

Takeaway: bundle exec rake compile (what this project's own test suite
and CI use) is not a reliable way to catch this class of bug — only building
through RubyGems' own extension builder (gem install /
gem build && gem install) reproduces it consistently. The explicit
-lmvec fix removes the dependency on the implicit, build-path-sensitive
AS_NEEDED behavior entirely, so it holds regardless of which path is used
to build the extension.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@botamochi0x12

Copy link
Copy Markdown
Author

NOTE: Not a duplicate of ryoga-chan's fork. Diffed ryoga-chan/phashion's extconf.rb against this repo — their $LIBS is still -lpthread -lpHash_gem -lstdc++ -ljpeg -lpng -lm, no -lmvec. Their fork addresses unrelated problems: PNG alpha-channel support (patches/png_alpha.diff), newer-architecture/ARM64 support via updated config.guess/ config.sub, and a newer bundled CImg.h. None of that touches linking, so this -lmvec fix doesn't overlap with their work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant