Skip to content

Add experimental Windows wheel build support#117

Open
zym1998year wants to merge 3 commits into
jmeyers314:releases/0.8from
zym1998year:windows-port
Open

Add experimental Windows wheel build support#117
zym1998year wants to merge 3 commits into
jmeyers314:releases/0.8from
zym1998year:windows-port

Conversation

@zym1998year

Copy link
Copy Markdown

Depends on GalSim PR GalSim-developers/GalSim#1359.

Draft PR against releases/0.8 (baseline 367a861, v0.8.0).
Branch on file: windows-port.

Status

  • Verified locally: pip install . -v (Ninja+MSVC), pip wheel,
    delvewheel repair, fresh-venv install with separately-built GalSim
    Windows wheel (no conda PATH), import batoid, LSST yaml load,
    raytrace, SellmeierMedium, ObscPolygon, Bicubic, and 176 / 204
    pytest cases passing
    .
  • Test failures (28) are all in tests/test_notebooks.py, all
    ModuleNotFoundError: No module named 'nbformat' — jupyter test
    infra not installed. Not Windows-specific.
  • Linux / macOS code paths are textually unchanged outside the
    member rename and the or->|| swap (which GCC/Clang accept
    identically). Existing CI should pass without changes; not re-run
    from this branch.

Dependency

batoid depends on GalSim. GalSim does not currently ship Windows
wheels; a companion PR adds Windows support to GalSim
(releases/2.8 branch, windows-msvc-port). Once that PR or a
subsequent GalSim release exposes a win_amd64 wheel on PyPI, this
PR's wheel becomes installable purely with pip install. Until then,
end users on Windows would need to install GalSim from the companion
branch.

This PR does not change install_requires=['galsim', ...]; it only
enables batoid itself to build.

Summary

Three small commits make batoid build, install, and pass its
non-notebook test suite on native Windows x64 with the MSVC
toolchain. The patches address one Windows-SDK macro collision, three
independent MSVC source-level issues, and two CMake / pybind11 build
quirks. Diff totals +40 / -11 across six files.

What this PR changes

# Commit Files Diff
1 Avoid Windows SDK macro collision in SellmeierMedium include/medium.h, src/medium.cpp +7 / -4
2 Make batoid C++ sources MSVC compatible src/batoid.cpp, src/obscuration.cpp, src/table.cpp +12 / -6
3 Make batoid CMake extension build portable on Windows setup.py +21 / -1

Branch tip: d28bc8a.

Per-commit detail (click to expand)

1. SellmeierMedium macro collision

The Windows SDK header ctype.h defines preprocessor macros named
_C1, _C2, _C3 (control-character classification masks).
Including Windows.h — which any Windows MSVC translation unit
eventually does — expands those macros over the matching
SellmeierMedium private members and the file no longer parses.

Rename to _sellC1 / _sellC2 / _sellC3 with a one-line comment
in the header explaining why. Updates the constructor's
member-initializer list, getN(), and the BATOID_GPU device-pointer
constructor. Public API and Sellmeier coefficient ordering unchanged.

2. C++ sources for MSVC

Three independent fixes:

  • src/batoid.cpp: convert the two #pragma omp parallel for loops
    in applyForwardTransformArrays / applyReverseTransformArrays
    from size_t i to ptrdiff_t i (with a ptrdiff_t nloop = static_cast<ptrdiff_t>(n) mirror so the upper bound width is
    preserved). MSVC OpenMP requires a signed integral loop counter.
    Other parallel loops in this file already used int i and did not
    need touching. Adds <cstddef> for ptrdiff_t.
  • src/obscuration.cpp: replace or with || in the polygon
    point-in-polygon test and the y-grid bounds early-exit. MSVC parses
    or as an identifier without /Za plus <ciso646>.
  • src/table.cpp: replace or with || in the two bounds-check
    expressions in Table::eval and the gradient variant.

3. CMake extension build for Windows

setup.py only:

  • Drop build_args += ["--", "-j8"] and replace with
    build_args += ["--parallel", str(jobs)] driven by
    CMAKE_BUILD_PARALLEL_LEVEL, falling back to a bare --parallel.
    MSBuild rejects -j8 outright (it expects /m / -maxCpuCount);
    --parallel is generator-agnostic.
  • Add -Dpybind11_DIR= from pybind11.get_cmake_dir() to
    cmake_args, guarded by a try/except around import pybind11.
    PEP 517 isolated builds install pybind11 in a temporary directory
    that CMake's find_package(pybind11) cannot otherwise discover.

Verification

Built and tested on Windows 11 x64, Python 3.11.15 (conda-forge), VS
2026 Community (MSVC 14.50.35717), CMake 3.30, Ninja 1.13, pybind11
3.0.3.

Build / wheel

pip install . -v --no-build-isolation                 # 0 errors
pip wheel . -w dist -v --no-build-isolation --no-deps # 0 errors
delvewheel show dist\batoid-*-win_amd64.whl           # see table
delvewheel repair --add-path %CONDA_PREFIX%\Library\bin \
                  -w wheelhouse dist\batoid-*-win_amd64.whl
Bundled into wheel Assumed present on host
msvcp140.dll vcruntime140.dll, vcruntime140_1.dll
vcomp140.dll python311.dll, kernel32.dll, api-ms-win-crt-*

batoid does not link FFTW directly (the FFT dependency goes through
GalSim), so the batoid wheel does not bundle fftw3.dll.

Smoke (touches each patch)

Test Result Patch covered
import batoid 0.1.dev1069+gd28bc8a36 build / setup.py
SellmeierMedium(BK7).getN(550 nm) 1.5185223876207927 SellmeierMedium rename
Optic.fromYaml('LSST_r.yaml') 4 sub-items end-to-end
LSST_r raytrace, polar, theta=(0.5deg, 0) 132 rays in, 114 not vignetted/failed, ~2 um focal-plane spread OpenMP parallel-for
ObscPolygon([0,1,1,0],[0,0,1,1]).contains(0.5,0.5) / (2,2) True / False obscuration.cpp or
Bicubic of 0.01*(x*x+y*y), sag at (0.2, 0.3) and off-grid (5,5) 0.0013... / nan table.cpp or

Full pytest

:: Run from outside the source tree so pytest resolves the installed
:: package, not the source-layout `batoid/` subdir.
python -m pytest batoid/tests/ -o "addopts=--durations=10 -ra" --timeout=180 --rootdir=batoid

176 passed, 28 failed in 62 s. The 28 failures are all in
tests/test_notebooks.py, all ModuleNotFoundError: No module named 'nbformat' (jupyter test infra not installed; not Windows-specific).

The 176 passing tests cover Asphere, Bicubic, Coating, CoordSys,
CoordTransform, Lattice, Medium, OPDScreen, Obscuration, Optic,
Paraboloid, Plane, Quadric, RayVector, Sphere, Sum, Tilted, Zernike,
analysis, plot, zemax. The OpenMP parallel paths and both alt-token
sites are exercised.

Slowest test cases (top 3)
Test Time
tests/test_zemax.py::test_HSC_huygensPSF 11.55 s
tests/test_analysis.py::test_doubleZernike 2.64 s
tests/test_Asphere.py::test_sag 1.47 s

Fresh-venv install (no conda PATH)

python -m venv .venv-clean
:: PATH set to .venv-clean\Scripts;System32;... (no conda Library\bin)
.venv-clean\Scripts\pip install <galsim-2.8.4-cp311-...whl> LSSTDESC.Coord astropy
.venv-clean\Scripts\pip install <batoid-...-cp311-...whl> pyyaml scipy matplotlib
.venv-clean\Scripts\python batoid_smoke.py

The cleaned PATH does not include %CONDA_PREFIX%\Library\bin. The
six smoke tests above pass with byte-identical numeric output to the
conda-env run, demonstrating the wheel is self-contained modulo the
GalSim wheel's bundled FFTW.

Wheel notes

  • Bundled DLLs: msvcp140.dll, vcomp140.dll. No FFTW (uses
    GalSim's).
  • vcomp140.dll is the Microsoft OpenMP runtime, used by the loops
    in batoid.cpp / obscuration.cpp. Choosing /openmp:llvm would
    switch this to libomp.dll. This PR keeps the simpler /openmp
    (set via CMake's find_package(OpenMP)).
  • Wheel size: ~8.5 MB; ~6 MB of that is the bundled
    LSST/Rubin/HSC/DECam YAML data files, not C++ code.

Follow-ups

None blocking; each is a separate PR candidate.

  • GitHub Actions CI: a windows-latest job + cibuildwheel
    matrix (cp39 .. cp313 -win_amd64). Better aligned after the
    GalSim Windows wheel is upstream so batoid CI can pip install galsim directly.
  • Notebook tests: install nbformat, nbclient, ipyvolume in
    CI Windows so the 28 test_notebooks.py cases run.
  • Python_EXECUTABLE -> Python3_EXECUTABLE migration:
    setup.py still passes the older form. Works today; can be
    modernized in a separate PR.

Risk and compatibility

  • Linux / macOS: every change is either Windows-only logic
    guarded by _WIN32 / generator-aware (setup.py) or a textual
    swap that GCC/Clang accept identically (or -> ||,
    size_t -> ptrdiff_t, member rename).
  • OpenMP loop counter signed-vs-unsigned change: switching
    size_t i to ptrdiff_t i on a 64-bit platform preserves range
    for any practical batoid input.
  • Sellmeier rename: members are private; not part of the
    documented API.
  • MSVC toolset vs bundled runtime: delvewheel warns when the
    bundled msvcp140.dll is older than the toolset that built the
    .pyd. Locally I built with VS 2026 (14.50) and bundled
    conda-forge's 14.44 redistributable. Smoke and pytest both ran
    cleanly. CI on windows-latest (VS 2022, 14.4x) would not see
    the mismatch.

Reviewer questions

  1. GalSim Windows wheel timeline: should I plan around it, or is
    it acceptable to land this batoid PR ahead of the GalSim PR (with
    a note that pip install is gated until GalSim ships a Windows
    wheel)?
  2. /openmp vs /openmp:llvm for the eventual CI/wheel build.
  3. PYTHON_EXECUTABLE -> Python3_EXECUTABLE migration: fold
    into this PR, or separate?
  4. CI: a windows-latest + cibuildwheel workflow can follow
    once questions 1-2 are settled. Draft strategy available locally
    if helpful.

Happy to split, rebase, or extend as the project prefers.

The Windows SDK ``ctype.h`` defines preprocessor macros named ``_C1``,
``_C2``, ``_C3`` (control-character classification masks).  Including
``Windows.h`` -- which any Windows MSVC translation unit eventually
does -- expands those macros over the SellmeierMedium private members
of the same name and the file no longer compiles.

Rename the private members to ``_sellC1`` / ``_sellC2`` / ``_sellC3``
and update the header, the constructor's member-initializer list, the
getN() formula, and the GPU device-pointer constructor accordingly.
Public API and Sellmeier coefficient ordering are unchanged.
Three independent MSVC fixes in src/:

- src/batoid.cpp: MSVC's OpenMP requires a signed integral loop
  counter; convert the two ``#pragma omp parallel for`` loops in
  applyForwardTransformArrays / applyReverseTransformArrays from
  ``size_t i`` to ``ptrdiff_t i`` (with a ``ptrdiff_t nloop`` mirror
  of ``size_t n`` to avoid clipping).  Adds ``<cstddef>`` for the
  type.  Other parallel loops in this file already used ``int`` and
  did not need touching.

- src/obscuration.cpp: replace the C++ alternative tokens ``or`` in
  the polygon point-in-polygon and grid bounds checks with ``||``;
  MSVC parses ``or`` as an identifier without ``/Za`` + ``ciso646``.

- src/table.cpp: same alternative-token replacement on the two
  bounds-check expressions in ``Table::eval`` and the gradient
  variant.

No behavioural change on Linux/macOS; the ``size_t``/``ptrdiff_t``
swap is safe for the practical ray-vector sizes batoid handles.
Two adjustments to ``setup.py``'s ``CMakeBuild.build_extension``:

- Drop the unconditional ``-- -j8`` forwarded to the underlying build
  tool.  MSBuild rejects ``-j8`` outright (it expects ``/m`` /
  ``-maxCpuCount``); Make/Ninja accept it but only for historical
  reasons.  Hand parallelism to ``cmake --build --parallel`` instead,
  honouring ``CMAKE_BUILD_PARALLEL_LEVEL`` when set, so the same code
  path works for every CMake generator.

- Add ``-Dpybind11_DIR=`` from ``pybind11.get_cmake_dir()`` to the
  CMake configure args.  PEP 517 isolated builds install pybind11 in
  a temporary directory that CMake's ``find_package(pybind11)``
  cannot otherwise discover, so the configure step would fail; this
  hint makes ``pip install . -v`` and ``pip wheel .`` work
  end-to-end on Windows without manually exporting CMAKE_PREFIX_PATH.

The Linux/macOS path is unaffected: those generators already accept
``--parallel`` and pybind11 is normally on CMake's default search
path there.
@jmeyers314 jmeyers314 force-pushed the releases/0.8 branch 3 times, most recently from 44c3a87 to c606b79 Compare May 13, 2026 04:47
@jmeyers314

Copy link
Copy Markdown
Owner

Hi! Thanks for this.

Same basic comments as the GalSim PR:

  • Please rebase onto / target main
  • Please add a Windows CI build since I have no direct access to a Windows machine.

I'll take a closer look when that's ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants