Skip to content

Modernize the Windows MSI: WiX v5, x64, service registration, upgrade from 5.1.1 - #701

Open
vharseko wants to merge 45 commits into
OpenIdentityPlatform:masterfrom
vharseko:features/windows-distribution-new
Open

Modernize the Windows MSI: WiX v5, x64, service registration, upgrade from 5.1.1#701
vharseko wants to merge 45 commits into
OpenIdentityPlatform:masterfrom
vharseko:features/windows-distribution-new

Conversation

@vharseko

@vharseko vharseko commented Jul 4, 2026

Copy link
Copy Markdown
Member

Modernizes the Windows MSI end to end. Builds on #664 (branched from it) and includes the Windows script quoting fixes from #671 (they merge cleanly once #671 lands).

Toolchain: WiX 3.11 + wine → WiX v5, Windows-native

  • WiX 3.11.1 (2017, EOL) → WiX v5.0.2 (.NET wix tool): heat/candle/light + wine + winetricks + dotnet40 collapse into a single wix build -arch x64. WiX v5 is the newest version without the Open Source Maintenance Fee (v7 refuses to run without accepting the OSMF EULA).
  • The MSI is built on Windows only. Verified empirically (wix 5.0.2/6.0.1/7.0.0 on macOS): non-Windows builds fail on WIX0389 for any Directory/@Name and, with no names at all, on the missing msi.dll — the MSI database is written by the Windows Installer library, so no cross-platform build exists. wine is removed from build.yml, deploy.yml and release.yml.
  • package.wxs rewritten to the v4+ schema: single <Package>, MediaTemplate CompressionLevel="high", InstallerVersion=500, <Files> harvest from Ant-staged payloads, empty instance directories shipped via CreateFolder components (the harvest ships files only).
  • The opendj-msi module is pulled in by a distribution-windows-msi profile that activates only when the wix .NET tool is installed, so a plain mvn install on a contributor's Windows machine no longer fails mid-reactor.

Installer behavior

  • x64: installs into C:\Program Files\OpenDJ (was x86 / Program Files (x86)).
  • The MSI registers the OpenDJ Windows service (key = display name; ServiceInstall on opendj_service.exe, not started during install — run setup first; a failed registration fails the install). The display name deliberately equals the key so windows-service.bat --enableService in additional zip instances can still register OpenDJ Server/OpenDJ Server-2. When a legacy OpenDJ is detected (an upgrade from a pre-5.2.0 package — proven by a detect-only UpgradeVersion row — or a fresh install adopting the legacy x86 default directory on a host with no location yet recorded by this package) and the legacy service's own ImagePath proves it points into the directory being installed to, a best-effort immediate net stop runs before RemoveExistingProducts and a check action refuses the upgrade (error 1722) if the service is still running — a deferred elevated stop is impossible there (ICE63), an immediate one cannot elevate under a filtered UAC token, and proceeding would strand locked jars (the JVM opens them without FILE_SHARE_DELETE, JDK-8224794); MSI's own ServiceControl then stops and removes the service. New-MSI-to-new-MSI upgrades, repairs, and installs into any directory the service does not point at leave it alone. windows-service.bat --disableService, --cleanupService and uninstall.bat all treat the MSI-managed service as an informational skip (dedicated wrapper return code; the ownership check fails closed when the SCM cannot prove the service orphaned, and the control-panel dialog reports the skip as "Windows Service Left Enabled"), so msiexec /x always finds it; repair does not stop a running server (Stop="uninstall"). The MSI also registers the event-log source for the service.
  • Upgrade over an existing install: the installer detects the location recorded in the registry by this package, or the legacy x86 default directory (two separate searches with explicit precedence: registry wins), and installs there — instance data (config, db, logs) is preserved; an explicit OPENDJ=... always wins, and an upgrade whose location cannot be determined (5.1.x wrote no registry value) refuses with guidance instead of relocating to the default — in the wizard as well as silently (the guard runs in both sequences). Verified in CI by upgrading from the released 5.1.2 MSI at its real x86 default with no OPENDJ given. AllowSameVersionUpgrades keeps a same-version hotfix re-release from producing two ARP entries.
  • Java: the installer does not check for or install a JRE (a JAVA_HOME/PATH launch condition was tried and dropped — it blocked upgrades of working servers and never consulted OPENDJ_JAVA_HOME, see the review discussion and Add CI install-test for the Windows MSI + document MSI install/upgrade/uninstall #664). setup and the server require Java 11+; the install guide points at https://adoptium.net. The MSI is not code-signed (no certificate) — SmartScreen note added to the docs.

Release / deploy

  • deploy.yml reuses the MSI already built by the triggering Build run (no rebuild).
  • release.yml: new release-msi job (windows, continue-on-error) installs the released server zip into the local repository and packages only :opendj-msi-standard, then attaches the MSI to the GitHub release.
  • opendj-msi-standard is intentionally no longer published to Maven Central: it was previously wine-built on Linux and deployed from the ubuntu deploy jobs; it is now built natively on Windows and distributed through GitHub Releases and the Package/Deploy OpenDJ MSI Package artifact. The Maven coordinate stays at 5.1.2 as its last published version. This also applies to sustaining/4.10.x: Package/Deploy always runs master's workflow, which no longer installs wine, while the 4.10.x pom still builds its MSI through it — accepted; the 4.10.x MSI remains available up to its last published release.

CI

  • test-msi: installs into the spaced x64 default directory, asserts the service ImagePath is quoted (CWE-428), runs setup, starts/stops the MSI-registered service, ldapsearch, asserts --disableService leaves the MSI-managed service in place, runs uninstall.bat --cli (must delete the instance files and leave the service to msiexec /x), uninstalls.
  • test-msi-upgrade: released 5.1.2 x86 MSI at its native Program Files (x86)\OpenDJ default → configure → legacy service left running → upgrade with the new x64 MSI with no OPENDJ given: the installer itself stops the running legacy service (the stop+check pair, elevated here), same directory kept, data intact, legacy service replaced by OpenDJ, upgrade.bat, start, search; a repair (REINSTALL=ALL) must leave an OpenDJ Server registered by a zip instance after the upgrade alone; plus fresh-install scenarios asserting the legacy default directory is auto-detected, the registry-recorded location wins over it, a silent upgrade from an undetectable custom directory refuses with guidance, and an install into a different directory leaves a zip instance's OpenDJ Server untouched despite a leftover legacy directory.
  • The windows/11 job validates the built package with wix msi validate (ICE): wix build itself runs no ICE validation, so a green build alone would not catch sequencing mistakes like ICE63.
  • The Windows job compares the committed launcher binaries under opendj-server-legacy/lib with what it just built from source (/Brepro makes the build reproducible; git status --porcelain, so a never-committed new launcher is caught too) and emits a loud warning on drift — warning-only, so an MSVC toolchain bump on the runner image cannot take down the artifact consumers (test-msi*, deploy.yml); nmake/xcopy failures now fail the step. The failing signal is the standalone Native launchers workflow: it triggers only when the native sources or the committed binaries change (exactly the stale-binaries case) and gates nothing, so it cannot cascade into the Build artifacts or deploy.

Windows script fixes surfaced by these tests (in #671)

Install paths with spaces/parentheses (C:\Program Files (x86)\OpenDJ — the old default!) broke the batch scripts in four places: unquoted java.io.tmpdir, setcp.bat argument parsing, "%VAR%" == "" checks with the now-quoted tmpdir, and the start-ds.bat tmp-cleanup block. All fixed and exercised by the MSI tests in this PR.

Deferred

  • Authenticode signing — no certificate.
  • A Burn bundle auto-installing the JRE — dropped; the installer neither bundles nor checks for a JRE (see above), the docs carry the download link.

The launcher binaries under opendj-server-legacy/lib are pending a refresh from the windows-exe-11 artifact of the next Build run (service.c changed again in the fifth review round — SCM-enumeration failures now report an error instead of "already disabled", and REG_EXPAND_SZ InstallDir values are expanded); until then the binaries-match-sources warning fires and the Native launchers check is red by design. The GUI wizard path (license-dialog skip, InstallDirDlg pre-population, the custom-directory upgrade refusal dialog) is not covered by CI — one manual run before merge is planned.

vharseko added 22 commits June 26, 2026 20:37
The opendj-msi package was built and uploaded but never installed or
exercised in CI. Add a test-msi job (needs: build-maven) that, on a
windows-latest runner, installs the built .msi silently (msiexec /i),
runs setup, registers and starts/stops the OpenDJ Windows service with
an ldapsearch liveness check, then uninstalls (msiexec /x).
…E 25)

The MSI ships no JRE, so add a WiX launch condition that fails the install
early with a clear message when Java is not detected (it does not install
Java). JAVA_HOME is captured from the environment before LaunchConditions;
`Installed` keeps uninstall/repair working regardless of Java.

Fix the test-msi CI job (it failed with `net start` exit 2 "service already
started"):
- setup.bat is now invoked with --doNotStart, so the server is started only by
  `net start "OpenDJ Server"` (setup.bat no longer starts a standalone instance
  first).
- Bump actions/setup-java from 21 to 25 (latest LTS; smoke-tests the MSI under a
  fresh JRE). The runner has JAVA_HOME from setup-java, so the new launch
  condition is satisfied and the install proceeds.
The launch condition (Installed OR JAVA_HOME_ENV) false-blocked valid installs: a JRE does not always set JAVA_HOME (it may be only on PATH). Drop it - the MSI again only copies files and Java availability stays the admin's responsibility. The test-msi fix (--doNotStart, JRE 25) is unaffected.
The install guide covered only the .zip and native .deb/.rpm. Add Windows MSI sections to chap-install/chap-upgrade/chap-uninstall: GUI and silent msiexec install, Java as a runtime prerequisite the installer does not enforce, configure via setup.bat, optional Windows service registration via windows-service.bat, MSI upgrade (disable service, back up, install newer .msi, upgrade.bat, re-enable), and uninstall via Apps & features / msiexec /x.
Migrate the Windows MSI off the EOL WiX 3.11.1 (heat/candle/light run under
wine + winetricks dotnet40) to the cross-platform WiX v5 .NET tool, and
address the MSI validation findings:

- package.wxs rewritten to the WiX v4+ schema: single <Package>, x64
  (ProgramFiles64Folder), MediaTemplate CompressionLevel="high",
  InstallerVersion=500, <Files> directory harvest (replaces heat), and a
  <ServiceInstall>/<ServiceControl> on opendj_service.exe so the MSI itself
  registers the "OpenDJ Server" Windows service (Start=auto, not started during
  install - setup must configure the instance first). UpgradeCode preserved.
- opendj-msi-standard/pom.xml builds the MSI with `wix build -arch x64`; removed
  the external-dependency-plugin (wix3111 + winetricks), heat/candle/light and
  the wine/winetricks antrun steps.
- opendj-msi/pom.xml: wine-path profiles replaced by an unconditional module so
  the MSI builds natively on every OS where opendj-msi is included.
- build.yml: drop wine (keep rpm); add a "Setup WiX (.NET tool)" step; test-msi
  no longer calls windows-service.bat --enableService (the MSI registers it).
- install guide: x64 install path; the MSI registers the service.

Signing (OpenIdentityPlatform#4) deferred (no certificate). The Burn bundle that auto-installs a JRE
is a follow-up, to be added after this MSI build is validated in CI.
WiX 5 ships as a net6.0 .NET tool, but the runners only have a newer .NET runtime (no .NET 6), so 'wix' failed to launch (exit 131, missing_runtime) on arm64 macOS. Set DOTNET_ROLL_FORWARD=Major in the Setup WiX step (exported and written to GITHUB_ENV) so the tool rolls forward to the installed runtime, both for that step and the Maven build that invokes 'wix build'.
wix build failed with WIX0004 ('Files element contains an unexpected attribute Exclude'): in WiX 5.0.2 exclusion is a child <Exclude Path=.../> element (with <Include Path=.../>), not an inline attribute. Convert both <Files> harvests to the child-element form.

macOS (arm64) runners have no .NET runtime for the net6.0 wix apphost (DOTNET_ROOT unset, exit 131); add actions/setup-dotnet (8.0.x) on macOS so the tool finds a runtime and rolls forward.
…only Include)

WiX 5.0.2 requires Include as an attribute on <Files> and rejects an Exclude attribute as well as <Include>/<Exclude Path> child elements (WIX0004/0005/0010). Move all exclusions to the Ant staging step: stagingRoot = payload minus lib (and macOS/Unix bits), stagingLib = lib minus opendj_service.exe (registered via an explicit component sourced from the package dir). Each <Files> now uses only the Include attribute, and there is no duplicate lib directory.
On Linux/macOS the wix tool treats backslash as a literal, not a path separator, which may cascade into the WIX0389 'not a relative path' errors. Use '/**' (accepted on Windows too) to test whether the cross-platform wix build then succeeds.
The WiX Toolset can create MSI databases on Windows only: it warns "only
supports Windows" and, on Linux/macOS, fails with WIX0389 on every Directory
name and (with no name) on a missing msi.dll (the Windows Installer library).
Verified locally on macOS with WiX 5.0.2, 6.0.1 and 7.0.0 - none can build.

- opendj-packages/pom.xml: build opendj-msi only in the distribution-windows
  profile (removed from distribution-unix and distribution-mac).
- build.yml: run the WiX setup step only on Windows; drop macOS setup-dotnet.
- package.wxs: keep the Windows-proven backslash glob in <Files Include>.
- .gitattributes: force LF for *.wxs/*.wxi.
The MSI can only be built on Windows. Instead of rebuilding opendj-server-legacy
on a Windows runner:

- deploy.yml: reuse the MSI already built by the triggering Build run (download
  the windows-latest-11 artifact, re-upload as "OpenDJ MSI Package"); both steps
  continue-on-error; drop wine (rpm only).
- release.yml: release-maven uploads the released server zip as an artifact;
  new release-msi job (windows, continue-on-error so an MSI failure does not
  break the release) installs the zip into the local Maven repository and only
  packages :opendj-msi-standard (no -am), then attaches the MSI to the GitHub
  release; drop the .msi from the ubuntu release file list; drop wine.
…e legacy service

- package.wxs: refuse to install when no Java is detectable (JAVA_HOME\bin\java.exe
  or java-looking PATH entries; MSI cannot scan PATH, so substring heuristic) with
  a message pointing at https://adoptium.net; detect an existing install (InstallDir
  registry value written by this package, else the legacy x86 default directory)
  and apply it only when OPENDJ is not set explicitly; drop the legacy
  windows-service.bat "OpenDJ Server" service before InstallServices (its display
  name collides with the MSI ServiceInstall).
- build.yml: test-msi asserts the installer fails without a JRE before one is set
  up; new test-win-upgrade installs the released 5.1.1 x86 MSI, configures an
  instance with the legacy service, upgrades with the newly built x64 MSI without
  OPENDJ and asserts same directory, intact data, MSI-managed service, then runs
  upgrade.bat, starts the service and searches the pre-upgrade data.
- docs: Java is required by the installer (adoptium.net link); MSI upgrade
  autodetects the directory and replaces the service; uninstall removes the
  service automatically.
- _script-util.bat: quote -Djava.io.tmpdir="%OPENDJ_TMP_DIR%" - any install
  directory containing spaces (including the x64 default C:\Program Files\OpenDJ
  and the legacy Program Files (x86)) broke the CheckJVMVersion probe and every
  script with "The detected Java version could not be used".
- build.yml test-msi: the no-JRE negative test now hides Java at MACHINE scope
  (the Windows Installer service evaluates launch conditions with the machine
  environment) and restarts msiserver, restoring everything afterwards; the
  positive install now goes to the spaced x64 default directory end-to-end.
- build.yml test-msi-upgrade (renamed from test-win-upgrade): the released 5.1.1
  scripts cannot run from a spaced directory, so the old install lives in
  C:\opendj and the upgrade passes OPENDJ explicitly; added a fresh-install
  scenario asserting the legacy default directory is auto-detected when OPENDJ
  is not given, plus uninstall/service cleanup checks.
- ADNotificationRequestControl: escape && in the javadoc code sample
  (javadoc "invalid input: '&'" warnings).
…aces)

_script-util.bat appends -Djava.io.tmpdir=%OPENDJ_TMP_DIR% to OPENDJ_JAVA_ARGS
without quotes, so in any install directory containing spaces (for example
C:\Program Files (x86)\OpenDJ) the CheckJVMVersion probe - and with it setup
and every other command-line tool - fails with "The detected Java version
could not be used with the set of Java arguments".

Quote the value: -Djava.io.tmpdir="%OPENDJ_TMP_DIR%".
…cp.bat)

_script-util.bat passed unquoted paths to setcp.bat, and setcp.bat compared
arguments with if ""%1""=="""". The argument-joining hack survived spaces, but
a parenthesis in the path (C:\Program Files (x86)\OpenDJ - the default MSI
directory) breaks the cmd parser with "... was unexpected at this time", so
setup.bat and every tool exit with 255.

Quote the setcp.bat arguments at the three call sites and switch setcp.bat to
%~1 with quoted comparisons.
…e "%VAR%" == "" comparisons

After -Djava.io.tmpdir="%OPENDJ_TMP_DIR%" is appended, OPENDJ_JAVA_ARGS
contains embedded quotes, and the subsequent if "%OPENDJ_JAVA_ARGS%" == ""
checks blow up the cmd parser ('...\tmp"" was unexpected at this time', every
tool exits 255) regardless of whether the install path has spaces. Compare
with "if defined", which does not expand the value.
…h quoting

- The <Files> harvest ships files only (heat had -ke), so the empty instance
  directories (bak, changelogDb, classes, db, import-tmp, ldif, locks, logs,
  tmp and the template ones) were missing from the installed tree and the
  server could not create its lock/pid files - setup aborted with "error
  stopping server". Create them with explicit CreateFolder components.
- [OPENDJ] ends with a backslash which escaped the closing quote in the
  service ImagePath ('start "C:\opendj\"' -> broken argv), so the MSI-managed
  service failed to start (NET HELPMSG 2186). Append an extra backslash so
  \\" parses as backslash + closing quote.
Components whose KeyPath is a directory cannot use auto-generated GUIDs, so the empty-instance-directory components broke wix build with WIX0230. Assign fixed GUIDs.
The tmp-cleanup block expands %OPENDJ_TMP_DIR% unquoted inside a ( ) compound statement, so a parenthesis in the install path - C:\Program Files (x86)\OpenDJ - terminates the block at parse time and start-ds fails; the Windows service then dies with a service-specific error -1 while setup (which never runs this block) succeeds. Quote the three path expansions.
opendj_service.exe located its own service entry by comparing the raw
SCM ImagePath with the exact string it builds itself, so a service
registered by the MSI (exe unquoted, instance dir with a trailing
backslash) was never found and "net start" failed with error 2186.
getServiceName now compares the executable path, the subcommand and the
normalized instance dir token by token instead. The MSI ServiceInstall
argument becomes 'start "[OPENDJ]."' and the sc-config workaround is
not needed.
@vharseko vharseko added the packaging deb/rpm/MSI, distribution layout, config.ldif label Jul 6, 2026
@vharseko vharseko added the docs label Aug 6, 2026
@vharseko

vharseko commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

Thanks for the thorough review — almost everything reproduced exactly as described. Addressed in 3aa24c5 unless noted otherwise.

Blockers

  • release-msi now has permissions: contents: write, and deploy.yml grants actions: read. On the "three independent ways the MSI silently vanishes" point, the deliberate decision is: an MSI failure must not block a release (the Windows runner + WiX are the least reliable part of the pipeline), so continue-on-error stays on the job — but with the 403 fixed, the remaining failure modes are visible red jobs rather than silent successes.
  • Dropping opendj-msi-standard from Maven Central is intentional — now stated in the PR description. The MSI is distributed via GitHub Releases and the Package/Deploy artifact; the coordinate stays at 5.1.2 as its last published version.

RemoveLegacyService — replaced with exactly the ServiceControl you suggested (Stop="install" Remove="install" Wait="yes"). Note this deliberately tightens semantics: a legacy service that fails to stop now fails the install instead of silently proceeding. The localized display-name gap (OpenDS on ko/zh_TW) applies to the ServiceControl row just as much; left as a documented limitation in the wxs comment.

EULA — restored the <Publish> overrides (Welcome → InstallDir); the license is shown by setup, as before the rewrite.

Java launch condition — dropped entirely rather than extended: given the #664 history, and that every install-time signal (registry keys included) can false-negative on a machine where the server would run fine (OPENDJ_JAVA_HOME, default.java-home), the installer now installs unconditionally and the docs state that setup/the server require Java 11+. The no-JRE CI test went with it.

--disableServiceopendj_service.exe now refuses to remove the MSI-managed key OpenDJ (guard in removeService()), CI asserts it against an MSI install, and "Before You Upgrade" scopes the disable/enable advice to zip installs. One correction though: the PR does not remove the only CI coverage of those two commands — the zip-package Windows test still runs --enableService/--disableService; what was missing was coverage against an MSI install, which is now added.

Start="auto" — keeping auto deliberately: nothing would promote a demand-start service back to auto after setup (windows-service.bat manages its own service, not the MSI one), so demand means the server never comes back after a reboot unless the user runs sc config by hand. The boot-time error events on an unconfigured tree until setup runs are the accepted cost.

Release never rebuilds the exes — added your git diff --exit-code -- opendj-server-legacy/lib, with the exe artifact upload moved just before it so a failing run already carries the refreshed binaries for the fix-up commit. Heads-up: the next Build run will be red on that step by design — service.c changed in this round and the committed launchers need a refresh from the windows-exe-11 artifact.

wix guard — new distribution-windows-msi profile activates only when %USERPROFILE%\.dotnet\tools\wix.exe exists (plain mvn install no longer breaks mid-reactor); the full three-command toolchain is in the module description; the MSI now builds on the java-11 job only, which also removes the double build.

AppSearch — split into OPENDJ_REG/OPENDJ_LEGACY with explicit SetProperty ordering as suggested (legacy gated on fresh installs; registry unconditional but still yielding to an explicit OPENDJ), and the registry branch got a CI scenario: planted InstallDir plus a legacy directory, asserting the registry wins.

Nits — applied: event-log source registered by the MSI itself, GetLongPathName on both tokens in serviceCmdsMatch, ImagePath quoting asserted in CI, attach-artifact moved after the wix build and renamed, exec-maven-plugin pinned to 3.6.3, null-safe sc.exe getkeyname parsing, retry on the 5.1.2 download, stale comments/docs/.gitattributes fixed. Two exceptions:

  • MajorUpgrade/@Schedule: kept the afterInstallValidate default. Late RemoveExistingProducts requires matching component GUIDs for matching paths, and the auto-generated GUIDs of the WiX3 x86 package vs this WiX5 x64 one cannot guarantee that — a late REP would delete freshly installed files. Early removal is safe here precisely because instance data is outside any component set; rationale is now a comment in the wxs. (AllowSameVersionUpgrades is applied.)
  • Two tiny factual notes: C:\Users\javier does not actually match ~>< "java" ("java" is not a substring of "javier" — the JavaScript-toolchain example is the real one), and sc delete followed by CreateService yields ERROR_SERVICE_MARKED_FOR_DELETE (1072) rather than ERROR_DUPLICATE_SERVICE_NAME. Neither changes your conclusions.

@vharseko
vharseko requested a review from maximthomas August 6, 2026 14:46
…(/Brepro)

lib/*.exe are taken from the windows-exe-11 artifact of Build run 31112196230:
opendj_service.exe now refuses to remove the MSI-managed service and expands
8.3 paths in serviceCmdsMatch; the launchers pick up only PE-timestamp changes.

/Brepro is added to cl and link so the PE TimeDateStamp becomes a content hash:
without it every build differs in the timestamp fields and the
binaries-match-sources CI guard could never pass. These binaries still predate
the flag, so the guard stays red for one more run; the follow-up refresh from
the first /Brepro build is final.
lib/*.exe are taken from the windows-exe-11 artifact of Build run 31114568111 -
the first build with /Brepro, so the PE timestamp fields are now content hashes.
No source change: this only realigns the committed bytes with the reproducible
toolchain output, after which the binaries-match-sources CI guard stays green.
The registry install-location action must run before the legacy-directory
fallback: both are guarded by NOT OPENDJ, so with the legacy action first an
existing Program Files (x86)\OpenDJ directory won over the InstallDir registry
value. Precedence is now explicit OPENDJ -> registry -> legacy directory.

The --disableService guard test failed on the expected non-zero exit code of
windows-service.bat (the pwsh wrapper turns the last exit code into the step
result): capture it, assert the refusal, and finish with an explicit exit 0.

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid work — the WiX 5 rewrite is coherent, the rationale comments are accurate, and the two new end-to-end MSI jobs cover ground that was untested. Everything below comes from moving service ownership into the installer: three regressions that the new CI cannot see, plus a few things worth hardening. The binaries are already refreshed, so the "next Build run is red by design" note no longer applies.

Corrections to my previous round: the custom-directory upgrade is not data loss and not a regression (5.1.2 had no location detection and the same default schedule, and config/db/logs belong to no component); ServiceControl has no Return attribute in any WiX version, so aborting on a failed stop is exactly the deliberate choice you described; the expandLongPath cost is noise next to the QueryServiceConfig calls already in that loop; and exec-maven-plugin genuinely cannot be version-less, since the root pom does not manage it.

uninstall.bat fails on every MSI install (blocker)

The refusal returns 3, which is already SERVICE_DISABLE_ERROR:

// opendj-server-legacy/src/build-tools/windows/service.c:2661
if (_stricmp(serviceName, MSI_SERVICE_NAME) == 0) { ...; returnCode = 3; }

serviceCmdsMatch now makes serviceState() resolve the MSI service, so Uninstaller runs the disable step and throws on the result:

// opendj-server-legacy/src/main/java/org/opends/quicksetup/uninstaller/Uninstaller.java:1375
default: throw new ApplicationException(ReturnCode.WINDOWS_SERVICE_ERROR, errorMessage, null);

deleteExternalDatabaseFiles0() / deleteInstallationFiles0() never run — the server is stopped and nothing is deleted. uninstall.bat is in the payload (opendj-packages/opendj-msi/opendj-msi-standard/pom.xml excludes the extension-less uninstall, not uninstall.bat), and since msiexec /x deliberately leaves config/db/logs, it is the natural next step for a user. The displayed message then tells them to run windows-service.bat -d, which can never succeed.

Give the refusal its own return code and treat it as a skip in Uninstaller.disableWindowsService() and InstallerHelper.disableWindowsService(). That also fixes the swallowed message (see Nits).

A second instance can never register its service again (blocker)

<!-- opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:135 -->
<ServiceInstall Id="OpenDJService" Name="OpenDJ" DisplayName="OpenDJ Server" .../>

The uniqueness loop that produces OpenDJ Server-2 compares key names only:

// opendj-server-legacy/src/build-tools/windows/service.c:2101
if (strcmp (serviceName, curService.serviceName) == 0)

Key OpenDJ Server is free, so createServiceName stops at i == 1 and CreateService(scm, "OpenDJ Server", "OpenDJ Server", ...) fails with ERROR_DUPLICATE_SERVICE_NAME against the MSI's display name. There is no retry (service.c:2247 maps it straight to DUPLICATED_SERVICE_NAME), and the name is not user-selectable — so on a host with the MSI product installed, windows-service.bat --enableService in any zip instance fails permanently. That is the normal multi-instance replication setup. DisplayName="OpenDJ", or comparing displayName in serviceNameInUse(), closes it.

Vital="no" hides a failed service registration (blocker)

<!-- opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:138 -->
Arguments="start &quot;[OPENDJ].&quot;" Vital="no"/>

If CreateService fails — e.g. the legacy service still has an open handle, so DeleteServices only marked it for deletion and InstallServices hits ERROR_SERVICE_MARKED_FOR_DELETE — MSI logs error 1923 and continues. msiexec exits 0, ARP shows the product installed, and there is no service. It is the only unexplained attribute in the file; dropping it restores the yes default.

RemoveExistingProducts runs before StopServices (should fix)

The comment at package.wxs:145 claims the ServiceControl row gives "right instance semantics", but with the default afterInstallValidate schedule RemoveExistingProducts is at ~1401 while StopServices is at 1900. Upgrading a running 5.1.x without stopping it first — which the PR advertises as no longer needing a manual --disableService — deletes lib\*.jar and lib\opendj_service.exe while the JVM holds them, so they get queued with MOVEFILE_DELAY_UNTIL_REBOOT and can remove the freshly installed binaries at the next boot. test-msi-upgrade runs net stop "OpenDJ Server" first, so it never sees this. Either stop the legacy service earlier (a Before="RemoveExistingProducts" custom action, or accept a custom action here) or document that the service must be stopped first.

Repair stops a running server and never restarts it (should fix)

<!-- opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:139 -->
<ServiceControl Id="OpenDJServiceControl" Name="OpenDJ" Stop="both" Remove="uninstall" Wait="yes"/>

Stop="both" fires on install, reinstall and repair; there is no Start, so msiexec /f (or Repair in Apps & Features) silently takes a production directory server offline with no message. Stop="uninstall" covers both the uninstall and the RemoveExistingProducts-driven upgrade, so both buys nothing here.

Legacy-service removal is not instance-aware (should fix)

<!-- opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:147 -->
<ServiceControl Id="RemoveLegacyOpenDJServer" Name="OpenDJ Server" Stop="install" Remove="install" Wait="yes"/>

MSI has no way to check which instance the service points at, so this deletes whichever instance happens to hold the bare name. On a host running an unrelated zip instance at D:\opendj, a fresh MSI install into C:\Program Files\OpenDJ stops and removes that instance's service — and Wait="yes" means a slow shutdown there fails this install. It also re-fires on every repair. The trade-off may be acceptable, but chap-install.adoc says nothing about it (only chap-upgrade.adoc does), and the localized case (OpenDS) silently matches nothing, leaving two auto-start services on one directory.

A release can publish with no MSI and stay green (should fix)

.github/workflows/release.yml:95 uploads release-server-zip with continue-on-error: true and the default if-no-files-found: warn; release-msi then fails at its download-artifact, and the job-level continue-on-error: true (:166) keeps the workflow conclusion green. Release on GitHub (:110-119) no longer lists the MSI, so it reaches the release only through release-msi. Net effect: a tagged release ships without an MSI and nothing is red. Same shape in .github/workflows/deploy.yml:116,124, where both the download and the upload are continue-on-error. Keeping the job non-blocking is fine — but set if-no-files-found: error on the uploads so the loss is visible.

The launcher-binary guard is a tripwire (should fix)

# .github/workflows/build.yml:100
run: git diff --exit-code -- opendj-server-legacy/lib

/Brepro replaces build-time timestamps with a content hash; it does not make codegen or the linked CRT toolchain-independent. This PR demonstrates it: between be03df284c and ac4eeedf19 — identical sources, both post-/Brepro — the three exes differ by ~3 KB each, because the runner's linker moved 14.44 → 14.51 mid-PR. The next runner-image bump therefore reds both Windows jobs at a step that runs before Build with Maven, so no windows-latest-11 artifact is produced, test-msi* and build-docker* skip, and on master deploy.yml does not run at all — until someone with a Windows box re-commits three binaries.

Worse, the guard cannot catch the case it exists for. The preceding step is shell: cmd and ends with git status, so its exit code is always 0:

# .github/workflows/build.yml:80-85
shell: cmd
run: |
  cd opendj-server-legacy\src\build-tools\windows
  nmake all
  xcopy /Y *.exe ..\..\..\lib\
  git status

A broken nmake all leaves lib/ untouched, so git diff --exit-code passes. Adding || exit /b 1 after nmake/xcopy fixes that; pinning toolset: on ilammy/msvc-dev-cmd, or downgrading the comparison to a warning, removes the tripwire.

A custom-directory upgrade still relocates (should fix)

Not the blocker I called it last round — instance data survives and 5.1.2 behaved the same way — but for a 5.1.x install outside [ProgramFilesFolder]OpenDJ, OPENDJ_REG and OPENDJ_LEGACY are both empty (5.1.2 writes no registry value), so a /quiet upgrade without OPENDJ installs to C:\Program Files\OpenDJ while RemoveExistingProducts empties the old tree. chap-upgrade.adoc:266 does warn, and there is no CI coverage because both upgrade steps pass OPENDJ=C:\opendj. A third search on the legacy service's ImagePath (HKLM\SYSTEM\CurrentControlSet\Services\OpenDJ Server\ImagePath embeds the old root), or a launch condition when WIX_UPGRADE_DETECTED is set but OPENDJ resolved to the untouched default, would close it.

Nits

  • cleanup bypasses the guard: the MSI_SERVICE_NAME check is only in removeService(); windows-service.bat --cleanupService OpenDJ reaches removeServiceWithServiceName() (opendj-server-legacy/src/build-tools/windows/service.c:2600) and deletes the key plus its event-log key anyway. Moving the check one level down makes the guard consistent.
  • The refusal message is never printed: service.c:2666 writes it to stdout, but ConfigureWindowsService.java:474 is Runtime.getRuntime().exec(cmd).waitFor() with no stream drained. Users see only "An unexpected error occurred trying to disable the server as a Windows service. Check that you have administrator rights" — resolved for free by the dedicated return code above.
  • Start="auto" before setup: package.wxs:137 plus an unconfigured tree means the SCM logs a start failure on every boot until setup.bat runs. Worth a sentence in chap-install.adoc, which currently only says the service is not started during installation.
  • The 5.1.2 CI comment is wrong: .github/workflows/build.yml:725 says the released 5.1.2 scripts cannot run from a directory with spaces, but the 5.1.2 tag (2026-07-17) post-dates #671 (merged 2026-07-08) — the scripts are byte-identical to this branch. Because of that the old product is installed into C:\opendj, so the headline path (real 5.1.2 at its own default, upgraded with no OPENDJ) is never tested end to end.
  • release.yml:215 relies on implicit profile activation: mvn ... package -pl :opendj-msi-standard never passes -Pdistribution-windows-msi, so any drift in USERPROFILE or the dotnet tool path yields "Could not find the selected project in the reactor" — swallowed by the job's continue-on-error. Adding the flag costs nothing.
  • Custom install dirs and LocalSystem: the service runs as LocalSystem with no Account, and a directory created directly under C:\ (as OPENDJ=C:\opendj in the docs and CI) inherits Authenticated Users:(OI)(CI)(IO)(M). Any standard user can then replace bat\start-ds.bat and get SYSTEM. A doc warning, or an ACL reset on [OPENDJ], would help.
  • Upgrades discard service hardening: because the old product is uninstalled first, sc config OpenDJ obj= DOMAIN\svc-opendj, recovery actions and dependencies are lost on every upgrade and the service silently returns as LocalSystem.
  • No "run as Administrator" note: chap-install.adoc:665,678 tell users to run setup.bat and net start from C:\Program Files\OpenDJ; a 64-bit JVM is never file-virtualized, so both fail without elevation. Pre-existing, but the x64 move makes the protected directory the default.
  • exec-maven-plugin versions diverge: 3.6.3 in opendj-packages/opendj-msi/opendj-msi-standard/pom.xml:112 vs 1.3.2 in opendj-server-legacy/pom.xml:1406, neither centrally managed. Root pluginManagement would let both converge.
  • GUI path is untested: every CI install uses /quiet /qn, so the restored license <Publish> overrides and the InstallDirDlg pre-population from a detected OPENDJ are unverified. One manual run before merge would be worth it.

- opendj_service.exe refuses to remove the MSI-managed key with a dedicated
  code (4): uninstall.bat and windows-service.bat now treat it as an
  informational skip instead of failing, and --cleanupService is covered too
- Name the MSI service 'OpenDJ' (key = display name): the windows-service.bat
  naming loop checks key names only, so any other display name made every
  additional zip instance's --enableService fail with a duplicate name
- Drop Vital="no": a failed service registration fails the install instead of
  reporting success with no service; Stop="uninstall" keeps repair from
  silently stopping a production server
- Stop a running legacy service before RemoveExistingProducts (immediate
  action, full net.exe path) and remove it only when an existing OpenDJ was
  detected - a fresh install must not touch an unrelated instance's service
- Refuse a silent upgrade when the existing location cannot be determined
  (5.1.x wrote no registry value) instead of relocating to the default
- CI: upgrade from 5.1.2 at its real x86 default without OPENDJ, uninstall.bat
  and refusal scenarios, if-no-files-found: error on the MSI-chain uploads,
  nmake/xcopy error propagation, binaries guard downgraded to a warning
- Manage exec-maven-plugin centrally (3.6.3), document the new behaviors
…stribution-new

# Conflicts:
#	.github/workflows/build.yml
@vharseko vharseko added the java Pull requests that update java code label Aug 7, 2026
@vharseko

vharseko commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

Thanks for the second pass — the service-ownership consequences were exactly the right place to look, and all three blockers reproduced. Addressed in af7d80f (plus a master merge in f0c9a11); notes below follow your headings.

uninstall.bat — the refusal now has its own wrapper return code (4). removeServiceWithServiceName carries the guard, so --cleanupService is covered too; ConfigureWindowsService maps it to a new SERVICE_MSI_MANAGED constant with a proper message (INFO_WINDOWS_SERVICE_MSI_MANAGED), Uninstaller.disableWindowsService() treats it as a skip, and InstallerHelper needed no change (it only throws on SERVICE_DISABLE_ERROR). That also resolves the swallowed-stdout nit — the message now comes from the Java side. CI runs uninstall.bat --cli --remove-all against an MSI install and asserts the instance files are gone while the service survives for msiexec /x.

Second instance — fixed by naming the service OpenDJ outright (key = display name), so the createServiceName loop's first candidate OpenDJ Server no longer collides with anything and no C change was needed. CI and docs now use net start OpenDJ; the upgrade test asserts the display name and that no OpenDJ Server service remains.

Vital="no" — dropped; with the default a failed CreateService (e.g. ERROR_SERVICE_MARKED_FOR_DELETE) fails the install with rollback instead of reporting success with no service.

REP before StopServices — one note: chap-upgrade.adoc step 1 has always required stopping the server (net stop) before installing the newer package, so the documented path was safe. But the PR does advertise no-manual-steps upgrades, so a running legacy service is now stopped explicitly: an immediate "[System64Folder]net.exe" stop "OpenDJ Server" action (full path — no working-directory resolution) sequenced before RemoveExistingProducts, WIX_UPGRADE_DETECTED only, Return="ignore". Deferred was not an option — with the afterInstallValidate schedule REP sits before InstallInitialize.

RepairStop="uninstall"; agreed that both bought nothing except taking a production server down on msiexec /f.

Instance-awareness — the legacy ServiceControl moved into its own component, conditioned on WIX_UPGRADE_DETECTED OR OPENDJ_LEGACY OR OPENDJ_REG: a fresh install on a host running an unrelated zip instance elsewhere no longer touches that instance's service. chap-install.adoc now states when the replacement happens (and the boot-time start-failure note until setup runs, plus the localized OpenDS gap staying documented in the wxs).

Release visibilityif-no-files-found: error on both the release-server-zip upload and the deploy.yml MSI upload; the jobs stay non-blocking per the earlier decision, but a missing MSI is now a red step instead of a warn. release.yml also passes -Pdistribution-windows-msi explicitly.

The guard — you're right about the cascade and about the swallowed nmake failure: nmake/xcopy now || exit /b 1, and the comparison is warning-only (continue-on-error + a ::warning annotation naming the artifact to refresh from), so a toolchain bump can no longer kill test-msi*/deploy. One correction to the evidence, not the conclusion: the ~3 KB delta between be03df2 and ac4eeed is the /Brepro flag being introduced (be03df2's committed binaries were still a pre-flag build), not linker drift — the 14.44→14.51 observation from your first round is the real precedent.

Custom-directory upgrade — a type-19 error action now refuses when WIX_UPGRADE_DETECTED AND NOT OPENDJ (execute sequence only: in the wizard the user sees and confirms the directory in InstallDirDlg, which sets OPENDJ). There are no false positives — x64 installs write the registry value and the legacy default is detected. CI covers it: 5.1.2 at C:\opendj-custom, silent upgrade without OPENDJ must exit non-zero with the guidance message and leave the original tree untouched.

5.1.2 comment — good catch, and it unlocked the honest headline test: test-msi-upgrade now installs 5.1.2 at its real Program Files (x86)\OpenDJ default (verified against the 5.1.2 tag's wxs) and upgrades with no OPENDJ at all; the explicit-OPENDJ path stays covered by the registry-detection scenario.

Remaining nitsexec-maven-plugin converged on 3.6.3 via root pluginManagement (the 1.3.2 pin in opendj-server-legacy is gone; the full-matrix build exercises the MIB generation with it). Docs gained the LocalSystem/ACL warning for root-level custom directories, the run-as-Administrator note, and the service-hardening-lost-on-upgrade caveat. The GUI wizard run stays manual — planned before merge.

The committed launcher binaries are pending one more refresh (the service.c guard moved and gained a return code), which now surfaces as the warning annotation rather than a red job.

Taken from the windows-exe-11 artifact of Build run 31191804954. Only the
service wrapper changed (the MSI-managed guard moved into
removeServiceWithServiceName and gained the dedicated return code 4);
winlauncher.exe and launcher_administrator.exe came out byte-identical to the
committed ones - the first live confirmation that the /Brepro build is
reproducible.
@vharseko
vharseko requested a review from maximthomas August 7, 2026 16:48

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-3 review of af7d80f9da + 4bf7927489. The service-ownership rework reads well and I could not fault service.c — the tokenizer bounds, normalizeInstanceDir edge cases and the round-trip against the MSI's ImagePath ("<root>\lib\opendj_service.exe" start "<root>\.") all check out, as do the message wiring and .gitattributes.

Two blockers below, both outside the paths CI exercises. Also note the MSI jobs have not run on round 2 yet — build-maven (ubuntu-latest, 11) is still in flight, so test-msi/test-msi-upgrade last passed on c287e5b3cf. Nothing is gated; they just need a green build.

Two things from my round-2 comments I withdraw after checking: dropping Vital="no" is a no-op (WiX 5.0.2 defaults Vital to noCompiler_Package.cs:3531), and the missing WixUI_ErrorProgressText is harmless (Windows Installer falls back to built-in localized ActionText/Error strings).

Released MSI ships the slim server zip (blocker)

.github/workflows/release.yml:100 uploads both zips:

path: target/checkout/opendj-server-legacy/target/package/*.zip

and .github/workflows/release.yml:210 picks the first one:

ZIP=$(ls server-zip/*.zip | head -1)

opendj-<ver>-slim.zip sorts before opendj-<ver>.zip (- = 0x2D < . = 0x2E, in both C and en_US). The slim zip excludes the backend drivers (opendj-server-legacy/pom.xml:1130):

lib/org.apache.cassandra.*,lib/com.oracle.database.jdbc*,lib/com.mysql.mysql-connector*,
lib/com.microsoft.sqlserver*,lib/org.postgresql.*

dependency:unpack expands it without error (same opendj/ prefix), so every released MSI silently lacks JDBC/Cassandra support while the same release's .zip has it. test-msi builds through the normal reactor with the full zip, so only the release path is affected.

ZIP=$(ls server-zip/*.zip | grep -v -- '-slim\.zip$' | head -1)

Legacy-service guard defeats itself after the first install (blocker)

opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:194:

<Component Id="RemoveLegacyServiceControl" Directory="OPENDJ"
           Condition="WIX_UPGRADE_DETECTED OR OPENDJ_LEGACY OR OPENDJ_REG">
  <ServiceControl Id="RemoveLegacyOpenDJServer" Name="OpenDJ Server"
                  Stop="install" Remove="install" Wait="yes"/>

OPENDJ_REG reads HKLM\SOFTWARE\OpenDJ\InstallDir (line 53) — the value this package's own InstallDirRegistry component writes (line 204). Same root, key, name and registry view. So the condition is false only on a clean first install; from then on it is permanently true on that host, and a repair (msiexec /f, or ARP → Repair via MaintenanceTypeDlg) transitions the component Absent→Install and deletes an unrelated zip instance's OpenDJ Server service — the exact case the comment on line 192 says must not be touched. Recovery needs windows-service.bat --enableService.

WIX_UPGRADE_DETECTED is a second path to true, so removing the OPENDJ_REG disjunct alone does not close it. Gating on something that proves this product owns the legacy service would.

Custom-directory upgrade guard cannot fire in the wizard (major)

package.wxs:90 schedules the guard in InstallExecuteSequence only:

<Custom Action="RequireDirOnCustomUpgrade" After="SetOpendjFromLegacyDir"
        Condition="WIX_UPGRADE_DETECTED AND NOT OPENDJ"/>

In a full-UI install the UI sequence runs first and its CostFinalize (1000) resolves the OPENDJ Directory row to C:\Program Files\OpenDJ\ before ExecuteAction. NOT OPENDJ is therefore false at ~403 and the guard never fires. Upgrading a 5.1.x install at, say, C:\opendj-custom (no registry value) installs to the default while RemoveExistingProducts empties the old tree.

The comment on line 75 says the user "sees and confirms the directory in InstallDirDlg" — but what InstallDirDlg shows is the default, not the detected install, and nothing tells them an existing install was found elsewhere. Comparing the resolved OPENDJ against the upgrade's recorded location, or duplicating the guard into InstallUISequence after the dialogs, would cover it.

StopLegacyServiceBeforeUpgrade runs unelevated (major)

package.wxs:86 has no Execute, so it is immediate (type 34 + Continue = 98):

<CustomAction Id="StopLegacyServiceBeforeUpgrade" Directory="TARGETDIR"
              ExeCommand="&quot;[System64Folder]net.exe&quot; stop &quot;OpenDJ Server&quot;"
              Return="ignore"/>

Immediate custom actions always run in the msiexec client process under the launching user's token — position relative to InstallInitialize is not the deciding factor. On a UAC double-click net stop returns access denied and Return="ignore" swallows it, so the legacy service keeps holding lib\*.jar through RemoveExistingProducts and the delayed-delete hazard the comment describes is unmitigated. The ServiceControl on line 198 only cleans up later, at StopServices (1900). CI cannot see this: hosted runners have a full admin token, and .github/workflows/build.yml:815 stops the service by hand first.

Return="ignore" is load-bearing for the service-absent case, so it cannot simply be dropped.

sustaining/4.10.x deploys stop publishing the MSI (major)

.github/workflows/deploy.yml is workflow_run-triggered, so GitHub always runs master's copy, but the job checks out ${{ github.event.workflow_run.head_branch }}. That branch's opendj-packages/pom.xml still lists <module>opendj-msi</module> under the /usr/bin/wine profiles. With wine no longer installed the profile stays inactive, no MSI is produced, and the upload step fails into continue-on-error: true — a yellow step and no MSI.

Native-exe staleness gate is continue-on-error (minor)

.github/workflows/build.yml:103. CI copies freshly compiled exes over opendj-server-legacy/lib/ before the Maven build, so test-msi validates the compiled wrapper, while releases ship the committed one. A PR that changes service.c without re-committing lib/*.exe goes green with a yellow warning and ships the old wrapper — which would happily delete the MSI-managed service. Worth a job that fails rather than a step that warns. (The binaries are in sync today.)

Nits

  • lib\extensions no longer created: <Files> harvests files only and OpenDJEmptyDirs (package.wxs:128) recreates the root and template/* dirs but not lib/extensions, so WARN_ADMIN_NO_EXTENSIONS_DIR logs on every startup and there is nowhere to drop a custom extension jar.
  • WindowsServicePanel not updated for code 4: opendj-server-legacy/src/main/java/org/opends/guitools/controlpanel/ui/WindowsServicePanel.java:337 still treats SERVICE_MSI_MANAGED as FINISHED_WITH_ERROR; Uninstaller.java:1374 treats it as a skip.
  • Vital comment is wrong: package.wxs:163 says "Vital defaults to yes". It defaults to no in WiX 4/5 (the XSD prose says otherwise — wixtoolset/issues#5974), so the fail-fast the comment promises needs an explicit Vital="yes". Separately, new-MSI → new-MSI upgrade is untested; the DeleteService/CreateService 1072 race yields a silently missing service.
  • SERVICE_MSI_MANAGED in the wrong constant block: ConfigureWindowsService.java:101 declares it among the disable codes but returns it from cleanupService, whose codes are deliberately renumbered and whose javadoc lists only SERVICE_CLEANUP_*. A SERVICE_CLEANUP_MSI_MANAGED would fit.
  • Duplicate refusal message: service.c:2615 prints the refusal and the Java side prints the localized INFO_WINDOWS_SERVICE_MSI_MANAGED. On non-UAC hosts the user sees both; the native one can drop to debug().
  • --cleanupService has no escape hatch: the guard keys purely on the name "OpenDJ", so an orphaned service (rolled-back install, hand-run create) can never be cleaned and needs sc delete. Checking actual MSI ownership would keep the escape hatch.
  • uninstall.bat ordering: opendj-doc-generated-ref/src/main/asciidoc/install-guide/chap-uninstall.adoc:175 recommends uninstall.bat --remove-all before msiexec /x, which deletes lib\opendj_service.exe while leaving the auto-start service registered — SCM 7000/7009 every boot until the package is removed. Worth a warning.
  • opendj-msi* no longer on Maven Central: dropped from distribution-unix/distribution-mac in opendj-packages/pom.xml, so the modules leave the Linux mvn deploy reactor. Probably intended, but it silently changes published coordinates.
  • wix lookup mismatch: opendj-packages/pom.xml:92 activates on %USERPROFILE%\.dotnet\tools\wix.exe but the build invokes bare wix via PATH. Same split in build.yml:116 / release.yml:198, which push the MSYS path /c/Users/runneradmin/.dotnet/tools into $GITHUB_PATH from a shell: bash step — it works only because the runner image already exposes the directory.
  • WixUI_ErrorProgressText parity: harmless (built-in fallback), but re-adding <UIRef Id="WixUI_ErrorProgressText"/> restores the pre-PR authoring at zero risk.

… zip in release MSI

- release.yml: package the MSI from the full server zip (the slim zip sorted
  first and silently dropped the JDBC/Cassandra drivers)
- package.wxs: LEGACY_MSI_DETECTED detect-only Upgrade row (< 5.2.0) replaces
  WIX_UPGRADE_DETECTED/OPENDJ_REG in the legacy-service removal condition;
  Transitive component so repair cannot re-fire it; the pre-upgrade net stop
  is deferred/no-impersonation (Schedule=afterInstallInitialize) and
  legacy-only; the custom-directory upgrade guard also runs in the UI
  sequence; explicit Vital="yes" (WiX 5 defaults to no); lib\extensions
  component; WixUI_ErrorProgressText restored
- service.c: the MSI guard now verifies ownership (InstallDir registry value
  matches the service's instance dir), keeping orphaned services removable;
  the native refusal message drops to debug (the Java side prints it
  localized)
- ConfigureWindowsService: SERVICE_CLEANUP_MSI_MANAGED in the cleanup code
  block; WindowsServicePanel treats the refusal as an informational skip
- check-native-launchers.yml: standalone failing check for stale committed
  launcher binaries, triggered only by native source/binary changes
- docs: uninstall ordering warning, tightened service-replacement note;
  test-msi asserts lib\extensions
@vharseko

Copy link
Copy Markdown
Member Author

Thanks for the third pass — both blockers and both majors reproduced, and the self-defeating guard condition was a particularly good catch. Addressed in 0d88657; notes follow your headings. The committed launchers are pending one more refresh (service.c changed again), which now shows up as the red Native launchers check plus the warning annotation — refresh from the next windows-exe-11 artifact, as before.

Slim server zip — fixed with exactly your filter (grep -v -- '-slim\.zip$'), plus a comment explaining the sort order so nobody "simplifies" it back.

Legacy-service guard — agreed: OPENDJ_REG read the very value this package writes. Ownership is now proven by version instead: a detect-only <UpgradeVersion Minimum="0.0.0" Maximum="5.2.0" IncludeMaximum="no"> row on the same UpgradeCode sets LEGACY_MSI_DETECTED (the Maximum is pinned to the first WiX5 release and deliberately does not track the package version), and the component condition is LEGACY_MSI_DETECTED OR (NOT Installed AND OPENDJ_LEGACY)OPENDJ_REG is gone and a new-MSI-to-new-MSI upgrade no longer matches. The component is also Transitive="yes" now. One mechanical note on the repair path: on a host whose first install evaluated the condition false, msiexec /f alone would not have flipped it Absent→Install (REINSTALL only touches installed components, and non-transitive conditions are not re-evaluated in maintenance) — but the hazard was real regardless: after any legacy upgrade the component was installed, and its ServiceControl rows re-fire on every repair; and every future major upgrade re-evaluated the old condition in a fresh session. Transitive plus the version gate closes both: in maintenance both signals are gone, so the component transitions out of the install set instead of re-firing. StopLegacyServiceBeforeUpgrade is conditioned on LEGACY_MSI_DETECTED too, for the same reason.

Custom-directory upgrade guard in the wizard — the guard is now scheduled in InstallUISequence as well (after the SetProperty pair, before costing, where OPENDJ is still unset), so a full-UI upgrade from an undetectable directory gets the same refusal; /qb and /qn keep hitting the execute-sequence row. The comment claiming InstallDirDlg "shows and confirms" the detected directory went with it.

StopLegacyServiceBeforeUpgrade unelevated — right, and the fix follows from your observation: MajorUpgrade/@Schedule="afterInstallInitialize" keeps RemoveExistingProducts early (the GUID-matching argument only rules out the post-InstallExecute variants), which lets the stop action become Execute="deferred" Impersonate="no" — LocalSystem, elevated. Return="ignore" stays for the service-not-running case.

sustaining/4.10.x — accepted as a deliberate loss and now stated in the PR description: Package/Deploy always runs master's workflow, wine is gone from it, so 4.10.x stops publishing the MSI; its last published release remains available.

Native-exe staleness gate — now a failing check without the cascade: a standalone Native launchers workflow triggers only on changes to src/build-tools/windows/** or lib/*.exe (exactly the stale-binaries case — a runner toolchain bump touches neither path, so it cannot red the world), builds with the same pinned msvc-dev-cmd and fails on git diff. Build's own step stays warning-only and points at it.

Nits

  • lib\extensions — this one did not reproduce: the payload ships lib/extensions/snmp-mib2605.jar (checked in the built opendj-server-legacy-5.2.0-SNAPSHOT.zip), so the harvest creates the directory. Added the CreateFolder component anyway so the directory survives if that jar ever leaves the package, and test-msi now asserts it exists.
  • Vital — confirmed against the WiX 5.0.2 source (var vital = false;, Compiler_Package.cs:3531, matching your citation): explicit Vital="yes" restores the promised fail-fast and the comment now tells the truth (wixtoolset/wix#5974). New-MSI-to-new-MSI upgrades stay untested in CI, but the DeleteService/CreateService 1072 race now fails the install loudly instead of reporting success with no service.
  • WindowsServicePanel treats code 4 as an informational skip; SERVICE_CLEANUP_MSI_MANAGED lives in the cleanup constants block with the javadoc updated.
  • Duplicate refusal message — the native fprintf dropped to debug(); only the localized Java message remains.
  • --cleanupService escape hatch — implemented as suggested: the guard (isMsiManagedService) refuses only when HKLM\SOFTWARE\OpenDJ\InstallDir exists and matches the service command's instance dir, after the same normalization serviceCmdsMatch uses; the value is read with KEY_WOW64_64KEY (the wrapper is 32-bit, the x64 package writes the 64-bit hive). An orphaned service that merely reuses the OpenDJ name is removable again.
  • uninstall.bat ordering — chap-uninstall.adoc now says to run msiexec /x promptly and why (the auto-start service points at deleted files until the package is removed).
  • WixUI_ErrorProgressText — restored.
  • wix lookup mismatch — left as is: the pom comment documents the split, -Pdistribution-windows-msi is the explicit override and release.yml already passes it.

The GUI wizard run (license skip, InstallDirDlg pre-population, and now the refusal dialog) stays planned before merge.

Built by CI (windows-exe-11 artifact of the Build run for 0d88657);
winlauncher.exe and launcher_administrator.exe are unchanged byte-for-byte.
@vharseko
vharseko requested a review from maximthomas August 10, 2026 07:03

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-4 review of 0d886578cd + 1417a25d5d. Most of round 3 is properly closed: the slim-zip filter, the ownership row, the UI-sequence guard, explicit Vital="yes", isMsiManagedService, the constant split, the restored WixUI_ErrorProgressText, the standalone launcher gate and the doc fixes all check out. I confirmed the detect-only <Upgrade> row really does fire alongside the one MajorUpgrade generates — FindRelatedProducts "runs through each record of the Upgrade table in sequence" and appends to each row's own ActionProperty — that its GUID matches the UpgradeCode master's WiX3 package already ships, that UIRef is still core-schema in WiX 5, and that Native launchers is green on the refreshed binaries.

Two of the round-3 fixes don't hold up, both in the same area. Note also that test-msi/test-msi-upgrade have not run on this head, so none of the WiX authoring below is CI-validated yet.

StopLegacyServiceBeforeUpgrade still cannot run before RemoveExistingProducts (blocker)

instance's service and must not be stopped. -->
<CustomAction Id="StopLegacyServiceBeforeUpgrade" Directory="TARGETDIR"
ExeCommand="&quot;[System64Folder]net.exe&quot; stop &quot;OpenDJ Server&quot;"
Execute="deferred" Impersonate="no" Return="ignore"/>
<InstallExecuteSequence>
<Custom Action="RequireDirOnCustomUpgrade" After="SetOpendjFromLegacyDir"
Condition="WIX_UPGRADE_DETECTED AND NOT OPENDJ"/>
<Custom Action="StopLegacyServiceBeforeUpgrade" Before="RemoveExistingProducts"
Condition="LEGACY_MSI_DETECTED"/>
</InstallExecuteSequence>

Execute="deferred" Impersonate="no" does buy elevation, but it also stops the action running where it has to. Per Deferred Execution Custom Actions: "The installer does not execute a deferred execution custom action at the time the installation sequence is processed. Instead the installer writes the custom action into the installation script." RemoveExistingProducts is not scripted — it can legally be sequenced before InstallInitialize, where no script exists, and it "removes the products in sequence by invoking concurrent installations" during the sequence pass itself. So the row at ~1500 is only appended to the script, the nested uninstall at ~1501 runs immediately, and net stop "OpenDJ Server" does not execute until the script does at InstallFinalize — long after the old tree was deleted.

ICE63 describes this authoring exactly. RemoveExistingProducts may be placed "immediately after InstallInitialize, or after InstallInitialize if the actions between InstallInitialize and RemoveExistingProducts do not generate any script actions", and the worked example is a custom action sitting between the two: "If MyCustomAction generates any actions in the script, this causes problems in the installation" / "Failure to fix a warning or error reported by ICE63 leads to failure of the upgrade." A deferred action is script-generating by definition. The RemoveExistingProducts doc carries the same restriction: "After the InstallInitialize action and before any actions that generate execution script."

So this is a regression on round 3, not a fix: the immediate action did run before RemoveExistingProducts on elevated and silent installs (admin console, SCCM, CI) and only no-opped under UAC. The deferred one is too late everywhere — or fails the upgrade outright — and the comment now promises ordering the engine will not deliver. (For the record, the placement is legal in the narrow sense the schedule change was made for: it lands after InstallInitialize, so error 2762 is not reachable. That is not the problem.)

There is no way to have both with an early RemoveExistingProducts: no custom action can precede it and run elevated. Two ways out — revert to the immediate action and say in the comment that it only covers elevated/silent installs, or drop the stop and make the outcome deterministic instead: an immediate action that detects a running OpenDJ Server and refuses the upgrade with "stop the service and re-run" is elevation-independent and cannot half-work.

The legacy-service component still fires on 5.2.x-to-5.2.y upgrades (blocker)

and a repair cannot re-run the removal against a zip instance registered later. -->
<Component Id="RemoveLegacyServiceControl" Directory="OPENDJ"
Guid="3F2A9C41-7D28-4E0B-9B1D-6C54A0D2E7F3" Transitive="yes"
Condition="LEGACY_MSI_DETECTED OR (NOT Installed AND OPENDJ_LEGACY)">
<CreateFolder/>
<ServiceControl Id="RemoveLegacyOpenDJServer" Name="OpenDJ Server"
Stop="install" Remove="install" Wait="yes"/>
</Component>

NOT Installed AND OPENDJ_LEGACY is not "a fresh install adopting the legacy directory", which is what both the comment and the new doc paragraph claim it is. Installed "is set only if the product is installed" — the same ProductCode — so it is empty during every major upgrade as well, and Package has no fixed Id, so each build gets a new ProductCode. NOT Installed means "not a maintenance session", nothing more.

OPENDJ_LEGACY is a bare DirectorySearch for C:\Program Files (x86)\OpenDJ and says nothing about where this install resolves. And test-msi-upgrade asserts the headline path keeps that directory:

$root = "C:\Program Files (x86)\OpenDJ"
# New package files landed in the old directory, not the x64 default
if (-not (Test-Path "$root\setup.bat")) { throw "upgrade did not keep the old install dir" }
if (Test-Path "C:\Program Files\OpenDJ") { throw "upgrade unexpectedly installed into the x64 default dir" }

So on any host that came through a 5.1.x upgrade, the MSI's own install root is the legacy default, OPENDJ_LEGACY is permanently true, and the second disjunct is true on every later 5.2.x-to-5.2.y upgrade regardless of LEGACY_MSI_DETECTED. That is the case round 3 was about: once the MSI holds key OpenDJ, a second zip instance's windows-service.bat --enableService starts its loop at OpenDJ Server, finds it free and takes it (createServiceName/serviceNameInUse compare key names only), and the next MSI upgrade stops and deletes that unrelated instance's service. Transitive="yes" does not help — it re-evaluates on reinstall of the same ProductCode, and a major upgrade is a fresh install.

This also makes the new sentence in the install guide wrong for exactly that host:

======
When the installer replaces a legacy OpenDJ installation (an upgrade from a pre-5.2.0 package, or a fresh install into a detected legacy default directory), it stops and removes the service registered by the legacy `windows-service.bat` command, replacing it with the installer-managed `OpenDJ` service. Services belonging to other server instances are not touched: upgrades between 5.2.0-or-later packages and repairs leave any `OpenDJ Server` service in place. Because the installer-managed service is registered with automatic start, Windows logs a start failure at every boot until the instance has been configured with `setup`.
======

Adding AND NOT OPENDJ_REG to the second disjunct would mirror SetOpendjFromLegacyDir's own condition and limit it to installs that genuinely are adopting the legacy directory.

Nits

  • isMsiManagedService fails open. If getServiceList errors, or getBinaryPathName failed for the OpenDJ entry so cmdToRun is NULL, the loop matches nothing and managed stays FALSE — round 3's name compare could not fail. uninstall.bat --remove-all would then delete the MSI's own service, which is what the guard exists to prevent. Treating an SCM-query failure as "managed" keeps the escape hatch without the fail-open.

}
free(serviceList);
}
}
else
{
debug("isMsiManagedService: could not get service list.");
}
return managed;
} // isMsiManagedService

  • WindowsServicePanel now claims success. Code 4 takes the FINISHED_SUCCESSFULLY branch, so the dialog closes on INFO_CTRL_PANEL_DISABLING_WINDOWS_SERVICE_SUCCESSFUL_SUMMARY — "The Windows service was successfully disabled" — directly contradicting the INFO_WINDOWS_SERVICE_MSI_MANAGED line printed just above it. Reusing the "already disabled" summary, or a third state, would read correctly.

{
returnCode = ConfigureWindowsService.disableService(outPrintStream, errorPrintStream);
// SERVICE_MSI_MANAGED is an informational skip, not an error: the
// MSI-managed service belongs to the installer and is removed by
// msiexec /x (disableService already printed the explanation).
if (returnCode != ConfigureWindowsService.SERVICE_ALREADY_DISABLED
&& returnCode != ConfigureWindowsService.SERVICE_DISABLE_SUCCESS
&& returnCode != ConfigureWindowsService.SERVICE_MSI_MANAGED)
{
state = State.FINISHED_WITH_ERROR;
}
else
{
state = State.FINISHED_SUCCESSFULLY;
}

  • The launcher gate is blind to untracked files. git diff --exit-code does not report a file that was never git added, so a newly added launcher that nmake builds and xcopy drops into lib/ passes the check while never shipping. git status --porcelain -- opendj-server-legacy/lib closes it.

xcopy /Y *.exe ..\..\..\lib\ || exit /b 1
- name: Committed launcher binaries must match the sources
shell: bash
run: |
if ! git diff --exit-code -- opendj-server-legacy/lib; then
echo "::error title=Stale launcher binaries::opendj-server-legacy/lib/*.exe do not match the sources. Refresh them from the windows-exe artifact of a Build run on this branch and commit them."
exit 1
fi

  • No concurrency: group in the new workflow; build.yml, codeql.yml and benchmark.yml all set one, so superseded pushes will run the Windows job to completion.

… guard

- StopLegacyServiceBeforeUpgrade back to Execute="immediate": a deferred
  action is only written into the installation script (ICE63 forbids
  script-generating actions between InstallInitialize and
  RemoveExistingProducts) and would run long after the old tree is gone.
  Immediate covers elevated/silent installs; a UAC double-click no-ops and
  RemoveLegacyServiceControl still stops and removes the service in the
  script, so a missed stop only costs files held until reboot.
- RemoveLegacyServiceControl: AND NOT OPENDJ_REG on the fresh-install
  disjunct - NOT Installed is also true during every major upgrade, and on
  hosts upgraded from 5.1.x the install root IS the legacy directory, so
  the component would otherwise fire on every 5.2.x-to-5.2.y upgrade and
  delete an unrelated zip instance's service. Install guide wording aligned.
- isMsiManagedService fails closed: an SCM query failure or unreadable
  command line now counts as MSI-managed instead of orphaned.
- Control panel no longer reports the MSI-managed skip as "Windows Service
  Disabled": the task overrides the success summary with "Windows Service
  Left Enabled" plus the INFO_WINDOWS_SERVICE_MSI_MANAGED explanation.
- Launcher gates use git status --porcelain (catches untracked binaries);
  concurrency group added to check-native-launchers.yml.
@vharseko

Copy link
Copy Markdown
Member Author

Thanks for the fourth pass — both blockers reproduced, and the deferred-action analysis is airtight: ICE63's worked example is literally this authoring. Addressed in 461644a; notes follow your headings.

StopLegacyServiceBeforeUpgrade — took the first way out: reverted to the immediate action, and the comment now states the real coverage (elevated consoles, SCCM/SYSTEM, CI) and the real failure mode (a UAC double-click gets access-denied, swallowed by Return="ignore"). Chose revert over refuse-with-guidance because the degradation is benign: RemoveLegacyServiceControl still stops and removes the service elevated during script execution, so a missed stop only costs the locked files going to delayed delete until reboot — the WiX3-era behavior — while a refusal would add a manual step to every interactive legacy upgrade with a running server. The MajorUpgrade comment no longer promises the deferred ordering; Schedule="afterInstallInitialize" stays, now justified by what it actually buys (removal inside the installation transaction, so a failed upgrade restores the old product).

Legacy-service component — added AND NOT OPENDJ_REG to the second disjunct, exactly mirroring SetOpendjFromLegacyDir. You were right that the old comment argued the wrong polarity: "permanently true on any host after the first install" is precisely the property that makes it the correct negative guard — once this package has recorded InstallDir, "OpenDJ Server" can only be an unrelated zip instance. The comment now spells out the 5.1.x-upgraded-host scenario, and the install-guide sentence is qualified accordingly ("on a host where no 5.2.0-or-later package has recorded its installation location").

Nits

  • isMsiManagedService fails closed: an SCM list failure, or a matching entry whose command line could not be read, now counts as MSI-managed. The escape hatch (an orphaned name-squatter with a readable, non-matching path) is unchanged.
  • The control-panel dialog no longer claims success: code 4 shows "Windows Service Left Enabled" with the INFO_WINDOWS_SERVICE_MSI_MANAGED explanation as the detail. Implementation note: launchOperation's success messages are fixed at launch time, so instead of a third Task.State (which would have needed a third message set threaded through the 9-arg static), the task can now override the success summary/detail and launchOperation prefers the override — no other panel is affected.
  • Both launcher gates (the standalone workflow and build.yml's warning-only step) now check git status --porcelain -- opendj-server-legacy/lib, so an untracked freshly built launcher fails/warns too.
  • concurrency: group added to check-native-launchers.yml, same shape as build.yml's.

service.c changed again (the fail-closed guard), so the committed launchers are pending one more refresh from the next Build run's windows-exe-11 artifact — until then Native launchers is red by design, as before. And agreed on CI: test-msi/test-msi-upgrade need a green run on this head before this round can be called closed.

@vharseko
vharseko requested a review from maximthomas August 10, 2026 10:27

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both round-4 blockers are fixed, and this round I checked them against the MSI CI actually built rather than the authoring — build-maven (windows-latest, 11) is green, so I pulled opendj-5.2.0-SNAPSHOT.msi from the windows-latest-11 artifact and read its tables. Confirmed: InstallInitialize 1500 → StopLegacyServiceBeforeUpgrade 1501RemoveExistingProducts 1502; CustomAction.Type = 98 (immediate, synchronous, exit code ignored); the component condition verbatim with Attributes = 320 (64-bit + Transitive); SecureCustomProperties carrying all four properties; ServiceInstall.ErrorControl = 32769 (Vital); the detect-only Upgrade row at Attributes = 258. Your point that REP removes the old files during the sequence pass is also right.

Two blockers remain: one is the round-4 root cause one step further out, the other is a side effect of the round-4 fix itself.

The legacy service is stopped and deleted without ownership evidence (blocker)

opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs

Condition="LEGACY_MSI_DETECTED OR (NOT Installed AND OPENDJ_LEGACY AND NOT OPENDJ_REG)"

NOT OPENDJ_REG correctly closed the 5.2.x→5.2.y case. But neither disjunct establishes that OpenDJ Server belongs to the tree being replaced.

OPENDJ_LEGACY is a bare DirectorySearch — it proves only that Program Files (x86)\OpenDJ exists, not that this install is going there. So msiexec /i opendj.msi OPENDJ="C:\path\to\opendj" on any host where that directory happens to exist stops and deletes OpenDJ Server while installing elsewhere. That is the command line RequireDirOnCustomUpgrade itself prints. An empty leftover legacy directory is enough.

LEGACY_MSI_DETECTED has the same gap. The pre-5.2.0 package has zero ServiceInstall/ServiceControl (checked at master, 5.1.0, 5.1.1, 5.1.2, 4.10.2), so the service came from windows-service.bat --enableService. Fair enough — the MSI stages bat\windows-service.bat and the old install guide told users to run it, so the assumption usually holds. It just isn't proof, and StopLegacyServiceBeforeUpgrade acts on the same signal.

Worth blocking on because OpenDJ Server is not an edge-case name: createServiceName starts at the bare baseName and only appends -2, -3 on collision, and baseName resolves to DynamicConstants.PRODUCT_NAME = <name>OpenDJ Server</name>. It is what the first zip instance on a host claims. That also makes chap-install.adoc:647 — "Services belonging to other server instances are not touched" — a promise the condition can't keep.

Before trying the obvious fix: AND NOT OPENDJ doesn't repair the branch, it deletes it. SetOpendjFromLegacyDir is a type 51 SetProperty at sequence 53, so OPENDJ is already non-empty whenever that disjunct is live; with MSI precedence NOT > AND > OR you'd get LEGACY_MSI_DETECTED OR (dead).

The cheapest correct fix is pure authoring — the service records where it points, so ask it:

<Property Id="LEGACY_SVC_IMAGEPATH" Secure="yes">
  <RegistrySearch Id="LegacySvcImagePath" Root="HKLM"
                  Key="SYSTEM\CurrentControlSet\Services\OpenDJ Server"
                  Name="ImagePath" Type="raw"/>
</Property>

then gate both the component and the net stop on LEGACY_SVC_IMAGEPATH ~>< OPENDJ_LEGACY (case-insensitive "contains"). Unset property → both actions correctly do nothing. Worth checking the value shape against what createServiceName writes ("...\lib\opendj_service.exe" start "C:\dir."); 8.3 or relative paths would want ~<< on a normalised prefix.

I should correct myself from round 4: the C code can not already do this. isMsiManagedService name-gates on MSI_SERVICE_NAME = "OpenDJ" (opendj-server-legacy/src/build-tools/windows/service.h:66), takes its reference directory from HKLM\SOFTWARE\OpenDJ\InstallDir (absent on the legacy path by definition), and is static with no subcommand exposing it. Only the primitives exist.

One more constraint: transitive component conditions must be "based only on computer states", not "properties set on the command line". All three here are Secure="yes", so msiexec /i opendj.msi LEGACY_MSI_DETECTED=1 forces the deletion regardless of whatever fix the disjunct gets.

net stop cannot elevate in the ordinary interactive install (blocker)

opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs

<CustomAction Id="StopLegacyServiceBeforeUpgrade" Directory="TARGETDIR"
              ExeCommand="&quot;[System64Folder]net.exe&quot; stop &quot;OpenDJ Server&quot;"
              Execute="immediate" Return="ignore"/>

Execute="immediate" is what gets the action ahead of RemoveExistingProducts — and it is also what stops it from ever elevating. Immediate custom actions impersonate the invoking user (Custom Action Security; ICE68 forbids NoImpersonate on immediate actions). Only the deferred script gets the elevated token.

So on a per-machine install started the normal way — double-click the .msi, approve UAC — the action runs under the filtered token, where BUILTIN\Administrators is deny-only. The default service security descriptor grants SERVICE_STOP to Administrators and LocalSystem and gives authenticated users query rights only, and nothing in service.c calls SetServiceObjectSecurity. net stop returns "System error 5", and Return="ignore" swallows it.

Flow net stop
Double-click .msi, UAC consent fails silently
msiexec /i from a non-elevated shell fails silently
Standard user + over-the-shoulder admin credentials fails silently
Elevated shell, SCCM/Intune, /qn from CI works

The last row is why test-msi-upgrade can't see this — the Actions runner holds a full unfiltered admin token.

Two consequences. The mechanism that made the ordering fix meaningful is inoperative in the flow most admins use. And since RemoveLegacyServiceControl runs in the deferred elevated script, in exactly that flow the harmless action is the one that gets blocked while the destructive one still succeeds.

Execute="deferred" Impersonate="no" is not the answer — that is the round-4 bug, since the script doesn't run until InstallFinalize. But detection works unelevated (SERVICE_QUERY_STATUS is in the authenticated-users grant), so an immediate CA can reliably tell whether OpenDJ Server is running and refuse with "stop the service, or run the installer elevated" instead of proceeding on a stop that silently didn't happen.

The "benign degradation" justification doesn't hold as written (major)

Choosing revert-over-refuse rests on a missed stop costing only "old files held until reboot". That is the documented common case, and it has an unstated precondition: the locked file must be renamable. Per Logging of Reboot Requests the in-use file "is first renamed with an .rbf extension and then deleted", so the delete-on-reboot targets Config.Msi\*.rbf, never the original path. The team's own post documents the fallback: "If the move to the rollback directory is blocked (by hard locking) the PendingFileRenameOperations hive is used."

Renaming an open file requires the holder to have opened it with FILE_SHARE_DELETE. Running images get that from the loader — which is why the benign case is the one everyone sees. JVM-held jars do not: since JDK 9 ZipFile.Source opens through new RandomAccessFile(f, "r")winFileHandleOpen, and io_util_md.c hardcodes FILE_SHARE_READ | FILE_SHARE_WRITE, byte-identical in 11u/17u/21u/master. JDK-8224794 is Closed/Won't Fix, so this is permanent, not a pending fix. lib\*.jar under a running OpenDJ are exactly the hard-lock case.

And the fallback MS documents is the overwrite case (temp→original, nets out). This is a removal, inside REP's nested uninstall, whose only primitive is MoveFileEx(path, NULL, MOVEFILE_DELAY_UNTIL_REBOOT) — naming the original path. The nested uninstall reaches its own InstallFinalize before the parent's InstallFiles (4000) runs, and I can find no documented mechanism cancelling the queued delete. If that's what happens, the failure mode isn't "held until reboot", it's upgrade reports success, next reboot deletes the new jars. Unverified in both directions — MS documents neither — but the burden sits with the change that chose to ignore the failure.

What actually covers the interactive case is Restart Manager, not the .rbf rename, and more weakly than it sounds: the jar handles belong to the java.exe grandchild that opendj_service.exe starts via start-ds.bat, which RM classes RmUnknownApp ("can only be shut down by a forced shutdown"). RM most likely rescues this only indirectly, by matching lib\opendj_service.exe and stopping the service through the SCM.

Either fix the comment, or settle it in one run — start the legacy service, upgrade by double-clicking the MSI as a filtered admin, and read PendingFileRenameOperations before rebooting. An entry naming lib\*.jar rather than a Config.Msi\*.rbf path falsifies the claim.

Nothing tests the condition that has now been wrong twice (major)

.github/workflows/build.yml

RemoveLegacyServiceControl deletes a Windows service that may belong to someone else's instance. Its condition was wrong in round 3 (OPENDJ_REG positive), wrong again in round 4 (missing NOT OPENDJ_REG), and is correct-by-inspection now. test-msi-upgrade covers the 5.1.2 upgrade, legacy-dir autodetect, registry detection and the refuse path — but never a new-MSI→new-MSI upgrade and never a repair.

A true major upgrade needs a second ProductCode and the job has no checkout. A repair doesn't, and it exercises the Transitive="yes" claim directly. After the existing upgrade step:

sc.exe create "OpenDJ Server" binPath= "C:\zip-instance\lib\opendj_service.exe start ""C:\zip-instance.""" start= demand
$p = Start-Process msiexec -Wait -PassThru -ArgumentList "/i `"$msi`" REINSTALL=ALL REINSTALLMODE=vomus /quiet /qn /norestart /l*v repair.log"
if ($p.ExitCode -ne 0) { Get-Content repair.log -Tail 80; throw "repair failed: $($p.ExitCode)" }
if (-not (Get-Service "OpenDJ Server" -ErrorAction SilentlyContinue)) { throw "repair deleted an unrelated instance's service" }
sc.exe delete "OpenDJ Server"

That is the assertion the last two rounds were about, and it would have caught the round-4 defect. (Repair is safe today for a stronger reason than the comment gives: FindRelatedProducts does not run in maintenance mode at all.)

Nits

  • The new fail-closed branch is mostly unreachable from --disableService: removeService resolves the name through getServiceName first, which returns 1 when enumeration fails → mapped to SERVICE_ALREADY_DISABLED → treated as success by both the panel and the uninstaller. So on the exact condition the change guards against, isMsiManagedService is never consulted. Nothing is deleted, so it's safe, but the user is told the service was already disabled. The branch earns its keep for --cleanupService and the cmdToRun == NULL case.
  • OPENDJ_REG is existence-validated: Type="directory" means AppSearch leaves the property unset if the recorded directory is renamed or on an offline volume, so NOT OPENDJ_REG silently becomes true on a machine that is properly registered. The comment above it still says "fresh installs only", which no longer matches the condition.
  • REG_EXPAND_SZ is accepted but never expanded: the registry read tolerates the type without calling ExpandEnvironmentStrings, so a path containing %ProgramFiles% compares literally.
  • New messages are English-only: INFO_CTRL_PANEL_WINDOWS_SERVICE_MSI_MANAGED_SUMMARY and its detail string land in admin_tool.properties but not in the localized bundles.
  • The ICE63 rule the comment cites is unenforced: the pom drives wix build via exec-maven-plugin, and wix build runs no ICE validation at all — only MSBuild/.wixproj or wix msi validate do. A green build proves the authoring compiles, not that it validates.

…a running legacy service

- LEGACY_SVC_IMAGEPATH (raw registry search of the legacy service's own
  ImagePath) gates RemoveLegacyServiceControl and both custom actions:
  the "OpenDJ Server" service - the name the first zip instance on a host
  claims - is only ever stopped or removed when it provably points into
  [OPENDJ], the directory this install resolves to.
- The immediate net stop cannot elevate under a filtered UAC token, so a
  CheckLegacyServiceStopped action (PowerShell state query, needs no
  elevation) now refuses the upgrade with error 1722 while the owned
  legacy service is still running, instead of proceeding into REP with
  locked jars (JVM opens them without FILE_SHARE_DELETE, JDK-8224794).
- test-msi-upgrade: the headline upgrade now runs with the legacy service
  RUNNING (exercises stop+check elevated), plus a repair scenario and a
  fresh-install-elsewhere scenario assert the component leaves an
  unrelated instance's service alone.
- OPENDJ_REG search Type raw: existence validation dropped the value for
  a renamed/offline directory, silently un-registering the host.
- removeService reports an error (exit 3) when the SCM cannot be
  enumerated instead of mapping to "already disabled";
  isMsiManagedService expands REG_EXPAND_SZ InstallDir values.
- Both MSI-managed messages localized into the nine bundles.
- New Validate the MSI (ICE) step: wix build runs no ICE validation.
@vharseko

Copy link
Copy Markdown
Member Author

Thanks for the fifth pass — reading the built MSI's tables rather than the authoring is exactly the right escalation, and both blockers plus both majors reproduced. Addressed in 0b19faf; notes follow your headings.

Ownership evidence — took your registry-search fix, with one refinement: the gate is LEGACY_SVC_IMAGEPATH ~>< OPENDJ (the directory this install resolves to), not ~>< OPENDJ_LEGACY. That covers the legacy upgrade at a custom directory too — there the service points at C:\custom and must still be replaced, or the upgraded tree ends up with two auto-start services — and it makes the fresh-install-elsewhere case fall out naturally (ImagePath can't contain a directory the install isn't going to). Both custom actions and the component carry the same gate. On value shape: [OPENDJ] from either search ends with a backslash, so the substring match is a real prefix test; an explicit command-line OPENDJ without one could prefix-match a sibling (C:\dj vs C:\dj2) — noted in the comment as an accepted residual, since a legacy-era signal must also hold. 8.3/relative forms fail the match and fail safe (nothing is touched). And thanks for the correction on the C side — agreed, only the primitives exist there; this stays pure authoring. On the LEGACY_MSI_DETECTED=1 command-line forcing: acknowledged as inherent to Secure properties; an admin who passes it is explicitly asking.

net stop cannot elevate interactively — took your detect-and-refuse, composed with the stop: best-effort immediate net stop first, then an immediate CheckLegacyServiceStopped (powershell -Command "if ((Get-Service 'OpenDJ Server' ...).Status -eq 'Running') { exit 1 }; exit 0", Return="check") that fails the upgrade with 1722 while the owned legacy service is still running. Elevated/SCCM/CI flows stop the service and sail through; the filtered-token flows get a deterministic refusal instead of a silent half-upgrade. PowerShell rather than sc | findstr so the exit code needs no parsing of localized output (and no square brackets — ExeCommand is a Formatted field). The friendly-message variant (a property-setting script CA plus a type-19 error) lost to VBScript's deprecation; the 1722 names the action and the install guide carries the guidance.

The degradation justification — conceded, and mooted rather than verified: with the refusal in place no upgrade proceeds against a running owned service, so the .rbf-vs-PendingFileRenameOperations question no longer has a reachable path. The comment now states the real mechanics (JVM jars lack FILE_SHARE_DELETE, JDK-8224794 Won't Fix) as the reason the check exists.

Nothing tests the condition — your repair scenario is in test-msi-upgrade nearly verbatim, plus a second new scenario: leftover legacy directory + zip-instance OpenDJ Server + OPENDJ=C:\opendj-elsewhere fresh install must leave the service alone (the exact round-5 case — it fails on the pre-fix condition). The headline upgrade also now leaves the legacy service running, so the elevated stop+check pair is exercised end-to-end rather than upgraded-from-a-stopped-state. The Transitive comment now cites the stronger reason (FindRelatedProducts does not run in maintenance at all).

Nits

  • removeService now distinguishes an SCM enumeration failure (new SERVICE_LIST_UNAVAILABLE) and exits 3 — mapped to the disable error — instead of 1's "already disabled". The other getServiceName callers treat the new code exactly as they treated the generic error.
  • OPENDJ_REG is Type="raw"; the comment explains why existence validation was wrong for a Remember-Property, and the stale "fresh installs only" phrasing is fixed.
  • isMsiManagedService expands REG_EXPAND_SZ via ExpandEnvironmentStrings, failing closed if expansion fails.
  • Both messages are now in all nine localized bundles (de/es/fr/ja/ko/pl/ca_ES/zh_CN/zh_TW).
  • New Validate the MSI (ICE) step runs wix msi validate on the built package in the windows/11 job — you're right that a green wix build proved nothing; if the first run surfaces pre-existing ICE noise I'll triage it there.

service.c/service.h changed again, so the committed launchers are pending one more refresh from this run's windows-exe-11 artifact — Native launchers is red by design until then, same drill as before. test-msi/test-msi-upgrade (with the three new/changed scenarios) and the ICE step all need a green run on this head before round 5 can close.

@vharseko
vharseko requested a review from maximthomas August 10, 2026 12:45

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed round 5 (0b19faf) along with the linked issue/PR history. The ImagePath ownership proof and the fail-closed guards are a real improvement over round 4. Two things block a merge, a few others look worth fixing before this lands.

MSI installs unconditionally become service installs (blocking)

ServiceInstall in opendj-packages/opendj-msi/opendj-msi-standard/resources/msi/package.wxs:232 is unconditional, so every msiexec /i registers the service:

<ServiceInstall Id="OpenDJService" Name="OpenDJ" DisplayName="OpenDJ"
                Type="ownProcess" Start="auto" ErrorControl="normal" Vital="yes"
                Arguments="start &quot;[OPENDJ].&quot;"/>

The new serviceCmdsMatch matches that ImagePath, so ConfigureWindowsService.serviceState() reports ENABLED and DirectoryServer.checkStartupCanProceed() returns START_AS_WINDOWS_SERVICE on every MSI install. Consequences for an admin who ran setup.bat --cli --doNotStart and never asked for service mode:

  • start-ds.bat no longer starts the server — it dispatches to StartWindowsService and needs elevation;
  • start-ds.bat -N/--nodetach hard-fails with ERR_DSCORE_ERROR_NODETACH_AND_WINDOW_SERVICE;
  • stop-ds.bat / ShutdownTask reroute through the SCM.

CI only exercises net start OpenDJ, never start-ds.bat, and the install-guide changes don't mention this. Either gate registration behind a property/feature, or make the semantics change explicit and cover it in test-msi.

Stale committed launchers can ship in a release (blocking)

.github/workflows/check-native-launchers.yml:24-33 only triggers on changes under opendj-server-legacy/src/build-tools/windows/** or opendj-server-legacy/lib/*.exe, and release.yml has no equivalent of the (warning-only) comparison in .github/workflows/build.yml:94. This PR is being merged with the committed launchers stale and Native launchers red by design — so any release cut between this merge and the follow-up refresh commit runs neither gate and ships an opendj_service.exe without the MSI-managed guard, the fail-closed ownership check or SERVICE_LIST_UNAVAILABLE. In that build windows-service.bat --disableService deletes the MSI's own service, while CI proved it doesn't. Please land the rebuilt binaries in this PR, or add the parity gate to release.yml first.

SERVICE_LIST_UNAVAILABLE handled in only one of four callers (major)

opendj-server-legacy/src/build-tools/windows/service.c:1169 introduces the code, but grep finds exactly one consumer (removeService, line 2806). serviceState() still collapses it into "disabled":

code = getServiceName(cmdToRun, serviceName);
if (code == SERVICE_RETURN_OK) { ... }
else { returnCode = 1; debug("Service '%s' is disabled.", serviceName); }  // service.c:2579

So when getServiceList() fails (SCM busy, access denied under a filtered token) ConfigureWindowsService.serviceState() returns SERVICE_STATE_DISABLED instead of SERVICE_STATE_ERROR: --serviceState prints INFO_WINDOWS_SERVICE_DISABLED, Uninstaller.java:1352 skips disabling the service entirely, ControlPanel offers Enable for a registered service, and isRunningAsWindowsService() returns false so stop-ds kills the JVM behind the SCM's back. startService() and serviceMain() have the same gap. This is the "already disabled" lie round 5 set out to remove, left in place for three of the four callers.

isMsiManagedService fails open on a hardcoded registry key (major)

service.c:2627 hardcodes the key while the package writes it under $(var.name):

if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, "SOFTWARE\\OpenDJ", 0, ...) != ERROR_SUCCESS)
{
  debug("... no HKLM\\SOFTWARE\\OpenDJ key, treating '%s' as orphaned.", serviceName);
  return FALSE;   // <- the only fail-OPEN path in this function
}
<RegistryValue Root="HKLM" Key="SOFTWARE\$(var.name)" Name="InstallDir" .../>

opendj-msi-standard/pom.xml passes -d name=${product.name}, so a rebranded build records InstallDir under SOFTWARE\<newname> while MSI_SERVICE_NAME stays the literal "OpenDJ". The lookup then fails and --disableService / --cleanupService cheerfully delete the MSI's own service — exactly what the guard exists to prevent. Every other failure path here is fail-closed; make this one match (read the name from the service key, or drive it from the same variable).

The legacy ImagePath is REG_EXPAND_SZ, not REG_SZ (medium)

package.wxs:65-76:

<!-- ... written by windows-service.bat as REG_SZ, which raw returns unmangled -->
<RegistrySearch Id="LegacySvcImagePathSearch" Root="HKLM"
                Key="SYSTEM\CurrentControlSet\Services\OpenDJ Server"
                Name="ImagePath" Type="raw"/>

createServiceInScm goes through CreateService, and the SCM stores ImagePath as REG_EXPAND_SZ. A raw RegistrySearch returns that unexpanded with a #% prefix, so LEGACY_SVC_IMAGEPATH is #%"C:\Program Files (x86)\OpenDJ\lib\opendj_service.exe" start .... It only works today because ~>< is a substring test — and the comment says otherwise. The documented follow-up it invites (tighten C:\dj vs C:\dj2 into a prefix test << or an equality check) would then match nothing and silently leave the legacy auto-start service registered against the upgraded tree.

A relocating legacy upgrade orphans the legacy service (medium)

msiexec /i new.msi OPENDJ=C:\opendj-new over a 5.1.2 at C:\opendj-old: RequireDirOnCustomUpgrade doesn't fire (OPENDJ is set), RemoveExistingProducts empties C:\opendj-old, but LEGACY_SVC_IMAGEPATH ~>< OPENDJ is false for both custom actions and RemoveLegacyServiceControl (package.wxs:281). The legacy service is never stopped or removed and keeps auto-starting a deleted lib\opendj_service.exe at every boot, with config/db/logs orphaned at the old path. Not covered by CI.

The GUI upgrade path dead-ends (medium)

package.wxs:150 schedules the refusal in InstallUISequence right after AppSearch:

<InstallUISequence>
  <Custom Action="RequireDirOnCustomUpgrade" After="SetOpendjFromLegacyDir"
          Condition="WIX_UPGRADE_DETECTED AND NOT OPENDJ"/>
</InstallUISequence>

Upgrading a 5.1.x installed at a custom directory by double-clicking the .msi aborts before WelcomeDlg/InstallDirDlg with a message offering only a msiexec ... OPENDJ="..." command line — even though WixUI_InstallDir ships the browse dialog that would resolve it. A GUI-only administrator can't complete the upgrade. Restricting the row to InstallExecuteSequence (or gating it on UILevel < 5) keeps the silent-install guarantee without the dead end.

Nits

  • ICE61 will always fire: AllowSameVersionUpgrades="yes" (package.wxs:40) is documented to require -sice:ICE61, but .github/workflows/build.yml:153 runs bare wix msi validate "$msi" — permanent noise the step can't distinguish from a real regression. Suggest wix msi validate -sice:ICE61 "$msi".
  • Three SCM enumerations on --disableService: removeServicegetServiceName (1), isMsiManagedService (2, new, service.c:2735), serviceNameInUse (3), each doing an OpenService+QueryServiceConfig per service on the host. winlauncher.c run() gives up at 30 s and returns 259, which disableService() maps to ERR_WINDOWS_SERVICE_DISABLE_ERROR even on success. isMsiManagedService could take the entry from the list getServiceName already walked.
  • Refusal scenario depends on the previous step's cleanup: .github/workflows/build.yml:924 never does Remove-Item HKLM:\SOFTWARE\OpenDJ (the next step does), so it relies on the prior uninstall having removed InstallDir. If that value survives, the guard under test is bypassed and the step fails for the wrong reason.
  • isMsiManagedService duplicates serviceCmdsMatch: service.c:2686-2697 re-tokenizes exe/subcommand/instance-dir and re-runs normalizeInstanceDir+expandLongPath, duplicating service.c:1082-1096. Any future ImagePath shape change has to be made twice; miss one and the guard becomes a no-op or a permanent refusal. A shared serviceInstanceDir(const char* cmd, char* out) helper removes it.
  • Hand-rolled path canonicalisation: normalizeInstanceDir (service.c:1022) + expandLongPath reimplement, less completely (.. and forward slashes are not collapsed), what common.c:469 getCanonicalDirectoryPath already does via GetFullPathName. Two normalizers over the same strings is exactly the drift the ownership guard can't afford.
  • Workflow duplication: .github/workflows/check-native-launchers.yml:47-70 is byte-for-byte build.yml:70-102 apart from ::error vs ::warning, pinned action SHA and FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 workaround included. A composite action under .github/actions/ keeps the warning and the blocking gate from drifting apart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build CI docs enhancement java Pull requests that update java code packaging deb/rpm/MSI, distribution layout, config.ldif security Security fixes / CodeQL code-scanning alerts Windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants