Isolate build history per output directory, redux - #4376
Merged
Conversation
This was referenced Jun 26, 2026
Contributor
Author
|
@behrmann @daandemeyer friendly review ping? Thanks! |
behrmann
reviewed
Jul 1, 2026
behrmann
left a comment
behrmann
left a comment
Contributor
There was a problem hiding this comment.
Thanks! Sorry, had absolutely forgotten about having to look at this.
martinpitt
force-pushed
the
history-output
branch
from
July 3, 2026 05:00
e095e55 to
a487b8f
Compare
Contributor
|
In the light of #4381 can you drop the |
behrmann
approved these changes
Jul 3, 2026
…tput dir A build can be told its output directory on the command line (`-O`) while a later verb that consumes the build (vm, boot, summary, …) only knows it from `OutputDirectory=` in the configuration. As long as both point at the same directory, the consumer must still find the build history the build wrote. This is what systemd does: its meson build passes `--output-dir` explicitly, while a developer running `mkosi vm` (without `--output-dir`) afterwards relies on `OutputDirectory=`. Capture this behaviour so we don't regress it again. This broke once in commit da49fe9 ("Put build history into the output directory"), which keyed the history location on the CLI value only, and was reverted in commit 582eade.
When building into separate output directories (e.g. the integration test suite running in parallel), all builds shared a single build history in the config directory. A verb that consumes a previous build (vm, boot, summary, …) then read back whichever build ran last instead of the one for the output directory it was pointed at. To fix that and keep an output directory isolated, store the history in it as well, and prefer it when given `--output-directory`. The config directory copy is kept and is still used when no `-O` is passed, so a consumer that gets the output directory from `OutputDirectory=` in the configuration keeps working (and the output-dir lookup falls back to it, status quo ante). This is what commit da49fe9 tried to do, but it stored the history *only* in the output directory keyed on the CLI value, which broke consumers that take the output directory from the configuration. Keying the read on the CLI value and keeping the config-dir copy avoids that regression. Cover this in the new `test_history_found_via_configured_output_directory()`.
The integration tests build into a per-test `--output-directory`, but the verbs that consume the build (vm, boot) did not pass it, so they recovered the build's configuration from the history in the config directory. This breaks parallel tests, as they read the history file from current global file (i.e. whichever build happened to finish last). This is the tests/__init__.py half of the reverted commit da49fe9. (That wasn't problematic.)
martinpitt
force-pushed
the
history-output
branch
from
July 3, 2026 07:50
a487b8f to
b3ec537
Compare
bluca
approved these changes
Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The original fixes for parallel tests were in #4360, but one of the key commits broke system and was reverted in #4373. I examined the regression in #4373 (comment) , captured the regression (and systemd-desired behaviour) in a unit test, and reworked the fix to not break existing behaviour (again including a unit test).
I already pushed these commits to #4357 to test them with barrage and actual parellelism, and they work well enough (i.e. sans the opensuse mirror and
apkdownload woes).I also tested this on systemd: this commit on my fork switches mkosi over to this branch, and the tests work in the sense of "all image builds pass". There are some individual test failures, but they also happen in e.g. systemd/systemd#42755
Details are in the individual commit messages.