Skip to content

fix: run ocamldep for single-module stanzas (#4572)#14079

Closed
robinbb wants to merge 10 commits intoocaml:mainfrom
robinbb:robinbb-issue-4572-singleton-fix
Closed

fix: run ocamldep for single-module stanzas (#4572)#14079
robinbb wants to merge 10 commits intoocaml:mainfrom
robinbb:robinbb-issue-4572-singleton-fix

Conversation

@robinbb
Copy link
Copy Markdown
Contributor

@robinbb robinbb commented Apr 7, 2026

Run ocamldep for single-module stanzas so that per-module library dependency
filtering works for all stanza sizes.

Dune previously skipped ocamldep for singleton stanzas as an optimization
(no intra-stanza deps to compute). This prevented the per-module filtering
in #14021 from working, since it relies on ocamldep output to determine
which libraries a module references. Without the data, single-module
stanzas fall back to all-library glob deps via the can_filter guard
(the dummy dep graph has dir = Path.Build.root, which doesn't match
Obj_dir.dir). Correctness is maintained, but the optimization doesn't
apply.

Removing the singleton shortcut enables filtering for single-module stanzas.
The cost is negligible: one extra ocamldep invocation per single-module stanza.

Depends on #14021.
Ref: #4572

@rgrinberg
Copy link
Copy Markdown
Member

I don't quite get the motivation behind this PR. It might be true that this optimization does not fire for single module libraries, but consider the case where it might fire: there must be libraries referenced in libraries that are not present in the only source file. Thus, those libraries are just unused and can be removed from the libraries field altogether. Conveniently, we already have a check to detect unused libraries. I suppose we can introduce a warning here as well?

@robinbb robinbb force-pushed the robinbb-issue-4572-singleton-fix branch from 91cd5db to cc2ba16 Compare April 8, 2026 00:00
@robinbb
Copy link
Copy Markdown
Contributor Author

robinbb commented Apr 8, 2026

Thanks for the feedback @rgrinberg. I think the value here might be different
from what the PR description conveyed, so let me clarify.

The scenario in the test isn't about unused libraries — it's about unused
modules within a used library. The consumer uses_alpha genuinely depends on
base_lib and references Alpha. But base_lib also contains Unused, and
modifying Unused's interface triggers recompilation of uses_alpha even
though uses_alpha never references Unused.

@unused-libs wouldn't flag this because the library IS used — the consumer
references one of its modules. The issue is that changing a different module
in the same library causes unnecessary recompilation. This is the core problem
described in #4572 for unwrapped libraries: per-module granularity within a
library dependency.

For multi-module stanzas, #14021 already handles this — the per-module
filtering reads ocamldep output and declares deps only on the specific library
modules each stanza module references. But for single-module stanzas, dune
skips ocamldep entirely (the singleton optimization in dep_rules.ml),
producing no .d files. Without ocamldep data, the filtering falls back to
glob deps on the entire library, and changing any module in the library
triggers recompilation.

Removing the singleton shortcut makes ocamldep run for all stanzas, giving
the filtering the data it needs. The cost is one extra ocamldep invocation per
single-module stanza — negligible compared to the compilation it avoids.

Let me know if I have misunderstood.

@robinbb robinbb force-pushed the robinbb-issue-4572-singleton-fix branch from cc2ba16 to fa3d4c3 Compare April 8, 2026 00:44
@robinbb robinbb force-pushed the robinbb-issue-4572-singleton-fix branch from fa3d4c3 to 900077f Compare April 8, 2026 05:13
@rgrinberg
Copy link
Copy Markdown
Member

I see. I guess this is relevant whenever there's a singleton library using another library that has (wrapped false). In such a case, running ocamldep is still beneficial because we might depend on a subset of the entry modules in that library.

@rgrinberg
Copy link
Copy Markdown
Member

If you prefer, you could also move this commit to the top of your chain. I think it can be merged without much scrutiny.

Finally, I believe this also changes how we build single module executables? I don't think we ran ocamldep for them before, but a test to confirm this or the contrary would be useful. I'm less excited about dropping this optimization for such single module executables. The compilation and linking are fused into a single command for single module executables, so this optimization can never correctly fire anyway.

robinbb added 10 commits April 8, 2026 19:07
When an internal module name shadows an unwrapped library's module name,
the internal module takes precedence and the library's module is
inaccessible. This means ocamldep will report the internal module, not
the library's, so dependency filtering that treats stanza-internal names
as non-library references is correct.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
…lds (ocaml#4572)

Add baseline tests documenting existing behavior:

- lib-deps-preserved: every non-alias module declares glob deps on its
  library dependencies' .cmi files
- sandbox-lib-deps: sandboxed package builds have dependency libraries'
  .cmi files available when a library depends on another library

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Add baseline test verifying that incremental builds succeed when a module
re-exports a library via a transparent alias (module M = Mylib) and the
library's interface changes. Regression reported by @Alizter.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Move Hidden_deps (library file dependencies) out of Includes.make and
into per-module computation in build_cm. This is a pure refactor with no
behavioral change — all modules still depend on all libraries in the
stanza's requires.

This separation is necessary for issue ocaml#4572: Includes carries -I flags
(which must be shared across all modules) while Hidden_deps can vary
per-module. With this refactor, a future change can use ocamldep output
to filter Hidden_deps per-module without affecting -I flags.

- Includes.make no longer takes ~opaque or bundles Hidden_deps
- New deps_of_entries in lib_file_deps.ml handles opaque logic
- Alias and Wrapped_compat modules skip library deps (matching their
  existing Includes.empty behavior)
- deps_with_exts removed (sole caller was the old opaque branch)

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
…issue ocaml#4572)

Add a new function `read_immediate_deps_raw_of` to the ocamldep module that
returns raw module names (as Module_name.Set.t) from ocamldep output, without
resolving them to Module.t values.

This function will be used to determine which external library modules a
module actually references, enabling finer-grained dependency tracking where
modules are only recompiled when libraries they actually use change.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
…4572)

Use ocamldep output to filter cross-library file dependencies per module.
Instead of every module depending on all libraries, each module now depends
only on libraries whose entry modules it actually references.

- Lib_index maps library entry module names back to libraries, computed
  once per stanza and stored in Compilation_context
- read_immediate_deps_raw_of returns raw (unresolved) module names from
  ocamldep, including cross-library references previously discarded
- build_cm unions external references across transitive intra-stanza deps
  to handle transparent module aliases (module M = Mylib)
- Filtering is disabled for stanzas whose requires include virtual library
  implementations, since their requires may not include the virtual library
- deps_of_entries extended to support per-file deps for unwrapped libraries
- Dep_graph.dir accessor added for link-time module detection
- deps_with_exts removed (dead code after Includes refactor)

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Single-module library consumers fall back to all-library glob deps because
dune skips ocamldep for singleton stanzas. Without ocamldep data, per-module
filtering cannot determine which libraries the module references, so
modifying an unused module in a dependency triggers unnecessary recompilation.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Dune previously skipped ocamldep for singleton stanzas as an optimization
(no intra-stanza deps to compute). This prevented per-module library
dependency filtering from working, since it relies on ocamldep output to
determine which libraries a module references.

Remove the singleton shortcut from dep_rules.ml so ocamldep runs for all
stanzas. The cost is negligible (one extra ocamldep invocation per
single-module stanza) and enables the filtering optimization for all cases.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb robinbb force-pushed the robinbb-issue-4572-singleton-fix branch from 900077f to f04f4ac Compare April 9, 2026 06:42
@robinbb
Copy link
Copy Markdown
Contributor Author

robinbb commented Apr 9, 2026

Superseded by #14103, which is based on upstream/main (independent of #14021). The baseline test documents current behavior, and the code change is the same.

@robinbb robinbb closed this Apr 9, 2026
@robinbb robinbb deleted the robinbb-issue-4572-singleton-fix branch April 9, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants