Skip to content

Test packages with gap --bare - #1183

Draft
fingolfin wants to merge 2 commits into
mainfrom
mh/bare
Draft

Test packages with gap --bare#1183
fingolfin wants to merge 2 commits into
mainfrom
mh/bare

Conversation

@fingolfin

@fingolfin fingolfin commented Aug 25, 2025

Copy link
Copy Markdown
Member

Experiment motivated by gap-system/gap#2434

@fingolfin
fingolfin marked this pull request as draft August 25, 2025 16:33
@gap-package-distribution-bot

gap-package-distribution-bot Bot commented Aug 25, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Package Evaluation Report for GAP master

Job Properties

Testing: master/2026/08/11_17-27-49 vs master/2026/08/11_17-13-10

Distribution revision: 628a7a33

Generated by Workflow: https://github.com/gap-system/PackageDistro/actions/runs/31513764128

In total, 172 packages were tested, out of which 149 succeeded, 20 failed and 3 were skipped.

‼️ Detected package(s) failing only on current version. ‼️

❗ ❗ Packages now failing

20 package(s) failed tests only on the current version.

Click to show packages!

✔️ Packages still succeeding

149 package(s) succeeded tests also on the previous version.

Click to show packages!

➖ Packages that were skipped

3 package(s) skipped tests also on the previous version.

Click to show packages!

fingolfin
fingolfin commented Aug 25, 2025
View reviewed changes
Comment thread .github/workflows/test-all.yml
This was referenced Aug 25, 2025
Closed
Open
@fingolfin fingolfin mentioned this pull request Apr 26, 2026
Merged
@fingolfin

fingolfin commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

I asked Claude to look into the failures here, these are its findings:

Why the 48 packages fail in PackageDistro PR #1183

Data source: run 29783367069
(2026-07-20), step "Run tests with OnlyNeeded". All 48 packages pass step "Run tests"
(full autoload); every failure below is in the gap --bare step.

Two important preliminary findings

1. --bare also drops GAPDoc, which creates a whole second failure class.
Unlike gap -A, gap --bare starts with no packages at all. Five of the 48 failures have
nothing to do with PrimGrp/SmallGrp/TransGrp — they are undeclared GAPDoc dependencies.
Verified locally: with GAPDoc loaded and no group DBs, alnuth, digraphs, hecke, idrel
and liealgdb all pass with 0 diffs.

Note also that SmallGrp and PrimGrp both list GAPDoc as a needed package, so anything that
loads them gets GAPDoc for free. Loading GAPDoc in the PR's workflow would isolate the
question you actually care about.

2. alnuth is a GAP library bug, not a package bug. WriteGapIniFile
(lib/userpref.g:741) unconditionally does ValueGlobal("FormatParagraph") — a hidden GAPDoc
dependency in GAP's own library. Same pattern at lib/userpref.g:460,548 and
lib/package.gi:3537 (that one is guarded by IsBoundGlobal).

How to read the "signal" column

GAP core binds stubs for exactly three names (lib/init.g:305-322):

call error without the package
SmallGroup(...) Error, the Small Groups library is required but not installed
IdGroup(...) Error, the Small Groups identification is required but not installed
PrimitiveGroup(...) Error, the Primitive Groups library is required but not installed

Everything else (NrSmallGroups, AllSmallGroups, OneSmallGroup, IdSmallGroup,
NumberSmallGroups, IdsOfAllSmallGroups, HasIdGroup, FrattinifactorSize,
SMALL_AVAILABLE_FUNCS, AllPrimitiveGroups, NrPrimitiveGroups, MinimalBlockDimension,
DegreeOfMatrixGroup, TransitiveGroup, NrTransitiveGroups, AllTransitiveGroups, …) is
simply unbound, so you get Syntax warning: Unbound global variable at load time plus
Error, Variable: 'X' must have a value at run time. The load-time syntax warnings are the
cheap, reliable signal for library-code usage — that is exactly what pointed at IRREDSOL.

Error, Transitive groups of degree <n> are not available comes from GAP's own
lib/galois.gi:505,557,817 (the GaloisType code) and always means TransGrp.

Failure categories

  • LIB — the package's own library code uses the DB (needs a real dependency or a code change)
  • TST — only the test suite uses it (fixable in tst/)
  • DOC — only manual/AutoDoc examples use it
  • INDIRECT — package is innocent; it fails because a dependency fails
  • GAP-LIB — the hidden dependency is in GAP's library, not the package
  • COSMETIC — code runs fine, but output differs because GAP uses SmallGrp data when present
  • GAPDOC — not a group-DB issue at all

The table

# package DB kind evidence
1 alnuth — GAPDOC / GAP-LIB tst/userprefs.tst → WriteGapIniFile → FormatParagraph. Passes with GAPDoc alone.
2 autpgrp smallgrp LIB gap/initmat.gi:88 List(max, IdGroup), gap/initperm.gi:11 IdGroup(U); surfaces via AutomorphismGroupPGroup(G,"Char") in tst/more.tst
3 circle smallgrp TST tst/circle03.tst:199,201 call IdGroup directly
4 classicpres primgrp LIB 3× Syntax warning at lib/standard-generators.gi:264,341,547 for DegreeOfMatrixGroup (declared in primgrp/lib/irredsol.gd:…, also in IRREDSOL); fails in tst/calltest.tst
5 corefreesub smallgrp (+GAPDoc) TST + GAPDOC tst/OtherTest.tst:20 SmallGroup(2000,120); also gap/corefreesub.gi:14,15 ExtractExamples/RunExamples
6 crisp smallgrp TST tst/samples.g:36 SmallGroup(48,29) in the shared group list → 9 test files fail identically
7 ctbllib all three LIB + DOC load-time syntax warnings in gap4/ctdbattr.g: AllPrimitiveGroups (1571,1581), IdGroupsAvailable (2099), IdsOfAllSmallGroups (2135), UnloadSmallGroupsData (2149), AllTransitiveGroups (2254), AllSmallGroups (2562,2664). Actual failures are in tst/docxpl2.tst:329 (OneSmallGroup) and :992 (PrimitiveGroup). Already suggests all three.
8 cubefree primgrp INDIRECT (irredsol) fails while reading IRREDSOL/read.g: Error, the number of arguments does not match a declaration of MinimalBlockDimension. Already declares smallgrp as needed.
9 datastructures primgrp TST tst/hashfunctions/permgroup.tst:12 AllPrimitiveGroups(NrMovedPoints,[2..10])
10 deepthought smallgrp TST tst/compare-dtpols.tst SmallGroup(23^4,5), tst/finite.tst:8 NrSmallGroups. Job also hit the 10-min timeout as a knock-on.
11 difsets smallgrp TST SmallGroup all over tst/*.tst (6 files, 71 diffs). Already suggests smallgrp.
12 digraphs — GAPDOC gap/utils.gi:130 ExtractExamples + 19 more GAPDoc symbols. Passes with GAPDoc alone.
13 genss primgrp TST tst/bugfix.tst:6 PrimitiveGroup(9,5)
14 groupoids smallgrp COSMETIC / GAP-LIB tst/manual/gpdaut.tst:203 asserts StructureDescription(NHa4) in names. GAP's lib/grpnames.gi:1901 uses ID_AVAILABLE/IdGroup + NAMES_OF_SMALL_GROUPS for order ≤ 2000, so names differ without SmallGrp. Reproduced: StructureDescription(WreathProduct(A4,C3)) gives … : ((C9 : C3) : C3) without and … : ((C3 x C3 x C3) : C3) with SmallGrp.
15 grpconst primgrp INDIRECT (irredsol) same MinimalBlockDimension error. Already declares smallgrp as needed.
16 hecke — GAPDOC gap/global.gi:34 StringPrint. Passes with GAPDoc alone.
17 help (HeLP) smallgrp LIB, at load lib/HeLP_no_solving.gi:54 / HeLP_internal_no_solving.gi:5: BindGlobal("HeLP_CT", CharacterTable(SmallGroup(1,1))) runs at load time
18 idrel — GAPDOC lib/logrws.gi:1523,1594,1599 SubstitutionSublist. Passes with GAPDoc alone.
19 inducereduce smallgrp TST tst/indredtest.tst SmallGroup(24,6)
20 irredsol primgrp LIB, at load lib/matmeths.gi:346 InstallMethod(MinimalBlockDimension, …, [IsMatrixGroup], …) — a 1-arg method for PrimGrp's attribute (primgrp/lib/irredsol.gd:249), while IRREDSOL itself only declares the 2-arg operation (lib/matmeths.gd:79). Fixed by bh11/irredsol#23. Root cause for 3 other failures.
21 jupyterviz smallgrp TST tst/high-level-api.tst uses NrSmallGroups as the sample function to plot
22 laguna smallgrp TST SmallGroup/AllSmallGroups/IdGroup in tst/bugfix.tst, elements.tst, laguna.tst, laguna04.tst (57 diffs, mostly cascading Variable: 'g' must have a value)
23 liealgdb — GAPDOC gap/liealgdb.gi:9 PrintTo1. Passes with GAPDoc alone.
24 liepring smallgrp LIB + TST lib/data.gi:48 NumberSmallGroups (syntax warning); tst/p12345.tst compares against NrSmallGroups(7^5)
25 localnr smallgrp LIB lib/lib_local.gi:21,71,177 SmallGroup, :219,248 IdGroup — 104 diffs across localnr01/02.tst
26 loops smallgrp LIB, at load fails while reading read.g: Error, Variable: 'FrattinifactorSize' must have a value (declared in smallgrp/gap/small.gd)
27 majoranaalgebras primgrp GAP-LIB tst/Embedding.tst:9 → Error, recursion depth trap (5000). Bisected locally: loading PrimGrp alone fixes it; SmallGrp and TransGrp do not. Root cause is in GAP's own maximal-subgroup / StructureDescription path. The recursion trap instead of a clean error looks like a GAP bug worth filing separately.
28 modisom smallgrp LIB + TST syntax warnings: gap/grpalg/check.gi:29 + detbins.gi:1158,1333 NumberSmallGroups, jenningsBounds.gi:41,134,136 IdSmallGroup, chkbins.gi:171 IdGroupsAvailable; tests in tst/manexamples.tst also use SmallGroup
29 origami smallgrp LIB lib/origami.gi:205 + lib/normalorigami.gi:97 AllSmallGroups, lib/io.g:87 OneSmallGroup; fails at AllNormalOrigamisByDegree(5)
30 permut smallgrp TST tst/permut.tst IdGroup + IdsOfAllSmallGroups
31 radiroot transgrp GAP-LIB / DOC tst/docexmpl.tst:40 RootsOfPolynomialAsRadicals(poly) → Error, Transitive groups of degree 5 are not available from GAP's lib/galois.gi (GaloisType). RadiRoot itself never names a TransGrp function.
32 rcwa smallgrp LIB lib/rcwagrp.gi:225,332 HasIdGroup/IdGroup inside Display/View methods → 55 diffs across 4 test files
33 rds smallgrp LIB + DOC lib/sigs.gi:945,963,1243,1244 IdSmallGroup; failure surfaces in doc/manual.example-2.tst (SmallGroup(24,3))
34 repsn smallgrp TST tst/repsn.tst SmallGroup([48,32])
35 scscp smallgrp TST tst/offline.tst:43 InstallSCSCPprocedure("GroupByIdNumber", SmallGroup : force); the OpenMath payload then needs IdGroup/SmallGroup
36 semigroups smallgrp LIB gap/semigroups/semirms.gi:35 Random(1, NumberSmallGroups(order)) inside RandomSemigroup → 13 diffs
37 sglppow smallgrp LIB, at load (hard dep) lib/3hoch8/sgl-6561.g:5,21 and lib/phoch7/sgl-p7.g:14,24 write into SmallGrp's SMALL_AVAILABLE_FUNCS. SglPPow is an extension of the Small Groups library — this one genuinely needs NeededOtherPackages.
38 simpcomp smallgrp + primgrp + transgrp LIB syntax warnings lib/glprops.gi:1798,1801 NrSmallGroups, :1826,1827 + lib/fromgroup.gi:996 NrPrimitiveGroups; run-time Error, NrTransitiveGroups: function is not yet defined
39 smallantimagmas smallgrp TST tst/*.tst use OneSmallGroup / AllSmallGroups
40 smallsemi smallgrp (+GAPDoc) TST + GAPDOC tst/small.tst SmallGroup(7,1); gap/utils.gi:24,44 ExtractExamples/RunExamples
41 sonata smallgrp LIB lib/compatible.gi, lib/grpsupp.gi use IdGroup → 72 diffs in tst/fpf.tst, tst/ideals.tst
42 sotgrps smallgrp LIB, at load gap/SOTinfo.gi:9 SMALL_AVAILABLE_FUNCS[i](n)
43 ugaly transgrp + primgrp DOC AutoDoc examples in gap/Examples.gd:205,430,432,461,492,… use PrimitiveGroup(5,3) and TransitiveGroup(3,1), generating ugaly01/03/04.tst (58 diffs)
44 unitlib smallgrp TST tst/testlib.g:29 NrSmallGroups(size); tests also use SmallGroup/IdGroup (58 diffs)
45 utils smallgrp COSMETIC tst/utils05.tst:60 LowerFittingSeries(S4) prints (1,3)(2,4) instead of (1,2)(3,4). Reproduced locally; loading SmallGrp alone fixes it, PrimGrp/TransGrp do not.
46 walrus smallgrp LIB gap/examples.gi:266,270 NrSmallGroups in RandomPregroupFromSmallGroups
47 wedderga primgrp (+smallgrp) INDIRECT (irredsol) + LIB fails because tst/testall.g:2 does LoadPackage("irredsol"). Independently it also has a hidden SmallGrp dep: lib/div-alg.gi:750,764,792 IdSmallGroup (syntax warnings).
48 yangbaxter smallgrp TST tst/*.tst use IdGroup (12 diffs)

Summary counts

Packages can fall in more than one bucket, so these don't sum to 48.

bucket n packages
LIB — package library code 19 autpgrp, classicpres, ctbllib, help, irredsol, liepring, localnr, loops, modisom, origami, rcwa, rds, semigroups, sglppow, simpcomp, sonata, sotgrps, walrus, wedderga
TST — test suite only 19 circle, corefreesub, crisp, datastructures, deepthought, difsets, genss, inducereduce, jupyterviz, laguna, liepring, modisom, permut, repsn, scscp, smallantimagmas, smallsemi, unitlib, yangbaxter
GAPDOC — hidden GAPDoc dep 7 alnuth, corefreesub, digraphs, hecke, idrel, liealgdb, smallsemi
DOC — manual examples 4 ctbllib, radiroot, rds, ugaly
GAP-LIB — GAP's own library 4 alnuth, groupoids, majoranaalgebras, radiroot
INDIRECT (all via irredsol) 3 cubefree, grpconst, wedderga
COSMETIC output difference 2 groupoids, utils

Of the 7 GAPDoc cases, 5 involve no group DB at all (alnuth, digraphs, hecke, idrel,
liealgdb); corefreesub and smallsemi have both a GAPDoc and a SmallGrp problem.

By database (packages where that DB is implicated at all — 5 packages involve none):

  • smallgrp: 34 — autpgrp, circle, corefreesub, crisp, ctbllib, deepthought, difsets, groupoids, help, inducereduce, jupyterviz, laguna, liepring, localnr, loops, modisom, origami, permut, rcwa, rds, repsn, scscp, semigroups, sglppow, simpcomp, smallantimagmas, smallsemi, sonata, sotgrps, unitlib, utils, walrus, wedderga, yangbaxter
  • primgrp: 11 — classicpres, ctbllib, cubefree, datastructures, genss, grpconst, irredsol, majoranaalgebras, simpcomp, ugaly, wedderga
  • transgrp: 4 — ctbllib, radiroot, simpcomp, ugaly
  • none: 5 — alnuth, digraphs, hecke, idrel, liealgdb

Verification

All 48 packages were re-run locally against GAP 4.17dev-45 as

gap --bare -c 'LoadPackage("GAPDoc");; LoadPackage("<pkg>":OnlyNeeded);; res:=TestPackage("<pkg>");; FORCE_QUIT_GAP(res);'

i.e. GAPDoc available, no group databases — which isolates exactly the PrimGrp/SmallGrp/
TransGrp effect that PR #1183 is meant to measure.

Result: exactly 5 packages pass — alnuth, digraphs, hecke, idrel, liealgdb — the GAPDoc-only
bucket. The other 43 still fail, and in every case the residual error is the group-DB error
attributed in the table above. (deepthought and sonata hit the 600 s timeout locally, as
deepthought also did in CI; their diffs before the timeout match.)

Note corefreesub and smallsemi still fail here — consistent with them having both a GAPDoc
and a SmallGrp problem.

Two attributions were established by bisection rather than by reading the error:

  • majoranaalgebras: loading PrimGrp alone makes tst/Embedding.tst pass; SmallGrp and
    TransGrp alone do not.
  • utils: loading SmallGrp alone makes tst/utils05.tst pass; PrimGrp and TransGrp do not.

Suggested next steps

  1. Fix IRREDSOL first (Make irredsol independent of primgrp bh11/irredsol#23) — it alone unblocks cubefree, grpconst and
    wedderga, i.e. 4 of 48 jobs.
  2. Load GAPDoc in the PR workflow (gap --bare -c 'LoadPackage("GAPDoc");' …) so the
    5 GAPDoc-only failures stop masking the real question. Also file the
    WriteGapIniFile/FormatParagraph issue against GAP itself.
  3. The 19 LIB cases need real dependencies or code changes. sglppow and sotgrps extend
    SmallGrp's internals and should just declare it needed. ctbllib and simpcomp need
    package extensions or guards.
  4. The 19 TST + 4 DOC cases are the cheap wins: add LoadPackage("smallgrp") at the top of
    the relevant test file, or replace SmallGroup(n,i) by PcGroupCode/explicit generators.
  5. File GAP issues for majoranaalgebras (recursion depth trap instead of a clean error
    when PrimGrp is missing) and consider whether StructureDescription should be
    deterministic regardless of SmallGrp (affects groupoids, utils).

This was referenced Jul 28, 2026
Merged
Merged
Open
@fingolfin
fingolfin force-pushed the mh/bare branch 4 times, most recently from 145b12c to 20c4293 Compare August 5, 2026 09:19
@fingolfin fingolfin mentioned this pull request Aug 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@fingolfin