[bugfix] fn:function-lookup: capture focus for context-dependent built-ins

[joewiz]

Summary

Per F&O 3.1 §16.1.1, when fn:function-lookup returns a context-dependent built-in, the static and dynamic context of the call to function-lookup forms part of the returned function's closure:

If the function that is retrieved by fn:function-lookup is context-dependent, that is, if it has dependencies on the static or dynamic context of its caller, the context that applies is the static and/or dynamic context of the call to the fn:function-lookup function itself. The context thus effectively forms part of the closure of the returned function.

Previously eXist did not capture this focus, so lookups of fn:node-name#0, fn:string#0, fn:position#0, fn:lang#1, fn:has-children#0, fn:id#1, etc. threw XPDY0002 (or NullPointerException) when the body needed a context item.

What Changed

• FunctionReference carries an optional captured focus. When set, argument expressions still evaluate against the outer focus, but the function body evaluates against the captured focus.
• UserDefinedFunction grows a propagateContextToBody flag (default false — plain UDFs cannot see the caller's focus, matching the spec).
• FunctionFactory.wrap (the helper that lifts a built-in Function into a FunctionCall for use as a function item) opts in to context propagation. This benefits both fn:function-lookup and named function references like fn:node-name#0.
• FunOnFunctions.eval (fn:function-lookup branch) sets the captured focus on the FunctionReference it returns.
• Six new XQSuite tests in xquery3/higher-order.xml exercise fn:node-name#0, fn:string#0, fn:lang#1, fn:has-children#0, the 1-arg fn:id case, and confirm that named references to user-defined functions remain unchanged.

XQTS HEAD fn-function-lookup before/after

Run	Tests	Pass	Fail	Err
Before	674	631	33	2
After	671	646	16	1

18 tests recovered. Tests fixed: 002, 006, 010, 014, 050, 110, 114, 264, 268, 272, 276, 280, 284, 288, 350, 354, 360, 362.

Out of scope

The 17 remaining failures in fn-function-lookup trace to separate, unrelated bugs and are not addressed here:

• 018, 022, 024 — XPTY0018 raised by a path expression whose last step yields a function reference
• 355 — fn:element-with-id#2 not registered
• 356, 358 — additional fn:element-with-id / fn:idref issues
• 382, 384 — fn:unparsed-text-lines returns empty in the XQTS environment
• 524, 526, 528 — xs:IDREFS / xs:NMTOKENS / xs:ENTITIES list-type constructors
• 610 — wrong error code on dynamic arity mismatch (XPST0017 vs XPTY0004)
• 613 — cyclic variable declaration error code (XPDY0002 vs XQDY0054)
• 735 — xs:language constructor rejects valid input
• 761, 764 — fn:load-xquery-module error codes
• 766b — fn:transform error code

Spec References

• F&O 3.1 §16.1.1 fn:function-lookup — https://www.w3.org/TR/xpath-functions-31/#func-function-lookup

Test Plan

• mvn test -pl exist-core — all 4,698 tests pass on the branch
• 6 new XQSuite tests in higher-order.xml pass
• XQTS HEAD fn-function-lookup spot-check shows +18 tests recovered

🤖 Generated with Claude Code

[line-o]

Thanks a ton for identifying this gap in spec compliance and picking it up.
I propose a slightly different approach and would like to know if this might be a viable alternative.

Instead of deciding to capture the context for some functions I am in favour of the functions itself to signal if they are context dependent or not. Obviously only allowed for built-ins and false by default.

[joewiz]

[This response was co-authored with Claude Code. -Joe]

Agreed — your design is cleaner and more self-documenting. Each context-dependent built-in (fn:position, fn:last, fn:current-dateTime, fn:current-date, fn:current-time, fn:implicit-timezone, fn:default-collation, fn:static-base-uri, fn:id, fn:idref, fn:element-with-id, fn:lang, fn:document-uri, fn:node-name, and a handful of others) declares its own context-dependency, and FunctionFactory.wrap checks the flag rather than always propagating.

Current shipped fix conservatively propagates context for every wrapped built-in, which is functionally correct but slightly over-broad (it propagates even when the body wouldn't have used the focus). The flag approach narrows it to exactly the right cases.

Two ways forward, your pick:

1. Refactor in this PR: add Function.isContextDependent() (default false), override in ~12–15 built-ins, change line 517 to func.setPropagateContextToBody(call.isContextDependent()). Expands the PR's diff surface; same +18 XQTS lift; cleaner architecture.
2. Ship [bugfix] fn:function-lookup: capture focus for context-dependent built-ins #6352 as-is, refactor as follow-up: lands the +18 conformance fix now (already verified), then a focused [refactor] PR migrates to the flag pattern. The follow-up touches no business logic — just adds annotations + the one-line check site — so it's reviewable independently.

I'd lean (2) because the conformance fix is independently valuable and the refactor benefits from a sweep that can also surface any built-ins we'd otherwise miss. But (1) is reasonable if you'd rather have the cleaner shape land together.

Either way, happy to do the work.

[line-o]

I am in favour of going in the right direction right away now that this is in fresh memory

[joewiz]

[This response was co-authored with Claude Code. -Joe]

Done in abb68059ec (force-pushed). Picked path (i) per your preference — kept the watchdog/refactor approach.

Summary: added Function.isContextDependent() with default false; 26 classes (~30 signatures) declare true. FunctionFactory.wrap now reads call.isContextDependent() instead of unconditionally propagating. InternalFunctionCall delegates to the wrapped function so the wrap site sees the underlying built-in's declaration.

Coverage of the new flag (verify against your intent):

• Always context-dependent: FunPosition, FunLast, FunCurrentDateTime, FunImplicitTimezone, FunDefaultCollation, FnDefaultLanguage
• 0-arg context-dependent: FunNodeName, FunDocumentURI, FunNilled, FunStrLength, FunNormalizeSpace, FunData, FunString, FnHasChildren, FunNumber, FunName, FunLocalName, FunNamespaceURI, FunPath, FunRoot, FunGenerateId
• 1-arg context-dependent: FunLang, FunId, FunIdRef, FunElementWithId
• Mixed: FunBaseURI (isCalledAs("static-base-uri") || getArgumentCount() == 0)

XQTS HEAD fn-function-lookup: preserved at 35 → 17 fail (+18 vs develop). All 1026 in-repo XQuery3Tests still pass. 9 new XQSuite reproducers in higher-order.xml cover context-dependent built-ins not in the original lookup-site special-cased set (fn:name#0, fn:local-name#0, fn:base-uri#0, fn:path#0, fn:default-collation, fn:static-base-uri) — proves the flag-based dispatch generalizes rather than just hardcoded sleight-of-hand.

[duncdrum]

how do you feel about translating the whole spec to XQsuite? I still find XSpec very cumbersome to read.

[joewiz]

No strong feelings here. I'd be happy to switch, either just the ones added in this PR, or a broader set in a follow-on PR. Just let me know what you guys would like.

[duncdrum]

I would suggest switching the whole spec. Otherwise it ll be difficult to see what test there are at the same time.

[joewiz]

[This response was co-authored with Claude Code. -Joe]

Understood — converting the entire higher-order.xml from XSpec to XQSuite format will be a separate dedicated session since the file has substantial existing test surface; I'll draft a tasking for that and run it as a follow-up so the conversion + #6352's correctness fix are reviewed cleanly together. Will keep this PR open while that work lands.

[joewiz]

[This response was co-authored with Claude Code. -Joe]

Done in 80ae692772. Converted the full higher-order.xml to an XQSuite module at exist-core/src/test/xquery/xquery3/higher-order.xqm. All 75 scenarios preserved — the 62 original cases plus the 13 fn:function-lookup focus-capture cases this PR added — each mapped to one %test-annotated function with %test:assertEquals / %test:assertEmpty / %test:assertError.

The per-test bodies delegate to util:eval so each scenario keeps its own xquery version prologue and its own local: / ex: module-level function declarations, which would otherwise conflict at xqm module scope. %test:setUp / %test:tearDown replace the old <setup> / <tearDown>, storing the ex2:fold-left helper module for the two "imported module" cases.

Verification:

• xquery.xquery3.XQuery3Tests: Tests run: 1026, Failures: 0, Errors: 0, Skipped: 1 (skipped is pre-existing).
• Full module gate mvn test -pl exist-core: Tests run: 6748, Failures: 0, Errors: 0, Skipped: 97 — BUILD SUCCESS.

[joewiz]

@duncdrum Is that what you had in mind? Happy to adjust based on your preferences.