Extend FLWOR expressions to maps

[rhdunn]

Edit: Current proposal (#31 (comment)):

for key $key in ...
return …

for value $value in ...
return …

for key $key value $value in ...
return …

---

With the addition of the for member syntax for arrays, it is possible to use a ForExpr/FLWORExpr to enumerate the contents of sequences and arrays, but not maps. In order to be consistent and symmetric across these types, the for member syntax should be extended to support maps by enumerating the key/value entries of the map.

Given a map of type map(K, V) the member RecordTest would be record(key as K, value as V). Given a map of type map(*), the member RecordTest would be record(key, value).

This would allow a user to write expressions like:

for member $entry in $map
return element { $entry?key } { $entry?value }

NOTE: With the addition of the array:values and map:entries functions in issue #29, it is possible to avoid the need of the for member syntax for arrays and maps, but may be worth keeping for people who prefer the wordy style of the XPath/XQuery syntax.

[joewiz]

I like this, but shouldn't it be for entry? Arrays have members, and maps have entries. From XDM:

An array is an ordered list of values; these values are called the members of the array.

XDM doesn't explicitly define entry, but it's used exclusively in FO, e.g.:

The function map:entry returns a map which contains a single entry. The key of the entry in the new map is $key, and its associated value is $value.

[benibela]

It could also be done without the temporary map. For example:

for key $key in $map return ...

for value $value in $map return ...

for key $key value $value in $map return ...

for value $value at $key in $map return ...

[rhdunn]

I would be happy with for entry ... for maps, so ForExpr/FLWORExpr would support:

1. for ... for sequences (existing);
2. for member ... for arrays (new in the draft spec);
3. for entry ... for maps (from this proposal).

[michaelhkay]

I'm not entirely convinced. Mainly because "entries" in a map aren't values in the data model; also because maps are unordered, and also because unlike processing arrays, it's not very difficult or inconvenient to do it using "for $k in map:keys($m), $v in $m($k) return....

[dnovatchev]

@michaelhkay commented:
"... I'm not entirely convinced. Mainly because "entries" in a map aren't values in the data model; also because maps are unordered,"

It is high time that we come up with a set type in XPath. We actually have to deal all the time with sets (not just node-sets, but sets of any-type values), and it is painful to read in the spec how two maps are compared for equality when explaing fn:deep-equal():

"If $i1 and $i2 are both ·maps·, the result is true if and only if all the following conditions apply:

Both maps have the same number of entries.

For every entry in the first map, there is an entry in the second map that:

has the ·same key· (note that the collation is not used when comparing keys), and

has the same associated value (compared using the fn:deep-equal function, under the collation supplied in the original call to fn:deep-equal)."

When if we had the set type the above would simply say:

"If $i1 and $i2 are both ·maps·, the result is true if and only if the sets of their keys are equal, and the corresponding values for each key in the two maps are deep-equal."

I propose that starting with XPath 4.0 we introduce the set type and define set equality, the union ( | ), intersection (intersect) and set difference (except) not only for node-sets but for sets of any-typed values.

Then we can have a function: to-set($collection as item()*) as set, which would produce a set (of the distinct values) of any collection-typed argument supplied to it: sequence(its distinct values) , map (a set of its entries), array (a set of its members).

This makes fn:distinct-values() almost unnecessary.

We will no longer have to explain in a "Remarks" section that the result of a function is "unordered" or that its order is "implementation-defined" -- just by making this function return a set.

How can almost all major programming languages (not even speaking of SQL), such as C#, Python and Java can have a set data type / interface, but even in XPath version 4 we still have to describe it in a free language narrative?

Thanks,
Dimitre

[michaelhkay]

From a technical point of view, I concur. However, I want to restrict the scope of what we're attempting in this version: the reason the last version took 10 years to complete is that we were too ambitious. The most expensive changes to make are those that change the data model, and I think we should strongly resist doing that.

[dnovatchev]

From a technical point of view, I concur. However, I want to restrict the scope of what we're attempting in this version: the reason the last version took 10 years to complete is that we were too ambitious. The most expensive changes to make are those that change the data model, and I think we should strongly resist doing that.

Let's see what other people have to say.

Should the language still be incomplete and encumbered, or can we do something good here?

[michaelhkay]

What we could contemplate, however, is a library of functions for manipulating sets of atomic values, using a map as the underlying representation. For example

set:of("a", "b", "c")
set:union($set1, $set2)
set:intersection($set1, $set2)

Going beyond sets of atomic values to sets of arbitrary values immediately gets you into the problem of defining equality between arbitrary values, which is a quagmire.

[ChristianGruen]

Let's see what other people have to say.

I completely agree with Michael Kay here. Smaller steps to start with won’t prevent us from doing bigger steps later on.

And I personally feel it's helpful not to go beyond the scope of the original issue in the discussion (here, it was “Extend the for member syntax to maps”). Otherwise, issues like this tend to be closed later on and replaced by new issues.

[dnovatchev]

Let's see what other people have to say.

I completely agree with Michael Kay here. Smaller steps to start with won’t prevent us from doing bigger steps later on.

And I personally feel it's helpful not to go beyond the scope of the original issue in the discussion (here, it was “Extend the for member syntax to maps”). Otherwise, issues like this tend to be closed later on and replaced by new issues.

Agreed, I submitted this as a new issue:

Proposal to introduce the set datatype in XPath 4

[dnovatchev]

Going beyond sets of atomic values to sets of arbitrary values immediately gets you into the problem of defining equality between arbitrary values, which is a quagmire.

People do it in other PLs. In the .NET world one simply uses object.GetHashCode()

[michaelhkay]

It's unfortunate that in 3.1, the map:entry() and map:merge() functions model a key-value pair as a singleton map. I'm coming to the conclusion that decomposing a map into (key, value) records (record(key as xs:anyAtomicType, value as item()*)* is much more useful. But mixing the two representations is awkward.