Saxon diaries

Draft Proposals for XSLT/XPath/XQuery 4.0

I've been working on translating the ideas in my XML Prague 2020 paper, entitled a Proposal for XSLT 4.0 into concrete specifications, and my first attempt at this can be found here:

I'm hoping to gather together a community group of some kind to take this forward; meanwhile I've published a very preliminary set of drafts:

• XSLT
• Functions and Operators
• XPath
• XQuery

I put these ideas up yesterday on the XML community Slack channel and got some great feedback. Unfortunately Slack isn't really a good vehicle for managing the response to this feedback. I'm going to organise some GitHub space for a more structured discussion, but meanwhile, here are my reactions to the initial comments:

Phil Fearon:

There’s a lot to digest here so hope to provide feedback after I’ve read this more thoroughly. One suggestion (inspired by ReactJS/JSX) is to provide some syntactical sugar for xsl:call-template. So a call to a named template appears more like a literal result element (perhaps with a special namespace), with attributes that correspond to template params. This could also allow the child items of the special-LRE to be passed to the named template, accessed via a special $children param.

Yes, I've been wanting to do something like this for years, and it's really not difficult, so I've added it. If EX is listed in extension-element-prefixes, and if there's a named template name="ex:action", then <ex:action a="expr" b="expr"/> is interpreted as an xsl:call-template with xsl:wth-param children for parameters a and b.

Liam Quin

first quick note, best NOT to have them say W3C Recommendation on them as this may cause confusion.

Yes, sorry about that, still working my way around the stylesheets that generate the boilerplate text...

2d, cam xsl:text have a select attribute? i don't think value-of can be deprecated :disappointed: but xsl:text select= would be consistent & may help.

It's one of these things that one would like to simplify, but we can only add things not remove them, so that's not easy.

" the tunnel parameters that are implicitly passed in a template call may have names that duplicate the names of non-tunnel parameters that are explicitly passed on the same call." is a major source of difficult debugging if you forget tunnel=yes. Maybe the answer is just a warning from impl'ns.

Yes. How to solve this without breaking compatibility? Perhaps a dynamic error if you declare a non-tunnel parameter, and at run-time there's a tunnel parameter with that name, but no non-tunnel parameter, or vice versa? Or, as you say, just rely on warnings. I agree it's a very common mistake that's hard to debug.

The "at $pos" of XQuery is super useful. position() is tricksy.Maybe for-each at="name" ?

In 3.0 we experimented with replacing some of the context functions with explicit variable bindings and it got a bit messy, but I think it's a shame we didn't persevere. The toughest one is last(), it would be awfully nice if we knew statically whether last() was going to be needed or not, but again, hard to fix without breaking code.

prefix binding didn't make the cut for XPath?

I did a design for this and didn't like it enough to put it in. I'll try again.

item-at() seems not much easier than $xxx ! let $p := position() return $yyy[$p]

I'm toying now with an alternative to item-at() that's much more powerful: slice(sequence, positions) so you can do slice($s, 5) or slice($s, 5 to 10) or slice($s, -1) or slice($s, 1 by 3 to count($s)) or slice($s, -2 by -1 to -count($s)). Here A by B to C is an extension of the current range expression where A to B means A by 1 to B.

replace-with() seems like perl's e flag (JS has one too) but alas no polymorphism so can't write replace(., $expr, myfunc#1m 'e')

Yes, I think it's a really useful capability, but I think it's cleaner to make it a separate function. Have to think about how subgroups might work.

Reece H. Dunn

I like the enum(...) syntax in addition to the union(...) syntax.

I like the extension of element and attribute type tests to be full name tests. The ability to define types for path expressions like (ol|ul) is missing, though.

Yes, I'm in two minds whether union(X, Y, Z) should be restricted to a union of atomic types, or whether it should allow a union of any types including node types. Orthogonality suggests the latter, but I was too timid to propose that.

For named item types, is it possible to make them available as part of the in-scope schema types (renamed to in-scope types that would include the schema and named types?), so you could say person-name instead of item-type(person-name). -- Having to qualify the named item types everywhere could get too verbose, especially if the name is short. _NOTE:_ This is done for MarkLogic types where you can refer to map:map, cts:query, etc.

Interesting idea. There's obviously a need to resolve conflicts but that's not a stopper. I think I was more concerned with the idea that if it's a QName then it must be atomic, and the messy fact that the sets of schema types and item types overlap, and the overlap contains all atomic types and some but not all union types.

Liam Quin

hmm, xsl:sequence could do with an "as" attribute.

Not convinced. You start wanting to put it anywhere e.g. on xsl:if or xsl:apply-templates.

Reece H. Dunn

In https://www.saxonica.com/qt4specs/XP/xpath-40-diff.html#id-itemtype-subtype, rule 2(d) is missing the reference to the EnumerationType symbol ("A is an ," instead of "A is an EnumerationType,").

Stylesheet trouble. The XSLT and XPath spec stylesheets have diverged, the XSLT spec allows <termref def="some-term"/> and picks up the term from the definition, but the XPath spec requires <termref def="some-term">term</termref>. I need to bring them back into line. Applies to your subsequent comments also.

Martin Honnen

I like the separator attribute on xsl:apply-templates and xsl:for-each. I wonder whether it would make sense to add it to xsl:for-each-group as well.

Yes. Also xsl:for-each-member. I'm not sure whether it should be an AVT or a general expression: with a general expression you could insert br or hr separators, especially if we have element constructor functions in XPath (separator="build:element('hr')")

Liam Quin

i've been wondering abut the possibility of an xsl:uri-resolver for some time.

Not sure what it would do?

Also about xsl:mode elements being able to contain xsl:template elements.

Yes, I've wanted that for a long time. To be honest, it's not in 3.0 because I couldn't convince Sharon.

Reece H. Dunn

The changes for XPath/XQuery look good. I see you changed the syntax for the context item and lambda syntaxes to a unified syntax. I like that the concise and full syntaxes are now consistent.

Given that . is allowed in a ParamList, does that mean I can now define a function that works on the path context item? For example: declare function local:f(.) { xs:integer(.) + 2 }; //values/local:f()

Actually, allowing "." here was an oversight caused by my changing the way the grammar rules worked. But there might be some benefit it keeping it.

... does that mean that the context-dependent functions in F&O should be defined using that syntax. For example: fn:data(.) as xs:anyAtomicType*

I hadn't thought of baking the "implicit . as parameter" convention into the language, but it might make sense if it can be done.

Liam Quin

in XQuery i don't understand "The for evaluation of the function body is absent, unless the signature uses the "." notation, in which case it is evaluated with a singleton focus bound to the supplied argument value."

Markup trouble again.

Martin Honnen

If xsl:for-each has a separator attribute, wouldn't xsl:for-each-member benefit from it as well?

Yes, see above.

Reece H. Dunn

For the schema import in XQuery, would it make sense to have: [22] SchemaPrefix ::= ("namespace" NCName "=") | ("default" ("element" | "type") "namespace") now that the element and type namespaces are separate, similar to how DefaultNamespaceDecl has changed.

I thought about this and decided not. If you want finer control, use multiple declarations.

For parameter lists and context items, would it be more useful to have the context item as an optional first parameter? That would mirror the proposed variadic argument syntax (defined for arguments at the end of the parameter list), and would allow arguments to be passed to the function, such as: declare function local:add(., $n) { xs:integer(.) + $n }; //values/local:add(2)

I quite like that in principle. Needs more thought.

Is there any description of the arity of context item based functions? -- There should be a note or something similar to say that the context item for a function definition or inline function expression does not count to its arity, so function () and function (.) both have an arity of 0.

I was thinking of them simply as arity-1 functions, suitable for callbacks in things like fn:filter and fn:sort. You're opening up new possibilities which I need to ponder.

line0

Great to see the spec coming to light!

I proposed to add two new signatures to for-each

for-each(item()*, function (item(), xs:positiveInteger) as item()*) as item()*

for-each(item()*, function (item(), xs:positiveInteger, item()*) as item()*) as item()*

I've proposed that the function coercion rules should allow you to supply an arity-1 function where an arity-2 function appears in the signature; so we can extend fn:for-each to take a function(item, integer) as the predicate callback, and you can still supply function(item) if you don't care about the position.

which would bring for-each on par with FLOWR expressions ( for window and for … in … at ).

It would be great to see windowing done with higher-order functions, but it's a significant piece of design and not my top priority - even though it would bring XSLT up to the level of XQuery for this kind of functionality.

Martin Honnen

For fn:transform, Saxon has already added the option source-location as that is needed to use fn:transform with streaming; I think it makes sense to integrate that option into the fn:transform specification of the FO 4 draft.

Good point.