Xee: A Modern XPath and XSLT Engine in Rust
241 comments
·March 28, 2025therealmarv
smartmic
Well, it's not as if this is the first free alternative. Here is a wonderful, incredibly powerful tool, not written in Java, but in Free Pascal, which is probably too often underestimated: Xidel[1]. Just have a look at the features and check its Github page[2]. I've often been amazed at its capabilities and, apart from web scraping, I mainly use it for XQuery executions - so far the latest version 0.9.9 has also implemented XPath/XQuery 3.1 perfectly for my requirements. Another insider tip is that XPath/XQuery 3.1 can also be used to transform JSON wonderfully - JSONiq is therefore obsolete.
smartmic
Forget to add, for latest XQuery up to 4.0, there is also BaseX [1] — this time a Java program. It has a great GUI/IDE for XQuery rapid prototyping.
mdaniel
https://github.com/BaseXdb/basex/blob/11.8/LICENSE (BSD 3-clause)
_micheee
We also use BaseX to write restful backends with RestXQ - https://docs.basex.org/12/RESTXQ - the documentation itself is written in XQuery as well and uses a BaseX database as a source.
therealmarv
interesting, did not know about that one! Thanks. (Small) but XSLT is not covered by it which is my main usage of XPATH unfortunately.
I will do some experiments with using newer XPATH on JSON... that could be interesting.
Finnucane
I've worked on archive projects with complex TEI xml files (which is why when people say xml is bad and it should be all json or whatever, I just LOL), and fortunately, my employer will pay for me to have an editor (Oxygen) that includes the enterprise version of Saxon and other goodies. An open-source xml processing engine that wasn't decades out of date would be a big deal in the digital humanities world.
sramsay
I don't think people realize just how important XML is in this space (complex documentary editing, textual criticism, scholarly full-text archives in the humanities). JSON cannot be used for the kinds of tasks to which TEI is put. It's not even an option.
Nothing could compel me to like XSLT. I admire certain elements of its design, but in practice, it just seems needlessly verbose. But I really love XPath, though.
miki123211
XML is great for documents.
If your data is essentially a long piece of text, with annotations associated with certain parts of that text, this is where XML shines.
When you try to use XML to represent something like an ecommerce order, financial transaction, instant message and so on, this is where you start to see problems. Trying to shove some extremely convoluted representation of text ranges and their attributes into JSON is just as bad.
A good "rule of thumb" would be "does this document still make sense if all the tags are stripped, and only the text nodes remain?" If yes, choose XML, if not, choose JSON.
tomjen3
>JSON cannot be used for the kinds of tasks to which TEI is put. It's not even an option.
```js import * as fastXmlParser from 'fast-xml-parser'; const xmlParser = new fastXmlParser.XMLParser({ ignoreAttributes: false }); ```
Validate input as required with jschema.
winstonewert
What actually prevents JSON from being used in these spaces? It seems to me that any XML structure can be represented in JSON. Personally, I've yet to come across an XML document I didn't wish was JSON, but perhaps in spaces I haven't worked with, it exists.
faassen
My hope is that we can get a little collective together that is willing to invest in this tooling, either with time or money. I didn't have much hope, but after seeing the positive response today more than before.
cat_multiverse
Oxygen was such a clunky application back when I used it for DH. But very powerful and the best tool in the game. Would love to see a modern tool that doesn't get in the way for all those poorly paid, overworked DH research assistants caffeinated in the dead of night banging out the tedious, often very manual, TEI-XML encoding work...
dfawcus
> I worked on projects which refused to use anything more modern than XSLT & XPATH 1.0 because of lack of support in the non Java/Net World (1.0 = tech from 1999).
For some things that may just be down to how uses are specified. For YANG, the spec calls out XPath 1.0 as the form in which constrains (must and when statements) must be expressed.
So one is forced to learn and use XPath 1.0.
infogulch
There are many humongous XML sources. E.g. the Wikipedia archive is 42GB of uncompressed text. Holding a fully parsed representation of it in memory would take even more, perhaps even >100GB which immediately puts this size of document out of reach.
The obvious solution is streaming, but streaming appears to not be supported, though is listed under Challenging Future Ideas: https://github.com/Paligo/xee/blob/main/ideas.md
How hard is it to implement XML/XSLT/XPATH streaming?
faassen
Anything could be supported with sufficient effort, but streaming hasn't been my priority so far and I haven't explored it in detail. I want to get XSLT 3.0 working properly first.
There's a potential alternative to streaming, though - succinct storage of XML in memory:
https://blog.startifact.com/posts/succinct/
I've built a succinct XML library named Xoz (not integrated into Xee yet):
The parsed in memory overhead goes down to 20% of the original XML text in my small experiments.
There's a lot of questions on how this functions in the real world, but this library also has very interesting properties like "jump to the descendant with this tag without going through intermediaries".
bambax
> I want to get XSLT 3.0 working properly first
May I ask why? I used to do a lot of XSLT in 2007-2012 and stuck with XSLT 2.0. I don't know what's in 3.0 as I've never actually tried it but I never felt there was some feature missing from 2.0 that prevented me to do something.
As for streaming, an intermediary step would be the ability to cut up a big XML file in smaller ones. A big XML document is almost always the concatenation of smaller files (that's certainly the case for Wikipedia for example). If one can output smaller files, transform each of them, and then reconstruct the initial big file without ever loading it in full in memory, that should cover a huge proportion of "streaming" needs.
faassen
XSLT has been a goal of this project from the start, as my customer uses it. XSLT 3.0 simply as that's the latest specification. What tooling do you use for XSLT 2.0?
infogulch
0.2x of the original size would certainly make big documents more accessible. I've heard of succinct storage, but not in the context of xml before, thanks for sharing!
faassen
I myself actually had no idea succinct data structures existed until last December , but then I found a paper that used them in the context of XML. Just to be clear: it's 120% of the original size; as it stands this library still uses more memory than the original document, just not a lot of overhead. Normal tree libraries, even if the tree is immutable, take a parent pointer, and a first child pointer and next and previous sibling pointers per node. Even though some nodes can be stored more compactly it does add up.
I suspect with the right FM-Index Xoz might be able to store huge documents in a smaller size than the original, but that's an experiment for the future.
jerf
"How hard is it to implement XML/XSLT/XPATH streaming?"
It's actually quite annoying on the general case. It is completely possible to write an XPath expression that says to match a super early tag on an arbitrarily-distant further tag.
In another post in this thread I mention how I think it's better to think of it as a multicursor, and this is part of why. XPath doesn't limit itself to just "descending", you can freely cursor up and down in the document as you write your expression.
So it is easy to write expressions where you literally can't match the first tag, or be sure you shouldn't return it, until the whole document has been parsed.
riedel
I think from a grammar side, XPath had made some decisions that make it really hard to generally implement it efficiently. About 10 years ago I was looking into binary XML systems and compiling stuff down for embedded systems realizing that it is really hard to e.g. create efficient transducers (in/out pushdown automata) for XSLT due to complexity of XPath.
ssdspoimdsjvv
Streaming is defined in the XSLT 3 spec: https://www.w3.org/TR/xslt-30/#streamability. When you want to use streaming, you are confined to a subset of XPath that is "guaranteed streamable", e.g. you can't just freely navigate the tree anymore. There are some special instructions in XSLT such as <xsl:merge> and <xsl:accumulator> that make it easier to collect your results.
Saxon's paid edition supports it. I've done it a few times, but you have to write your XSLT in a completely different way to make it work.
mintplant
Would it be possible to transform a large XML document into something on-disk that could be queried like a database by the XPath evaluator?
jerf
Given the nature of this processing, I think even an NVMe-based disk storage would be awfully slow. (People often forget, or never realize, that the "gigabytes per second" that NVMe yields is for sequential access. Random access is quite a bit slower; still stomps spinning rust, but by much less. And this is going to be a random access sort of job, so we're in the "several multiples slower than RAM" regime of access.) This sort of thing really wants RAM, and even then, RAM with an eye towards cache coherency and other such performance considerations.
chii
You'd basically be building an index into each node.
There's some fast databases that store prefix trees, which might be suitable for such a task actually (something like infinitydb). But building this database will basically take a while (it will require parsing the entire document). But i suppose if reading/querying is going to happen many times, its worth it?
econ
It seems to me one could replace each text node with the offset. Perhaps limit it to longer instances?
phonon
Like MarkLogic?
wongarsu
100GB doesn't sound that out of reach. It's expensive in a laptop, but in a desktop that's about $300 of RAM and our supported by many consumer mainboards. Hetzner will rent me a dedicated server with that amount of ram for $61/month.
If the payloads in question are in that range, the time spent to support streaming doesn't feel justified compared to just using a machine with more memory. Maybe reducing the size of the parsed representation would be worth it though, since that benefits nearly every use case
infogulch
I just pulled the 100GB number out of nowhere, I have no idea how much overhead parsed xml consumes, it could be less or it could be more than 2.5x (it probably depends on the specific document in question).
In any case I don't have $1500 to blow on a new computer with 100GB of ram in the unsubstantiated hope that it happens to fit, just so I can play with the Wikipedia data dump. And I don't think that's a reasonable floor for every person that wants to mess with big xml files.
philipkglass
In the case of Wikipedia dumps there is an easy work-around. The XML dump starts with a small header and "<siteinfo>" section. Then it's just millions of "page" documents for the Wiki pages.
You can read the document as a streaming text source and split it into chunks based on matching pairs of "<page>" and "</page>" with a simple state machine. Then you can stream those single-page documents to an XML parser without worrying about document size. This of course doesn't apply in the general case where you are processing arbitrary huge XML documents.
I have processed Wikipedia many times with less than 8 GB of RAM.
nicoburns
Shouldn't parsed XML be smaller than the raw uncompressed text? (as you could deduplicate strings). I'd expect that to be a significant saving for something like wikipedia in XML
hyhjtgh
Xml and textual formats in general are ill suited to such large documents. Step 1 should really be to convert and/or split the file into smaller parts.
heelix
I used to work for a NoSQL company that was more or less an XQUERY engine. One of the things we would complain about is we did use wikipedia as a test data set, so the standing joke was for those of us dealing with really big volumes we'd complain about 'only testing Wikipedia' sized things. Good times.
dleeftink
StackExchange also, not necessarily streamable but records are newline delimited which makes it easier to sample (at least the last time I worked with the Data Dump).
01HNNWZ0MV43FF
Is that all in one big document?
magicalhippo
We regularly parse ~1GB XML documents at work, and got laughed at by someone I know who worked with bulk invoices when I called it a large XML file.
Not sure how common 100GB files are but I can certainly image that being the norm in certain niches.
null
vessenes
This, thirty years later, is the best pitch for XML I’ve read. Essentially, it’s a slow moving, standards-based approach to data interoperability.
I hated it the minute I learned about it, because it missed something I knew I cared about, but didn’t have a word for in the 90s - developer ergonomics. XML sucks shit for someone who wants to think tersely and code by hand. Seriously, I hate it with a fiery passion.
Happily to my mind the economics of easier-for-creators -> make web browsers and rendering engines either just DEAL with weird HTML, or else force people to use terse data specs like JSON won out. And we have a better and more interesting internet because of it.
However, I’m old enough now to appreciate there is a place for very long-standing standards in the data and data transformation space, and if the XML folks want to pick up that banner, I’m for it. I guess another way to say it is that XML has always seemed to be a data standard which is intended to be what computers prefer, not people. I’m old enough to welcome both, finally.
tannhaeuser
> XML has always seemed to be a data standard which is intended to be what computers prefer, not people.
On one hand, you aren't wrong: XML has in fact been used for machine-to-machine communication mostly. OTOH, XML was just introduced as a subset of SGML doing away with the need of vocabulary-specific markup declarations for mere parsing in favor of always requiring explicit start- and end-element tags. Whereas HTML is chock full of SGMLisms such as tag inference (for example inferring paragraph ends on block elements), empty ("self-closing") elements and enumerated ("boolean") attributes driven by per-element declarations.
One can argue to death whether the web should work as a mere document delivery network with rigid markup a la XML, or that browsers should also directly support SGML authoring idioms such as the above shortform mechanisms. SGML also has text macros/shared fragments (entities) and even allows defining own parsing tokens for markdown, math, CSV, or custom syntaxes. HTML leans towards SGML in that its documentation portrays HTML as an authoring language, but browsers are lacking even in basic SGML features such as entities.
IgorPartola
That’s a flame war that’s been raging for decades for sure.
I do wonder what web application markup would look like today if designed from scratch. It is kind of amazing that HTML and CSS can be used for creating beautiful documents viewable on pretty much any device with a screen AND also for creating dynamic applications with pixel-perfect rendering, special effects, integrations with the device’s hardware, and even external peripherals.
If there was ever scope creep in a project this would be it. And given the recent discussion on here of curses based interfaces it reminded me just how primitive other GUI application layout tools can be while still achieving amazing results. Even something like GTK does not need the intense level of layout engine support and yet is somehow considered richer in some ways and probably more performant for a lot of stuff that’s done with it.
So I am curious what web application development would look like today if it wasn’t for HTML being “good enough”.
caspper69
Had we had better process isolation in the mid-90s, I assume web application development would mostly be Java apps, with a mini-vm for each one (sort of a qubes like environment).
We just couldn't keeps apps' hands out of the cookie jar back then.
Devasta
If a browser was designed from scratch today it wouldn't have a markup language, documents would be PDF and everything else would be Javascript to canvas.
Suggesting something like HTML would have you laughed out of the room.
weinzierl
"This, thirty years later, is the best pitch for XML I’ve read."
I wish someone would write "XML - The Good Parts".
Others might argue that this is JSON but I'd disagree:
- No comments is a non-starter
- No proper integers
- No date format
- Schema validation is a primitive toy compared what we had for XML
- Lack of allowed trailing commas
YAML ain't better. I hated whitespace handling in XML, it's a miracle how YAML could make it even worse.
XML is from era long past and I certainly don't want to go back there, but it had its good parts and I feel we have not really learned a lot from its mistakes.
In the end maybe it is just that developer ergonomics is largely a matter of taste and no language will ever please everyone.
jlarocco
It's funny to hear people in the comments here talk about XML in the past tense.
I know it's passé in the web dev world, but in my work we still work with XML all the time. We even have work in our queue to add support for new data sources built on XML (specifically QIF https://qifstandards.org/).
It's fine with me... I've come to like XML. It's nice to have a standard, easy way to do seschemas, validators, processors, queries, etc. It can be overdone and it's not for every use case, but it's pretty good at what it does.
intrasight
I've come to think that XML will be with us for decades and probably follow us when we leave the small blue planet.
In my military work, I've heard the senior project managers refer to a modern battleship as a floating XML document.
bigstrat2003
> I know it's passé in the web dev world...
That is because the web dev world is unfortunately obsessed with the current thing. They chase trends like their lives depend on it.
tzcnt
Developer ergonomics is drastically underappreciated, even in modern times. Since we're talking about textual data formats, I'll go out on a limb here and say that I hate YAML. Double checking exactly how many spaces are present on each line is tedious. It manages to make a simple task like copy-pasting something from a different file (at a different indentation level) into an error-prone process. I'll take angle brackets any day.
chuckadams
You haven’t felt hate until you’ve counted spaces in your Helm templates in order to know what value to put after `nindent`. The punchline is that k8s doesn’t even speak yaml, the protocol is all json and it’s the tooling that inflicts yaml on us. I can live with yaml as a config format, but once logic starts creeping in, give me anything else.
null
formerly_proven
Working with large YAML documents is incredibly annoying and shows the benefit of closing tags.
4ndrewl
It all went downhill after we stopped using .ini files
01HNNWZ0MV43FF
JSON5 is a real sweet spot for me. Closing brackets, but I don't have to type every tag twice. Comments and trailing commas.
consteval
I find for deeply hierarchical data that XML is much easier to read.
Pet_Ant
> Developer ergonomics is drastically underappreciated, even in modern times.
When was the last time you had an editor that wouldn't just auto close the current tag with "</" ? I mean it's a god-send for knowing where you are at in large structure. You aren't scrolling to the top to find which tag you are in.
iamthepieman
>XML has always seemed to be a data standard which is intended to be what computers prefer, not people
Interesting take, but I'm always a little hesitant to accept any anthropomorphizing of computer systems.
Isn't it always about what we can reason and extrapolate about what the computer is doing? Obviously computers have no preference so it seems like you're really saying
"XML is a poor abstraction for what it's trying to accomplish" or something like that.
Before jQuery, chrome, and web 2.0, I was building xslt driven web pages that transformed XML in an early nosql doc store into html and it worked quite beautifully and allowed us to skip a lot of schema work that we definitely were ready or knowledgeable enough to do.
EDIT: It was the perfect abstraction and tool for that job. However the application was very niche and I've never found a person or team who did anything similar (and never had the opportunity to do anything similar myself again)
vitaflo
I did this for many years at a couple different companies. As you said it worked very well especially at the time (early 2000’s). It was a great way to separate application logic from presentation logic especially for anything web based. Seems like a trivial idea now but at the time I loved it.
In fact the RSS reader I built still uses XSLT to transform the output to HTML as it’s just the easiest way to do so (and can now be done directly in the browser).
madkangas
Re xslt based web applications - a team at my employer did the same circa 2004. It worked beautifully except for one issue: inefficiency. The qps that the app could serve was laughable because each page request went through the xslt engine more than once. No amount of tuning could fix this design flaw, and the project was killed.
Names withheld to protect the guilty. :)
intrasight
Most every request goes through xslt in our team's corporate app. The other app teams are jealous of our performance.
wiremine
> developer ergonomics
That was a huge reason JSON took over.
Another reason was the overall XML ecosystem grew unwieldy and difficult to navigate: XPath, XSLT, SOAP, WSDL, Xpointer, XLink, SOAP, XForms... They all made sense in their own way, but it was difficult to master them all. That complexity, plus the poor ergonomics, is what paved the way for JSON to become preferred.
tonyedgecombe
I quite liked it when it first came out, I'd been dealing with a ton of bespoke formats up until then. Pretty much every one was ambiguous and painful to deal with. It was a step forward being able to push people towards a standard for document transfer.
I suspect it was SOAP and WSDL that killed it for a lot of people though. That was a typical example of a technical solution looking for a problem and complete overkill for most people.
The whole namespace thing was probably a step too far as well.
velcrovan
You should try using a LISP like Racket for XML. Because XML can be expressed directly as S-expressions, XML and LISP go together like peanut butter and jelly.
<greeting attr="val" href="#">Hello <thing>world</thing><greeting>
(greeting ((attr "val") (href "#")) "Hello " (thing "world"))koito17
In my experience, at least with Clojure, it's much more convenient to serialize XML into a map-like structure. With your example, the data structure would look like so.
{:tag :greeting
:attrs {:href "#" :attr "val"}
:content ["Hello" {:tag :thing :content ["world"]}]}
[:greeting {:href "#" :attr "val"} "Hello" [:thing "world"]]
I've written a rudimentary EPUB parser in Clojure and found it easier to work with zippers than any other data structure to e.g. look for <rootfile> elements with a <container> ancestor.
Zippers are available in most programming languages, thankfully, so this advantage is not really unique to Clojure (or another Lisp). However, I will agree that something like sexps (or Hiccup) is more convenient than e.g. JSX, since you are dealing with the native syntax of the language rather than introducing a compilation step and non-standard syntax.
velcrovan
I have not looked into the use of zippers for this purpose, but I will do so!
Racket has helper libraries like TxExpr (https://docs.racket-lang.org/txexpr/index.html) that make it pretty easy to manipulate S-expressions of this kind.
zoogeny
This looks like it loses the distinction between attributes and nested tags?
As in, I don't see a difference between `(attr "val")` which expresses an attribute key/value pair and `(thing "world")` which expresses a tag/content relationship. Even if I thought the rule might be "if the first element of the list is a list itself then it should be interpreted as a set of attribute key value pairs" then I would still be ambiguous with:
(foo (bar "baz") "content")
<foo bar="baz">content</foo>
<foo><bar>baz</bar>content</foo>
shawn_w
There's no ambiguity. The first element is a symbol that's the name of a tag. If the second element is a list of two element symbol + string lists, it's the attributes. If it's one of the other recognized types, it's part of the contents of the tag.
See a grammar for the representation at https://docs.racket-lang.org/xml/index.html#%28def._%28%28li...
Most Scheme tools for working with XML use a different layout where a list starting with the symbol @ indicates attributes. See https://en.wikipedia.org/wiki/SXML for it.
immibis
> In fact, this ambiguity between attributes and children has always been one of the head scratching things for me about XML. Well, the thing I've always disliked the most is namespaces but that is another matter.
Just remember that it's a markup language, and then it's not head-scratching at all: the text is the text being marked up, and the attribute values are the attribute of the markup - things like colour and font.
When it was co-opted to store structured data, those people didn't obey this rule (which would make everything attributes).
Namespaces had a very cool use in XHTML: you could just embed an SVG or MathML directly in your HTML and the browser would render it. This feature was copied into HTML5.
froh
a lisp... like dsssl ? ;-)
bambax
I used to do a lot of XSLT coding, by hand, in text editors that weren't proper IDEs, and frankly it wasn't very hard to do.
There's something very zen-like with this language; you put a document in a kind of sieve and out comes a "better" document. It cannot fail; it can be wrong, full of errors, of course (although if you're validating the result against a schema it cannot be very wrong); but it will almost never explode in your face.
And then XSLT work kind of disappeared; I miss it a lot.
bigstrat2003
I'm gonna be honest, I find terseness to be highly overrated by programmers. I value it in moderation, but for a lot of people they say things like "this language is verbose" like that is a problem unto itself. If verbosity is gaining you something (generally clarity), then I think that's a reasonable cost to pay. Terseness is not, in my opinion, a goal unto itself (though many programmers certainly treat it as such). It's something you should seek only to the extent that it makes a language easier to use.
montroser
Fun fact: XSLT still enjoys broad support across all major browsers: https://caniuse.com/?search=xslt
eyelidlessness
I can’t say this with certainty, but I have some reason to suspect I might be partially to blame for this fun fact!
A couple years ago, I stumbled on a discussion considering deprecation/removal of XSLT support in Chrome. At some point in the discussion, they mentioned observing a notable uptick in usage—enough of an uptick (from a baseline of approximately zero) that they backed out.
The timing was closely correlated with work I’d done to adapt a library, which originally used XSLT via native Node extensions, to browser XSLT APIs. The project isn’t especially “popular” in the colloquial sense of the term, but it does have a substantial niche user base. I’m not sure how much uptake the browser adaptation of this library has had since, but some quick napkin math suggested it was at least plausible that the uptick in usage they saw might have been the onslaught of automated testing I used to validate the change while I was working on it.
vanderZwan
And this kids is one more reason for us to use testing while developing
null
Telemakhos
This is true only of XSLT 1.0. The current standard is 3.0.
falcor84
Oh, a shame. Is there any way to track browser version adoption on caniuse, or any other site?
Also, is it up to browser implementations, or does WHATWG expect browsers to stay at version XSLT 1?
tannhaeuser
There's nothing to track here really. For better or worse, browsers are stuck with 1999's XSLT 1.0, and it's a miracle it's still part of native browser stacks given PDF rendering has been implemented using JS for well over a decade now.
XSLT 2 and 3 is a W3C standard written by the sole commercial provider of an XSLT 2 or 3 processor, which is problematic not only because it reduces W3C to a moniker for pushing sales, but also because it undermines W3C's own policy of at least two interworking implementations for a spec to get "recommendation" status.
XSLT is of course a competent language for manipulating XML. It would be a good fit if your processing requires lots of XML literals/fragments to be copied into your target document since XSLT is an XML language itself. Though OTOH it uses XPath embedded in strings excessively, thereby distrusting XML syntax for a core part of its language itself, and coding XPath in XML attributes can be awkward due to restrictive contextual encoding rules for special characters and such.
XSLT can be a maintenance burden if used casually/rarely, since revisiting XSLT requires substantial relearning and time investment due to its somewhat idiosyncratic nature. IDE support for discovery, refactoring, and test automation etc. is lacking.
Telemakhos
I think it's up to browser implementations, but JSON and JavaScript stole much of XML's thunder in the browser anyway, plus HTML5's relaxed tags won out over XHTML 4's strictness (strictness was a benefit if you were actually working with the data). There are still plenty of web-available uses of XML, like RSS and RDF and podcasts/OPML, but people are more likely to call xmlhttp.responseXML and parse a new DOM than wrap their head around XSL templates.
The big place I've successfully used XSLT was in TEI, which nobody outside digital humanities uses. Even then, the XSLT processing is usually minimal, and Javascript is going to do a lot of work that XSL could have done.
ajxs
Being interested in archaic technologies, I built a website using XML/XSLT not that long ago. The site was an archive of a band I was in, which made it fundamentally data oriented: We recorded multiple albums, with different tracks, and a different lineup of musicians each time. There's lots of different databases I could built a static site generator around, but what if the browser could render the page straight from the data? That's what's cool about XML/XSLT. On paper, I think it's actually a pretty nice idea: The browser starts by loading the actual data, and then renders it into HTML according to a specific stylesheet. Obviously the history of browser tech forked in a different direction, but the idea remains good. What if there was native browser support for styling JSON into HTML?
athanagor2
The fact it could be compiled in WASM is a good thing, given the Chrome team was considering removing libxml and XSLT support a few years back. The reasons cited were mostly about security (and share of users).
It's another proof that working on fundamental tools is a good thing.
therealmarv
not WASM but there is also https://www.npmjs.com/package/saxon-js
sntran
This is pretty slow compared to libxslt.
egh
Very cool! I recently wrote an XSLT 2 transpiler for js (https://github.com/egh/xjslt) - it's nice to see some options out there! Writing the xpath engine is probably the hard part (I relied on fontoxpath). I'm going to be looking into what you have done for inspiration!
airstrike
What problems are {elegantly, neatly, best} solved by using XPath and XSLT today that would make them reasonable choices over alternatives?
jerf
XPath is a very nice language for querying over XML. Most places pitch it as a "declarative" syntax, but as I am quite skeptical of "declarative" as a concept, you can also look at the vast majority of the XPath standard as a way to imperatively drive a multicursor over an XML document, diving in out and out nodes and extracting bits of text and such, without having to write the equivalent code in your language to do so, which will be inevitably quite a bit more verbose. When you need it, it's really useful.
In my very opinionated opinion, XPath is about 99% of the value of XSLT, and XSLT itself is a misfire. Embedding an XML language in XML, rather than being an amazing value proposition, is actually a huge and really annoying mistake, in much the same way and for much the same reason as anyone who has spent much time around shell scripting has found trying to embed shell strings in shell strings (and, if the situation is particularly dire, another third or fourth level of such nesting) is quite unpleasant. Imagine trying to deal with bash, except you have to first quote all the command lines as bash strings like you're using bash -c, all the time. I think "XPath + your favorite language" has all the power of XSLT and, generally, better ergonomics and comprehensibility. Once you've got the selection of nodes in hand, a general-purpose programming language is a better way to deal with their contents then what XSLT provides. Hence why it has always languished.
int_19h
XQuery is the best of both worlds - you get almost all the benefits of XSLT like e.g. the ability to define your own functions, but with non-XML-based syntax that is a superset of XPath.
Basically the only thing it's missing in XQuery vs XSLT is template rules and their application; but IMO simple ones are just as easy to write explicitly, and complex rulesets are hard to reason about and maintain anyway.
jrpelkonen
It’s been a while since I’ve had to deal with XML, but I remember finding it fairly convenient to restructure XML documents with XSLT. Modifying the data in those documents, much less so. I think there’s a sweet spot.
akshayshah
To someone who hasn’t worked much with XML, this seems like a reasonable take!
For cases where a host system wants to execute user-defined data transformations safely, XSLT seems like it might be useful. When they mature, maybe WASM and WASI will fill the same niche with better developer ergonomics?
therealmarv
Interesting take about XSLT. But I agree... XSLT could be something much more simple (and non XML initself) and combined with XPATH. It feels like a lot of boiler code to write XSLT.
password4321
XPATH+XSLT is SQL for XML, declarative selection and transformation.
Using an XML library to iterate through an entire XML document without XPATH is like looping through entire database tables without a JOIN filter or a WHERE clause.
XSLT is the SELECT, transforming XML output with a new level of crazy for recursion.
mickeyp
XPath is a superb query language for XML (or anything that you can structure as a DOM) --- it is also, with some obscure exceptions, the only query language with serious adoption, so it's an easy choice and readily available in XML tools. The only caveat is there are various spec versions and most never added support for newer versions.
Let's look at JSON by comparison. Hmm, let's see: JSONPath, JMESPath, jq, jsonql, ...
never_inline
JQ is the most feature-rich of the bunch. It's defacto standard and I usually just default to it because it offers so much - assignment, various builtins such as base64 encoding.
The disadvantage is that it's not easily embeddable in your own programs - so programs use JSONPath / Go templates often.
bbkane
I also don't think there's a specification written for the jq query language, unlike https://jmespath.org/ , which as you mentioned also has more client libraries.
I too am probably going to embed jmespath in my app.I need it to allow users to fill CLI flags from config files, and it'll replace my crappy homegrown version ( https://github.com/bbkane/warg/blob/740663eeeb5e87c9225fb627... )
BoingBoomTschak
And it's yet another terrible DSL that you must learn when it could have been a language everybody already knows, like Python. The query part isn't even that well done, compared to XPath/JSONPath.
I said goodbye to it a few weeks ago, personally (https://world-playground-deceit.net/blog/2025/03/a-common-li... https://world-playground-deceit.net/blog/2025/03/speeding-up...)
trallnag
Recently discovered Jsonata thanks to AWS adding it to Step Functions. Feel free to add it to your enumeration
Devasta
I manage a team who build and maintain trading data reports for brokers, we have everything generate in a fairly standard format and customize to those brokers exact needs with XSLT. Hundreds of reports, couldnt manage without it.
therealmarv
E.g. massive XML documents with complexity which you need to be transformed into other structured XML. Or if you need to parse complex XML. Some people hate XSLT, XPATH with a passion and would rather write much more complex lxml code. It has a steep learning curve but once you understand the fundamentals you can transform XML more easily and especially predictable and reliable than ever.
Another example: If you have very large XML you cannot fit even into memory you can still stream process them with XSLT.
It makes you the master of XML transformations and fetching information out of complex XML ;)
jeffbee
What alternatives exist for extracting structured data from the web? I have several ETL pipelines that use htmltidy to turn tag soup into something approximately valid and xmlstarlet to transform it into tabular data.
never_inline
I have used it when using scraping some data from web pages using scrapy framework. It's reliable way to extract something from web pages compared to regex.
mdaniel
don't overlook the ability to mix and match them, because each "axis" is good at its own things
response.xpath("//div[string-contains(@data-foo, "foo")").css(".some-class").re(r"[a-z][a-zA-Z]+")
riedel
Love to see stuff outside the Java space since I really like thedoing stuff in XSLT. Question: Does this work on a textual XML representation or can you plug in different XML readers? I have had really great fun in the past using http://www.ananas.org/xi/ transforming arbitrarily for formated files using XSLT. Also it is today really important that XML Reader has error correction capabilities, since lots of tools don't write well-formed XML, which often is a showstopper for employing to transforms from my experience.
jchw
I wonder if this could perhaps some day be used in Wine, for the MSXML implementations. Maybe not, since those implementations need to be bug-compatible where applications depend on said bugs; but the current implementation(s) are also not fantastic. I believe it is still using libxml2.
(Aside: A long time ago, I had written an alternate XPath 1.1 implementation for Wine during GSoC, but rather shamefully, I never actually got it merged. Life became very hectic for me during that time period and I never really looped back to it. Still feel pretty bad about it all these years later.)
samsk
Nice ! I've a scrapper using XPath/XSLT extensively and 90% of the XPath selectors work like for years without a change. With CSS selectors I've had more problems...
ebruchez
CSS selectors have spent the last few decades reinventing XPath. XPath introduced right from the beginning the notion of axes, which allow you to navigate down, up, preceding, following, etc. as makes sense. XPath also always had predicates, even in version 1.0. CSS just recently started supporting :has() and :is(), in particular. Eventually, CSS selectors will match XPath's query abilities, although with worse syntax.
samsk
The problem with CSS selectors (at least in scrapers) is also that they change relatively often, compared to (html) document structure, thats why XPath last longer. But you are right, CSS selectors compared to 20 years old XPath are realy worse.
masklinn
On the other hand:
- XPath literally didn't exist when CSS selectors were introduced
- XPath's flexibility makes it a lot more challenging to implement efficiently, even more so when there are thousands of rules which need to be dynamically reevaluated at each document update
- XPath is lacking conveniences dedicated to HTML semantics, and handrolling them in xpath 1.0 was absolutely heinous (go try and implement a class predicate in xpath 1.0 without extensions)
mdaniel
> - XPath literally didn't exist when CSS selectors were introduced
[citation required]
https://www.w3.org/TR/1999/REC-xpath-19991116/
https://www.w3.org/TR/REC-CSS1-961217
> W3C Recommendation 17 Dec 1996, revised 11 Jan 1999
There are various drafts and statuses, so it's always open to hair-splitting but based only on the publication date CSS does appear to win
bambax
> CSS selectors have spent the last few decades reinventing XPath
YES! This is so true! And ridiculous! It's a mystery why we didn't simply reuse XPath for selectors... it's all in there!!
masklinn
> It's a mystery why we didn't simply reuse XPath for selectors... it's all in there!!
It's not really a mystery:
> CSS was first proposed by Håkon Wium Lie on 10 October 1994. [...] discussions on public mailing lists and inside World Wide Web Consortium resulted in the first W3C CSS Recommendation (CSS1) being released in 1996
> XPath 1.0 was published in 1999
CSS2 was released before XPath 1.0.
mattrighetti
I will definitely try this out!
I have a service that extracts <meta> tags in webpages and to do that I'm currently using (and need) three different dependencies: html5ever, markup5ever_rcdom, markup5ever. I don't like those to be honest, the documentation is quite bad and it was difficult to understand how I should have used the libraries to achieve such a simple task.
XPath on the other hand makes this extremely easy in comparison, I wonder how this will perform compared to my current solution.
faassen
Thanks!
Unfortunately at this point there's no HTML parser frontend for Xee (and its underlying library Xot) yet (HTML 5 parser serialization is supported at least in code). It shouldn't be too hard to add at least HTML 5 support using something like html5ever.
nickm12
This is fantastic to see! I've used XML off and on since it was the red hot tech of the early 2000s. I wouldn't choose it today for a green field project, but it's still around in so many places, so we definitely need a high-performance, high-quality library written in Rust for this.
This could become a great foundation for a typed, (mostly) etree-compatible, python library built on top of this. I've used lxml for years and it's still my goto, but there are lots of places where it could be modernized.
Great to see that somebody else creates a true open source XSLT 3 and XPATH 3 implementation!
I worked on projects which refused to use anything more modern than XSLT & XPATH 1.0 because of lack of support in the non Java/Net World (1.0 = tech from 1999). Kudos to Saxon though, it was and is great but I wished there were more implementations of XSLT 2.0 & XPATH 2.0 and beyond in the open source World... both are so much more fun and easier to use in 2.0+ versions. For that reason I've never touched XSLT 3.0 (because I stuck to Saxon B 9.1 from 2009). I have no doubt it's a great spec but there should be other ways than only Saxon HE to run it in an open source way.
It's like we have an amazing modern spec but only one browser engine to run it ;)