Arabica is an XML and HTML processing toolkit, providing SAX, DOM, XPath, and partial XSLT implementations, written in Standard C++.

Arabica is written in Standard C++ and should be portable to most platforms. It is parameterised on string type. Out of the box, it can provide UTF-8 encoded std::strings or UTF-16 encoded std::wstrings, but can easily be customised for arbitrary string types.

Arabica is available for download under a BSD-style license.

Latest News

[RSS 0.91]
Monday 20 October, 2008
#FAQ: When will Arabica's XSLT library be finished?

To tell the truth, I have no idea. Development is of Mangle, Arabica's XSLT engine, is ongoing, although progress varies according to the vagarities of how busy I am, how energetic I'm feeling, whether the kids have a swimming gala, and so on and so forth.

Although it's not done yet, it might well be done enough. I'm using the OASIS XSLT test suite to help drive development, and so it also provides a measure of how much has been done, what's working and what isn't. The results are published here, but all the code and test data is included in the download. The executive summary is the core stuff that you use every day works, but some of the bits round the edges (edges defined by my experience, anyway) are missing.

To my knowledge there's nothing that causes Mangle to crash, and anything that I haven't yet implemented generates a warning when the stylesheet is compiled.

Give it a go. It might do what you need.


[Add a comment]
Sunday 19 October, 2008
#FAQ: What are all those failing tests, and why are they ignored?

If you run the tests, the final testsuite exercises the XSLT engine and it will list a number of failures. Quite a large number. XSLT development is ongoing, and I'm using the OASIS XSLT test suite to guide that. Consequently, the tests that fail generally indicate something I haven't done yet, rather than an actual bug. The XSLT tests are, therefore, ignored by make check (should you be lucky enough to be working on a Unixy platform).

Failures in any other tests are, however, indicative of a problem that needs investigating.


[Add a comment]
Saturday 18 October, 2008
#Arabica October 2008 Release

The "Probably long overdue release" bringing a big chunk of new functionality.

Source tar.bz2
http://downloads.sourceforge.net/arabica/arabica-2008-october.tar.bz2

Source tar.gz
http://downloads.sourceforge.net/arabica/arabica-2008-october.tar.gz

Source zip
http://downloads.sourceforge.net/arabica/arabica-2008-october.zip

Exciting New Stuff

The exciting new stuff is Taggle, a port of John Cowan's rather super TagSoup package.

TagSoup, if you're not familiar with it, is

a SAX-compliant parser written in Java that, instead of parsing well-formed or valid XML, parses HTML as it is found in the wild: poor, nasty and brutish, though quite often far from short. TagSoup is designed for people who have to process this stuff using some semblance of a rational application design. By providing a SAX interface, it allows standard XML tools to be applied to even the worst HTML.
Obviously, if you have a SAX parser you can apply all your standard XML techniques - not only SAX filters, but building a DOM, applying XPaths, or XSLT transformations as well.

Cowan describes what TagSoup does as

TagSoup is designed as a parser, not a whole application; it isn't intended to permanently clean up bad HTML, as HTML Tidy does, only to parse it on the fly. Therefore, it does not convert presentation HTML to CSS or anything similar. It does guarantee well-structured results: tags will wind up properly nested, default attributes will appear appropriately, and so on.

The semantics of TagSoup are as far as practical those of actual HTML browsers. In particular, never, never will it throw any sort of syntax error: the TagSoup motto is "Just Keep On Truckin'". But there's much, much more. For example, if the first tag is LI, it will supply the application with enclosing HTML, BODY, and UL tags. Why UL? Because that's what browsers assume in this situation. For the same reason, overlapping tags are correctly restarted whenever possible: text like:

This is <B>bold, <I>bold italic, </b>italic, </i>normal text
gets correctly rewritten as:

This is <b>bold, <i>bold italic, </i></b><i>italic, </i>normal text.
Looks straightforward, doesn't it? Well, that's a simple example and it's still a tricky and awkward result in practice. Cowan's patience in persuing this and what looks like a rather elegant solution is to be applauded. Porting his code to C++ was quick and painless, and Taggle is a useful addition to Arabica. Thanks, John.

Arabica Taggle chews through HTML, providing the same SAX XMLReader interface as the XML parser, and can be used in exactly the same way. HTML source can be fed through SAX filter stacks, used to build DOM trees, queried with XPath, or transformed using XSLT.

Changes and Bug Fixes

There are, of course, many other fixes and changes. Most are relatively minor, and if you haven't been bitten by them you won't notice. The most significant changes are in Arabica's XSLT engine, Mangle. While still not feature complete and under development, it takes, in this release, a fairly big step forward.

SAX

  • Fixed AttributesImpl.getIndex. Thanks to Isak Johnsson for that, and what on earth was I thinking to me
  • Return attribute type as "CDATA" not the empty string
  • After all this time, realised I had too many template parameters on XMLReaderInterface. It only needs the string_type and string_adaptor. Any addition parameters are only of interest the implementing parser class

DOM

  • Output DocumentFragment properly
  • Output <elem/> for empty elements
  • Slipped a TextCoalescer filter into the DOM builder, so that consecutive bits of text get applied to a single Text or CDATA node, rather than as a series of nodes. (A series of nodes is perfectly legal, it's just slightly unexpected. Even to me, and I work with DOMs a lot :)

XPath

  • Some time ago, it was gently suggested to me that XPathValuePtr and XPathExpressionPtr both exposed implementation details and provided an interface that was inconsistent with the DOM classes, because you accessed the member functions via -> rather than . At the time, I was just pleased to have got the XPath stuff done and wasn't really fussed, so I left it. Since then though, it's niggled and niggled away at the back of my mind and now I've done something about it. XPathValuePtr has become XPathValue and XPathExpressionPtr has become XPathExpression, with the member functions accessed through the . operator. The XPathValuePtr and XPathExpressionPtr name and -> member access are retained for the meantime, so that existing code won't be broken. Existing code using XPathValuePtr will still work, but new stuff should use XPathValue
  • Correctly implemented Namespace Nodes. The XPath data model requires that namespace nodes are associated with an element, and sort ahead of attribute nodes in document order. Until now, Arabica's namespace node had no parent, or owner document and so was failing these requirements
  • The default namespace is included when constructing namespace nodes
  • Amazingly, the XPath prefix:* didn't compile. I had no test for it, and had overlooked it. Now I do, and it isn't
  • Unbound namespace prefixes throw an exception
  • Corrected text() test to match CDATA nodes as well as text nodes
  • XPaths are now evaluated as if the DOM had been normalised, even if it hasn't. That is, consecutive text nodes are treated as a single node

XSLT

  • Params are not passed on through an xsl:apply-imports call
  • Template names are now QNames
  • Template mode is now QName
  • In XPath node() matches any node of any type. In an XSLT match pattern, node() matches everything except attributes and the document root node. Fixed.
  • Fixed variable scoping in xsl:for-each, xsl:if, and xsl:choose
  • Escape naughty text when outputting processing instructions and comments (eg ---)
  • Use std::stable_sort instead of std::sort. When xsl:sort specifies a numerical sort, but you've got some string data in there we need to maintain the relative positions of that string data. This is the first time I can recall actually using std::stable_sort. I will mark it down in my big book of programming accomplishments.
  • Fixed local-name for namespace nodes
  • xsl:message can contain another xsl:message - now handled properly
  • Empty comments output correctly
  • Ensure xsl:choose has at lease one xsl:when
  • Make sure any xsl:template mode attribute is not empty
  • Verify xsl:sort attribute values
  • xsl:call-template now throws if it can't find a matching template
  • Duplicate variable and parameter names are rejected
  • Disallowed current() in match patterns
  • Verify xsl:for-each selects a node-set
  • Disallow pcdata ahead of an xsl:param
  • xsl:stylesheet now allows top-level elements when they are in a foreign namespace
  • Implemented position(), last() and positional predicates in match patterns
  • Throw error if transform is run with no input
  • Verify QNames at transform compile time
  • Detect circular variable references
  • Reject variables and parameters which have both a select attribute and text content
  • Top level variables and parameters handled according to import precedence
  • Fixed internal QName resolution - unprefixed names are not in the default namespace
  • Fixed xsl:element unprefixed names - when no namespace uri is supplied are in the default namespace
  • Don't suppress output of element namespace prefixes or attributes which are in the XSL namespace
  • ensure @xmlns|@xsmlns:* selects no nodes
  • direct information messages to std::cerr, not std::cout

Build and installation

  • Fix for problem installing headers on FreeBSD, where install doesn't understand -D
  • Changes to help out-of-tree builds
  • Added build files for Visual Studio 2008
  • Added configure tests for std::mbstate_t and/or mbstate_t. Some platforms don't have it (VxWorks, for example)
  • Visual Studio 2005 and 2003 project files are now munged from the Visual Studio 2008 files. (Don't try this at home, folks)

Other bits and bobs

  • Fixed for base URIs with leading ../
  • Convert \ to / for relative paths as well as absolute Windows paths.


[Add a comment]
Friday 17 October, 2008
#Arabica: Cutting October 2008 release

A couple of months ago a release was, I said, impending. And it really was, but then I found a niggly thing I really want to fix. And went on holiday. And got really busy at work. And all that other stuff that happens when you're not programming.

There really is a release coming now, because I'm cutting it now. The source bundles will probably go are up on Sourceforge this evening now, and tagged in subversion. Release notes should follow later this weekend or early next week. I'll write up the niggly thing too, because it's quite a nice one.

The last release was just over a year ago. That's probably a bit too long.

Is Taggle part of this release? If so, I'll propagate this to the TagSoup community.
John Cowan [e] [w], 18th Oct 2008
Hi John, Taggle is indeed part of this release.
jez, 20th Oct 2008

[Add a comment]
Wednesday 06 August, 2008
#Arabica: impending release

Now my latest gentle stroll has concluded, there are one or two platform specific build issues to resolve. With them done, I expect to be dropping a new release around the end of August or start of September. The release will include the Taggle HTML parser and improved XSLT support, along with various little bug fixes, minor build improvements.

If you can't wait, there's always the subversion repository.
[Add a comment]
Older news ...

Get in touch Your questions, requests, updates and patches are all welcome. I can be contacted at jez@jezuk.co.uk.

Have fun

SourceForge Project Page
Jez Higgins