xprocspec

Abstract

xprocspec is a tool for testing XProc scripts. This specification defines the grammar used to define XProc tests for use with xprocspec. This document is automatically generated based on the Relax NG schema for the xprocspec grammar.

1. Definitions

1.1 The `description` element

Content model:: documentation ?; script ?; import *; scenario *; pending *
Content attributes:: focus ?; pending ?; script ?; version ?; Zero or more attributes from any namespace.

The description element is the root element of an xprocspec test and describes the functionality of a specific XProc script.

Assertions or scenarios with the focus attribute will be the only ones tested and reported. When present, only that assertion or scenario will be evaluated. Its content describes why you are focusing on this particular assertion or scenario.

Assertions or scenarios with the pending attribute will remain untested, but will be reported as (eventual) desired behaviour. Its semantics is the same as for the pending element. Its content describes why the assertion or scenario should not be evaluated.

The script attribute is a URI that points to the XProc script you want to test.

The version attribute can be used to aid development over time.

1.2 The `documentation` element

Content model:: Zero or more elements from any namespace.
Content attributes:: None.

The documentation element contains human-readable documentation. There are no constraints on the content of the p:documentation element. Documentation is ignored when running xprocspec tests.

1.3 The `script` element

Content model:: declare-step ?; pipeline ?; library ?
Content attributes:: None.

The script element can be used insted of the script attribute. It allows you to inline the XProc script to test. The base URI of the script is the same as the containing xprocspec document, unless otherwise specified using xml:base. One of the p:declare-step, p:pipeline or p:library elements must be used.

1.4 The `import` element

Content model:: documentation ?
Content attributes:: href; Zero or more attributes from any namespace.

An import runs the xprocspec description in the referenced file. It has no effect on the scenarios in the current document, but can be a useful method to group xprocspec descriptions into a test suite. Importing is recursive and may be circular (although only one copy of a given imported document will actually be imported).

The href attribute contains a URI pointing to the xprocspec file to be imported.

1.5 The `scenario` element

Content model:: documentation ?; call ?; scenario *; pending *; context *; expect *
Content attributes:: focus ?; label; pending ?; xfail ?; Zero or more attributes from any namespace.

A scenario groups together the definition of the script environment (the call element) and the script assertions (the context and expect elements). The call element defines the inputs, options and parameters passed to the step and the context and expect elements are used to perform assertions on the result. Child scenarios inherit the contents of the call element, but can override its inputs, options and parameters by in its own call element.

The label attribute is used to describe the current element in human-readable words.

Assertions with the xfail attribute will behave just like other assertions, but will succeed where other assertions fail, and will fail where other assertions succeed (i.e. the result is inverted). The xfail attribute can also be applied to scenarios, in which case xfail will be implicitly enabled for all of the scenarios assertions. Its content describes why you are expecting the test to fail.

1.6 The `call` element

Content model:: documentation ?; input *; option *; param *
Content attributes:: step ?; Zero or more attributes from any namespace.

A call element defines a step call and the inputs, options and parameters passed to it.

The step attribute is the name of the step (as declared in XProc scripts by the p:declare-step/@type attribute). For instance step="example:mystep". The step attribute must be specified on either the call element or one of the ancestor call elements.

1.7 The `pending` element

Content model:: documentation *; scenario *; pending *
Content attributes:: label ?; Zero or more attributes from any namespace.

TODO: the element pending has not been documented yet.

The label attribute is used to describe the current element in human-readable words.

1.8 The `context` element

Content model:: documentation *; document *
Content attributes:: focus ?; label ?; pending ?; Zero or more attributes from any namespace.

Defines the context against which assertions are made.

The label attribute is used to describe the current element in human-readable words.

1.9 The `expect` element

Content model:: document *; documentation *
Content attributes:: code ?; equals ?; focus ?; grammar ?; href ?; label ?; max ?; message ?; min ?; normalize-space ?; pending ?; step ?; test ?; type ?; xfail ?; Zero or more attributes from any namespace.

Defines what is expected from the context document(s).

When type is error, the code attribute is used to declare what the expected error code is. The code attribute is a string. If you need to evaluate XPath expressions, then set the type attribute to xpath instead.

When type is xpath, the equals attribute defines the value that the test expression must evaluate to for the test to succeed. By default, true() is used. For example, <x:expect test="/*/@result='value'" .../> is equivalent to <x:expect test="/*/@result" equals="'value'" .../>. It is recommended that you use this attribute when possible, since it will give you better error messages if it fails (you will see the actual value as well as the expected).

The grammar attribute is used to set the grammar of the x:document element (used when type is validate). The assertion succeeds if all of the documents in context validates against the schema given by the child x:document element.

When type is custom; the href attribute is a URI pointing to the XProc script that must be imported for the custom step to be available.

The label attribute is used to describe the current element in human-readable words.

If max is specified, then there must be at most this many documents in context for the assertion to succeed.

If min is specified, then there must be at least this many documents in context for the assertion to succeed.

By default, whitespace normalization are performed on documents before comparing them with eachother. You can disable this by setting normalize-space to false.

The step attribute is the name of the custom step (as declared in XProc scripts by the p:declare-step/@type attribute). For instance step="example:mystep".

When type is xpath, you can use the test attribute to define a XPath expression. By default, the expression must evaluate to true() for the test to succeed.

The type attribute is used to define the type of assertion you want to make.

When type is compare, you can put a sequence of x:document elements inside the x:expect element. The assertion succeeds if the document sequence in context equals the document sequence inside the x:expect element.

When type is validate, the documents in context is validated against the schema provided in the x:document element inside the x:expect element.

When type is count, the number of documents in context is compared with the min and max attributes.

The type attribute is used to define the type of assertion you want to make.

When type is custom, you can provide your own XProc implementation of an assertion. The step signature must be as follows:


<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" type="(...)" (...)>
   <p:input port="context" sequence="true"/>
   <p:input port="expect" sequence="true"/>
   <p:input port="parameters" kind="parameter" primary="true"/>
   <p:output port="result" primary="true"/>
   (...)
</p:declare-step>

How custom steps are invoked:

All attributes on the expect element are provided as parameters in a c:param-set document on the primary parameter port of the step. In addition, the temp-dir variable (path to temporary directory that can be used for the test) and the temp-base-uri variable (path to the xprocspec test document) are passed as parameters in the namespace http://www.daisy.org/ns/xprocspec. For instance, <x:expect xmlns:ex="example" type="custom" ex:foo="bar"/> will result in:
```
<c:param-set>
    <c:param name="type" namespace="" value="custom"/>
    <c:param name="foo" namespace="example" value="bar"/>
    <c:param name="temp-dir" namespace="http://www.daisy.org/ns/xprocspec" value="file:/path-to-temp-dir"/>
    <c:param name="temp-base-uri" namespace="http://www.daisy.org/ns/xprocspec" value="file:∕some-path/test.xprocspec"/>
</c:param-set>
                                    
```
The sequence of context documents are provided on the context port.
The sequence of documents defined inside the x:expect element are provided on the expect port.

The document on the result port must be a x:test-result, with a result attribute containing either 'passed', 'failed' or 'skipped', and can contain x:expected and x:was elements (in that order, at most one of each). The contents of x:expected and x:was is either text or XML. For instance:


<x:test-result xmlns:x="http://www.daisy.org/ns/xprocspec" result="passed">
    <x:expected xml:space="preserve">Description of what the value was expected to be.</x:expected>
    <x:was xml:space="preserve">Description of what the value actually was.</x:was>
</x:test-result>

1.10 The `input` element

Content model:: documentation *; document *
Content attributes:: port ?; Zero or more attributes from any namespace.

The input element is used to provide documents on the input ports of the XProc step you are testing.

1.11 The `option` element

Content model:: documentation *
Content attributes:: base-uri?; name ?; select ?; Zero or more attributes from any namespace.

The option element is used to provide values to XProc options on the XProc step you are testing.

If the base-uri attribute is temp-dir, then the base URI in the context will be the temporary directory used for the test, instead of the base URI of the xprocspec document. This can be used with for instance base-uri() and resolve-uri('...')

The name attribute is the name of the option used in the XProc step.

The select attribute is a XPath expression used to select a value for the XProc option (equivalent of the p:with-option/@select attribute in XProc).

The variable "$temp-dir" is available for use in the select expression. The variable refers to the temporary directory used for the test. This is useful (for instance) if the value of the option is a URI pointing to a directory where the XProc steps will store files. Every xprocspec scenario is assigned its own temporary directory so that subsequent evaluations of the step does not conflict with eachother.

1.12 The `param` element

Content model:: documentation *
Content attributes:: base-uri?; name ?; port?; select ?; Zero or more attributes from any namespace.

The param element is used to provide values to XProc paramaters on the XProc step you are testing.

The name attribute is the name of the parameter to be provided on the XProc steps primary parameter input port.

The port is the name of a XProc input parameter port. The parameter will be sent to this port if provided, or to the default parameter port if not provided.

The select attribute is a XPath expression used to select a value for the XProc parameter (equivalent of the p:with-param/@select attribute in XProc).

The variable "$temp-dir" is available for use in the select expression. The variable refers to the temporary directory used for the test. This is useful (for instance) if the value of the parameter is a URI pointing to a directory where the XProc steps will store files. Every xprocspec scenario is assigned its own temporary directory so that subsequent evaluations of the step does not conflict with eachother.

1.13 The `document` element

Content model:: documentation ?; One element from any namespace. (optional)
Content attributes:: base-uri ?; href ?; method ?; ordered ?; port ?; position ?; recursive ?; select ?; type ?; Zero or more attributes from any namespace.

The document element is used to define which documents are provided on a steps input ports, what the context is when making assertions, and for assertions when making comparisons or performing validations. It can be used in a number of ways:

If the base-uri attribute is temp-dir, then the URI in the href attribute (or the xml:base attribute of the document or inline element in the case of inline documents) will be resolved against the temporary directory used for the test, instead of the base URI of the xprocspec document.

When type is file; the href attribute is a URI pointing to the file you want the contents of.

When type is directory; the href attribute is a URI pointing to the directory you want listed.

When type is zip; the href attribute is a URI pointing to the ZIP file you want listed.

By default, files loaded by using method="file" will be loaded as XML files. Non-XML files can be loaded by specifying another method.

When ordered is true, the ZIP entries are listed in lexicographical order.

The port is the name of a XProc input or output port. The x:document element will represent the sequence of documents provided on the input port, or the sequence of documents returned on the output port.

By specifying the position attribute, a certain document in the sequence of documents appearing on the port can be selected.

By default, the result is the same as invoking the standard XProc step p:directory-list. If you specify the recursive attribute and give it the value true, you will get a recursive directory listing, listing all subfolders and their contents as well.

The select attribute can be used to select portions of the document(s). The matching elements will be provided as a sequence of documents. It is equivalent to the XProc step p:filter.

If the type attribute is inline, then the document element is replaced by its content. There must be exactly one element inside the document element. To specify the base URI of the document, you can specify a xml:base on either the document element (implicit base URI) or the child element itself (explicit base URI).

If the type attribute is port, then the document element will be replaced with the documents on that port. The position attribute can be used to select only part of the sequence of documents appearing on that port.

If the type attribute is file, then the document element will be replaced by the contents of the file located at the location pointed to by the href attribute. The default method for reading files is xml, but by using the method attribute you can also choose html, text and binary as methods for loading the file. The href URI is by default resolved against the base URI of the xprocspec test document. However, if you provide the base-uri attribute with a value of temp-dir, then the href URI will be resolved against the temporary directory used for the test instead.

If the type attribute is directory, then the document element will be replaced with a directory listing of the directory pointed to by the href attribute. The href URI is by default resolved against the base URI of the xprocspec test document. However, if you provide the base-uri attribute with a value of temp-dir, then the href URI will be resolved against the temporary directory used for the test instead.

If the type attribute is zip, then the document element will be replaced with a listing of the ZIP file pointed to by the href attribute. The href URI is by default resolved against the base URI of the xprocspec test document. However, if you provide the base-uri attribute with a value of temp-dir, then the href URI will be resolved against the temporary directory used for the test instead.

If the type attribute is errors, then a c:errors document (as defined in the XProc spec) will be made available if any errors occur during step execution. If no errors occur, an empty sequence is returned.

xprocspec

A tool for testing XProc scripts

Unofficial Draft 2 February 2017

Abstract

Status of This Document

Table of Contents

1. Definitions

1.1 The `description` element

1.2 The `documentation` element

1.3 The `script` element

1.4 The `import` element

1.5 The `scenario` element

1.6 The `call` element

1.7 The `pending` element

1.8 The `context` element

1.9 The `expect` element

1.10 The `input` element

1.11 The `option` element

1.12 The `param` element

1.13 The `document` element

A. Acknowledgements

Abstract

Status of This Document

Table of Contents

1. Definitions

1.1 The description element

1.2 The documentation element

1.3 The script element

1.4 The import element

1.5 The scenario element

1.6 The call element

1.7 The pending element

1.8 The context element

1.9 The expect element

1.10 The input element

1.11 The option element

1.12 The param element

1.13 The document element

A. Acknowledgements

1.1 The `description` element

1.2 The `documentation` element

1.3 The `script` element

1.4 The `import` element

1.5 The `scenario` element

1.6 The `call` element

1.7 The `pending` element

1.8 The `context` element

1.9 The `expect` element

1.10 The `input` element

1.11 The `option` element

1.12 The `param` element

1.13 The `document` element