Collections

Metanorma supports collections: groupings of individual Metanorma documents into a whole.

Collections are of particular use when the component documents are tightly linked with each other. For example:

multilingual documents: where each component document has the same content in a different language
documents that extensively cross-reference each other, for instance, definitions of an informational model spanning multiple documents.

A collection can rendered into the following output formats (other than in Metanorma XML):

HTML: treated as a single web site, with each Metanorma document a component web site.
PDF: treated as a single document, composed of the individual Metanorma documents.
Word DOC: treated as a single document, composed of the individual Metanorma documents. [added in https://github.com/metanorma/metanorma/releases/tag/v1.7.5]

Compiling collections

Collections are compiled using the metanorma executable, as follows:

Usage:
  metanorma collection FILENAME [..options]

Options:
  -x, [--extensions=EXTENSIONS]     # Type of extension to generate
  -w, [--output-folder=FOLDER]      # Folder to generate collection in
  -c, [--coverpage=COVERPAGE]       # Cover page as Liquid template for collection (currently HTML only)

Where:

FILENAME: The collection manifest. This is a YAML file, outlining the structure of the collection, including the location and identifier for each of the component files.
COVERPAGE: A Liquid template file for the index page to the collection in HTML.

Both are described further below.

Example 1. Example command for generating a Metanorma collection

The command below compiles the collection described in the collection manifest si-brochure-bilingual.yml:

bundle exec metanorma collection si-brochure-bilingual.yml \
  -x xml,html,presentation,pdf \
  -w bilingual-brochure \
  -c collection_cover.html

Specifically, it:

generates the collection as XML, Presentation XML, HTML and PDF in the folder bilingual-brochure; and
uses the HTML index page template collection_cover.html.

The metanorma collection executable presupposes that the individual metanorma collections are already compiled into XML.

The compilation of the collection resolves any cross-references between files in the collection during preprocessing, so those inter-document links become simple hyperlinks.

The output folder contains those preprocessed individual files, and files named collection in the target formats.

Specifying collections

Collection manifest

The collection manifest contains the following keys:

directives

Directives on how the collection should be generated. Accepted values are listed below.

documents-inline: (default) Indicates that the files should be concatenated into a single XML file for processing. If neither documents-inline nor documents-external is supplied, documents-inline is presupposed.

Currently, if you are generating PDF or DOC output, you can only specify documents-inline.
documents-external: Indicates that the files are kept separate in processing, and the overall collection XML file does not contain the XML of the individual documents.
presentation-xml: Indicates that the XML supplied is in Metanorma Presentation XML format, and does not need to be converted to Presentation XML by Metanorma [added in https://github.com/metanorma/metanorma/releases/tag/v1.3.5].

This is automatically included when sectionsplit is set in the Metanorma file, to break a single document up into multiple HTML files.
coverpage: …: Gives the location of the HTML coverpage, as an alternative to the -c argument of metanorma collection [added in https://github.com/metanorma/metanorma/releases/tag/v1.5.7].

Any locally specified coverpage will be treated as a Liquid template, which can be populated with metadata extracted from the bibliographic metadata about the collection; refer to Metadata and boilerplate.
collection-word-coverpage: …: Gives the location of the Word DOC coverpage for the entire collection [added in https://github.com/metanorma/metanorma/releases/tag/v1.7.5].

If left out, or given as null, no such coverpage is added.
collection-word-intropage: …: Gives the location of the Word DOC introductory section for the entire collection [added in https://github.com/metanorma/metanorma/releases/tag/v1.7.5].

As with HTML and Word DOC output of standalone documents, this is intended for prefatory material appearing before user-specified prefatory content: typically boilerplate content, and a placeholder for the table of contents.

If left out, or given as null, no such introductory material is added.
document-word-coverpage: …: Gives the location of the Word DOC coverpage for each individual document [added in https://github.com/metanorma/metanorma/releases/tag/v1.7.5].

If left out, or given as null, the default cover page that would appear in a standalone document is used. If given as "", no such coverpage is added.
document-word-intropage: …: Gives the location of the DOC introductory section for each individual document [added in https://github.com/metanorma/metanorma/releases/tag/v1.7.5].

If left out, or given as null, the default introductory section that would appear in a standalone document is used. If given as "", no such introductory section is added.
coverpage-style: Gives the style of the PDF and HTML coverpage, if multiple styles are offered [added in https://github.com/metanorma/metanorma/releases/tag/v1.5.7].
bare-after-first: Compiles the first HTML document in the collection complete (with coverpage and boilerplate), and all subsequent files with the bare option (i.e. without coverpage and boilerplate) [added in https://github.com/metanorma/metanorma/releases/tag/v1.3.5].

This is automatically included when sectionsplit is set in the Metanorma file, to break a single document up into multiple HTML files.

format

Specifies the possible output formats for the collection as a list, as an alternative to the -f argument of metanorma collection. Allowed values are:

xml: Metanorma Semantic XML
presentation: Metanorma Presentation XML. This is added automatically if any of the following formats is specified.
html: HTML.
doc: Word DOC.
pdf: PDF.

bibdata

Metadata about the collection. Entered in the Relaton format.

docid

type: (mandatory) docid/type is used by Metanorma to determine the flavour of the collection. Currently a collection can only contain documents of one flavour.

manifest

A manifest listing the documents contained in the collection, in nested hierarchy.

manifest can appear recursively in a manifest.

level

Names the current hierarchical level of the manifest.

title

Gives the title of the current level of the manifest.

docref

Lists the documents to be used at that level of the manifest. It is a list of file paths relative to the manifest file (fileref) and document identifiers (identifier).

The documents are expected to be Metanorma Semantic XML documents.

attachment: When set to true, the file is not a Metanorma document but an attachment, and therefore will not be compiled but directly included by Metanorma [added in https://github.com/metanorma/metanorma/releases/tag/v1.2.9].
sectionsplit: When set to true, the HTML output for the specified file is arranged as one HTML file per clause, with an index page created for the overall document.

The index page for the entire document links to the index page for the sectionsplit document.
index: Defaults to true. When set to false, the file is not to be included in any listing of manifest contents (i.e. in the collection cover page).

prefatory-content

Content to put at the beginning of the collection container.

final-content

Content to put at the end of the collection container.

Example 2. Example collection manifest

directives:
  - documents-inline
  - coverpage: index.html
  - coverpage-style: JACK
bibdata:
  title:
    type: title-main
    language: en
    content: ISO Collection 1
  type: collection
  docid:
    type: iso
    id: ISO 12345
  edition: 1
  date:
    - type: created
      value: "2020"
    - type: issued
      value: "2020"
  copyright:
    owner:
      name: International Organization for Standardization
      abbreviation: ISO
    from: "2020"
format:
  - xml
  - presentation
  - pdf
manifest:
  level: collection
  title: ISO Collection
  manifest:
    - level: subcollection
      title: Standards
      docref:
        - fileref: rice-en.final.xml
          identifier: ISO 17301-1:2016
        - fileref: dummy.xml
          identifier: ISO 17302
        - fileref: rice1-en.final.xml
          identifier: ISO 1701:1974
        - fileref: config.xml
          identifier: config.xml
          attachment: true
    - level: subcollection
      title: Amendments
      docref:
        fileref: rice-amd.final.xml
        identifier: ISO 17301-1:2016/Amd.1:2017
        sectionsplit: true
prefatory-content:
|
  == Clause
  Welcome to our collection

final-content:
|
  == Exordium
  Hic explicit

Site manifest

The collection manifest is expected to reference Metanorma Semantic XML documents.

The starting point for generating a collection is Metanorma AsciiDoc documents. In order to specify a collection and generate it as straightforwardly as possible, the collection manifest should be accompanied by a site manifest, named metanorma.ym1, specifying both the component AsciiDoc files, and the collection manifest, as dependency files.

Site compilation will compile both the component files, and the collection depending on them. This is done by running metanorma site generate in the same directory as metanorma.yml.

Since Metanorma site compilation compiles documents to a _site/documents directory, the collection manifest needs to reference the Semantic XML documents in that same _site/documents directory.

Example 3. Example site manifest

The following two files are examples of a site manifest and a collection manifest compiled through metanorma site generate.

metanorma.yml:

---
metanorma:
  source:
    files:
      - document.1.adoc
      - document.2.adoc
      - collection.yml

  collection:
    organization: "British Standards Institute"
    name: "Retrofitting dwellings for improved energy efficiency -- Specification and guidance"

In the site manifest, the files to be compiled are listed under metanorma.source.files; any YAML file in the list is assumed to be a collection manifest.

The collection is specified in the site manifest with two attributes: a name for the collection document, and an organization treated as the corporate author of the collection. Both will feature in the index file of the documents generated in the site (_site/index.html), and correspond to bibdata.title.content and bibdata.copyright.owner.name in the collection manifest.

Example 4. Example collection manifest

collection.yml:

---
directives:
  - documents-inline
bibdata:
  type: collection
  docid:
    type: bsi
    id: bsidocs
format:
  - xml
  - html
  - presentation
  - pdf
manifest:
  docref:
    - fileref: _site/documents/document.1.xml
      identifier: bsidocs-1
    - fileref: _site/documents/document.2.xml
      identifier: bsidocs-2
prefatory-content:
|

final-content:
|

Note

document.1.adoc and document.2.adoc are compiled to _site as part of site compilation. The fileref attributes in the collection manifest need to point to the Semantic XML where the site compilation deposits them, under _site/documents. The collection generation also generates the collection in the same location, so there is no need to specify a collection destination directory, --output-folder under metanorma collection.

Index page template

The HTML index page template is currently realised as a Liquid template, which forms a sidebar for the display of the HTML content of each file.

The following fields are defined:

doctitle, docnumber, etc.

Information derived from the Relaton YAML description in the manifest of the entire collection.

The field names are as defined for Liquid templates in Metanorma: see Metadata and Boilerplate.

navigation

A nested list giving hyperlinks to the constituent documents, following the specification in the manifest field of the collection manifest.

nav_object

The same nested list, presented as a recursive object, in order to allow users to select only a subset of the navigation list for presentation [added in https://github.com/metanorma/metanorma/releases/tag/v1.6.4].

It contains the following fields:

title: The list title.
docrefs: A hyperlinked list of the documents at that level of the manifest.
children: An array of child manifests.

prefatory-content

Prefatory content from the collection manifest [added in https://github.com/metanorma/metanorma/releases/tag/v1.5.6].

final-content

Final content from the collection manifest [added in https://github.com/metanorma/metanorma/releases/tag/v1.5.6].

Cross-references

Direct cross-references

A source document can link to a target document in the same collection, or a specific location within the target document.

Documents are processed one document at a time; so such a link is encoded as a bibliographical reference, to an external document, as described in Bibliography.

This means that an author needs to define a bibliographic entry for each hyperlinked document in the same collection; those bibliographic entries will be suppressed from display in the collection.

Note	If the documents are to be used in isolation, those bibliographic entries still need to be displayed: otherwise, the reference cannot be made sense of.

The bibliographic reference for another document in the same collection is specified using the following syntax.

* [[[myanchor,repo:(current-metanorma-collection/docid)]]]

where docid is the document identifier as it appears in the collection manifest.

If no such anchor is given in the document, but the document identifier in the collection manifest matches a document identifier in the bibliography, then collection processing will still recognise that the document is referencing that other document [added in https://github.com/metanorma/metanorma/releases/tag/v1.3.12].

For instance, if the manifest includes an instance of identifier: ISO 44001, and the bibliographic reference of another document includes * [], then collection processing will automatically link all references to ISO 44001 to the collection instance of the document.

This allows documents to be included in a collection, without requiring their references to be edited.

The location to link to in the target document can be specified as a clause number, as in a typical citation:

Example 5. Example of specifying a cross-reference to a clause in the target document

<<myanchor,clause=3.1>>

The processor will then navigate the target document, to try to resolve the reference.

Note	Currently only one level of nesting of locations is implemented: the processor will not resolve references like `clause=3.1,note-3`.

Alternatively, the location can be specified as an anchor, e.g. <<myanchor,anchor=ident>>. The hyperlink will then be made directly to the element with anchor ident in the the target document. That approach is to be preferred as simpler.

For instance, we wish to link from the French BIPM Brochure to the English BIPM Brochure, and specifically to an example in the English document. We start by assigning the target document example an anchor identifier:

[[english_example]]
[NOTE]
====
For example: The maximum electric potential difference is
stem:[ii(U)_("max") = 1000 " "rm(V)] but not
stem:[ii(U) = 1000 " "rm(ii(V)_(max))].
The mass fraction of copper in the sample of silicon is
stem:[w("Cu") = 1.3 xx 10^(-6)] but not
stem:[1.3 xx 10^(-6) " "rm(w)//rm(w)].
====

We then define a citation in the source document, using that anchor:

Ce n'est que lorsque l'on écrit le nom de l'unité en toutes lettres que l'on
applique les règles grammaticales ordinaires (voir un exemple en anglais page
<<english-doc,anchor=english_example>>).

Finally, we define a bibliographic entry in the source document for the English-language target document:

[bibliography]
== Bibliography

* [[[english-doc,repo:(current-metanorma-collection/si-brochure-en)]]] (Version anglaise de la brochure BIPM).

The identifier given to the target document needs to match that given in the collection manifest:

manifest:
  level: brochure
  title: Brochure/Brochure
  docref:
    - fileref: si-brochure-fr.xml
      identifier: si-brochure-fr
    - fileref: si-brochure-en.xml
      identifier: si-brochure-en

This form of direct cross-reference is also used to reference attachments [added in https://github.com/metanorma/metanorma/releases/tag/v1.3.2]. For example, if you wanted to link to a text file from a collection document, the manifest would look as follows:

manifest:
  level: brochure
  title: Brochure/Brochure
  docref:
    - fileref: si-brochure-fr.xml
      identifier: si-brochure-fr
    - fileref: attachment.txt
      identifier: ABC
      attachment: true

And the hyperlink to the attachment, and the bibliographic entry for it, would be as follows:

Download the attachment from: <<theattachement>>.

....

[bibliography]
== Bibliography

* [[[theattachment,repo:(current-metanorma-collection/ABC)]]]

Indirect cross-references

In some documents, anchors (targets for cross-references) are inserted in various files in the collection, and we do not necessarily know at the time of authoring which files those anchors will end up in. A good example of that is computer-generated documentation of schemas: schema documentation is organised by entity, and the documentation of one entity can cross-reference attributes in a different entity. But at the time of authoring, we may not know which document the target entity will appear in, so we cannot supply a bibliographic entity naming that document.

To deal with that circumstance, Metanorma implements a special class of cross-references, which are namespaced and which use containers:

<<namespace:container>>
<<namespace:container,text>>
<<namespace:container:locality>>
<<namespace:container:locality,text>>

The namespace is provided to deal with the fact that such anchors can have different provenance, and they may have particular rendering requirements. (So if we are documenting two different schemas, we may want to differentiate their references, and render them differently.)
The container relies on the fact that such anchors can be grouped together in a target document, under a clause. (For example, a schema instance.) For efficient processing, we treat each of those container clauses as a single bibliographic reference, and use the identifier of that clause as the bibliographic anchor. We also assign the container clause the namespace as a type, again for efficiency and to enforce consistent rendering. This is mandatory.
The locality is the identifier of the particular component addressed within the container. It is an identifier in the target document, and will typically point to a subclause of the container clause.
The text is the text to be rendered for the cross-reference. If not provided, Metanorma will provide a clause reference for the target.

To give a worked example:

We are generating documentation for a set of schemas in the EXPRESS language as a Metanorma collection. We wish to point to the identifier basic_attribute_schema.id_attribute.identified_item from our source document. We do not know (or care) what document that identifier will turn up in: we will have collection processing deal with that.

basic_attribute_schema.id_attribute.identified_item is an identifier within the basic schema, and we are grouping the definitions of the basic schema together, under a single clause in the target document.

The target document will thus contain a container clause with identifier basic, containing all those definitions, including basic_attribute_schema.id_attribute.identified_item. The container clause is made to be of type express (because its content comes from that language, and we want to follow the conventions of that language in any processing).

[[basic]]
[type=express]]
=== Basic Schema

....

[[basic_attribute_schema.id_attribute.identified_item]]
===== Identified Item

The cross-reference to that identifier, from either the same document or a different document in the same collection, is:

<<express:basic:basic_attribute_schema.id_attribute.identified_item,Identified Item>>

We do not need to indicate which document basic_attribute_schema.id_attribute.identified_item is in, unlike for direct cross-references. Because of the namespacing, we know that we are looking for the identifier basic_attribute_schema.id_attribute.identified_item inside a clause with id basic and type express: that narrows down our search while generating the collection. The basic collection identifier is actually optional; but if you don’t provide it, you will need to put [type=express] on any cross-reference target, and collection processing will be more expensive.

Multilingual documents

Metanorma currently supports multilingual documents in its PDF output, as document collections.

By default, Metanorma treats multilingual documents as a concatenation of documents, each in its own language;
Metanorma also supports rendering multilingual documents as parallel columns of aligned text.

In order to control such alignment, Metanorma supports the following markup [added in https://github.com/metanorma/metanorma-standoc/releases/tag/v1.10.8]:

An attribute tag can be added to any block.

This is used to indicate that blocks with the same tag value across documents in different languages are to be aligned in parallel columns, subject to the multilingual-rendering attribute.
An attribute multilingual-rendering can be added to any block.

This indicates how that block is to be rendered in a multilingual columnar text.

The options are:
- common for blocks that are shared across all languages;
- all-columns for blocks that span all columns of text, and are displayed consecutively;
- parallel for a block that is to be aligned to the block occupying the same position in the document hierarchy in each language;
- tag for all blocks sharing the same tag attribute as the current block.
The document attribute align-cross-elements indicates the Metanorma XML elements that are always to be aligned in multilingual text. It consists of a comma-delimited list of Metanorma XML tags; e.g. p,note,term.

Document embedding

Metanorma documents supports embedding of a document within another Metanorma document, using the embed:: command [added in https://github.com/metanorma/metanorma-standoc/releases/tag/v2.0.2].

The embed:: command acts similar to the AsciiDoc include:: macro, taking as its target argument the name of the file to be included.

embed::received.adoc[]

The named file is included at the location of the embed:: command, with the following consequences:

The document header of the included document is ignored.
If any top-level headings of the included document are identical to headings in the including document, the entire clause in the included document is skipped.

For example, if

= Document 1
Local Secretariat
:docnumber: 123A

[[id2]]
[language=en-UK]
== Introduction

This is an introduction added to the Received document

embed::received.adoc[]

The embedded document can be

== Colophon

embeds the document

= Document 1
Central Secretariat
:no-pdf:
:docnumber: 123
:issued-date: 2017-06-29

== Foreword

This is the foreword

[[id2]]
== Introduction

Original introduction

=== Introduction Subclause

Introduction subclause

== Content

This is content

then the resulting document uses the including document header (so the :docnumber: value will be 123A instead of 123, and the :issued-date: value will be ignored).

In addition, since both documents have a top-level clause labelled Introduction, the matching clause in the included document, and all its subclauses, are ignored.

The resulting document is:

= Document 1
Local Secretariat
:docnumber: 123A

[[id2]]
[language=en-UK]
== Introduction

This is an introduction added to the Received document

== Foreword

This is the foreword

== Content

This is content

== Colophon

Note

As with AsciiDoc include::[], AsciiDoc is unable to locate assets such as images if the embedded document is in a different directory from the containing document: AsciiDoc lacks a mechanism of specifying asset location relative to the current document, as opposed to the initial document. For that reason, if you embed documents that reference files, you will need to put both documents in the same folder, in a flat hierarchy.

Documents can be embedded at multiple levels; the current implementation presupposes that any document can only embed a single document, and that the embedding documents only involve substituted clauses, or prefatory clauses (e.g. the original foreword, supplemented by a regional foreword, and/or a national foreword).

It is possible to cite the embedded document from the embedding document, through a cross-reference to an anchor on the embed:: directive: if no display text is supplied, the cross-reference is populated with the document identifier of the embedded document [added in https://github.com/metanorma/metanorma-standoc/releases/tag/v2.3.10]:

For example, if

= Document 1
:docidentifier: SHQ XYZ 123A

== Introduction

This is an introduction added to the Received document <<id>>

[[id]]
embed::received.adoc[]

embeds the document

= Document 1
:docidentifier: XYZ 123

== Foreword

This is the foreword

Then [id] will be rendered as XYZ 123, the embedded document identifier; and the [id] cross-reference will hyperlink to the first header within the embedded document (the Foreword, in this case).