Using YAML as data source
Import YAML data into Metanorma has never been so easy
Introduction
When authoring technical documents, there is often a need to represent structured data one way or the other. If the data happens to be stored in one of the many machine-readable formats, one may be compelled to write a helper script that turns them into document elements like tables and bullet points (if your document format allows you to do so 😉).
Metanorma now supports using YAML^ files as a data source. In English, that means you can now bust out YAML data as AsciiDoc elements with the newly implemented template mini-language.
In this blog post, we'll see how this is done.
Meet the yaml2text block
The yaml2text block defines a template, and the template gets its data from the specified YAML file.
[yaml2text,my_data.yaml,my_yaml_context] <1>
----
This template will be rendered as AsciiDoc content. <2>
{... my_yaml_context ...} <3>
----That's the gist of it.
Next, we'll see how various YAML data structures can be represented:
Accessing object properties
EXAMPLE:
Given:
# strings.yaml
---
foo: bar
dead: beefThe following block...
[yaml2text,strings.yaml,data]
----
I'm heading to the {data.foo} for {data.dead}.
----\... will render as:
I'm heading to the bar for beef.Data from strings.yaml are being accessed via the context name data. The foo attribute is accessed via data.foo, which is interpolated in the template with curly braces.
Array length
EXAMPLE:
Given:
# array.yml
---
- lorem
- ipsum
- dolorThe following block...
[yaml2text,array.yml,data]
----
The length of the YAML array is {data.*}.
----\... will render as:
The length of the YAML array is 3.{data.*} evaluates to the length of the array.
Enumerating items in an array
EXAMPLE:
Given:
# strings.yaml
---
- lorem
- ipsum
- dolorThe following block...
[yaml2text,strings.yaml,arr]
----
{arr.*,item,EOS}
=== {item.#} {item}
This section is about {item}.
{EOS}
----\... will render as:
=== 0 lorem
This section is about lorem.
=== 1 ipsum
This section is about ipsum.
=== 2 dolor
This section is about dolor.Here, the expression {arr.*,item,EOS} tells the template engine to define a new context, item, to represent each individual item from the array arr. The context item is accessible (=== is under scope) within the lines between this expression and the first occurrence of {EOS}.
EOS is just an example --- it can be anything (any alphanum) --- as long as it is unique for each intended scope for the context item.
This template is then concatenated for each array item, in the original order of the array, as one might reasonably expect.
{item.#} gives the zero-based position of item item in the parent array arr.
Generally, given an array array_name, array_name[i] returns the value at index i (zero-based: starts with 0); negative indices count from the end: index -1 refers to the last item, -2 the second last, etc., etc.
Object size
EXAMPLE:
Given:
# object.yaml
---
name: Lorem ipsum
desc: dolor sit ametThe following block...
[yaml2text,object.yaml,data]
----
=== {data.name}
{data.desc} {data.*}
----\... will render as:
=== Lorem ipsum
dolor sit amet 2If data represents a YAML object, then {data.*} gives you the number of key-value pairs in that object.
Enumerating keys in an object
EXAMPLE:
Given:
# object.yaml
---
name: Lorem ipsum
desc: dolor sit ametThe following block...
[yaml2text,object.yaml,my_item]
----
{my_item.*,key,EOI}
=== {key}
{my_item[key]}
{EOI}
----\... will render as:
=== name
Lorem ipsum
=== desc
dolor sit ametkey gives the key of each key-value pair of the object my_item.
Like in common programming languages, my_item[key] gives the value corresponding to the key key.
Enumerating using attributes `.keys` and `.values`
EXAMPLE:
Given:
# object.yaml
---
name: Lorem ipsum
desc: dolor sit ametThe following block...
[yaml2text,object.yaml,item]
----
.{item.values[1]}
[%noheader,cols="h,1"]
|===
{item.*,key,EOK}
| {key} | {item[key]}
{EOK}
|===
----\... will render as:
.dolor sit amet
[%noheader,cols="h,1"]
|===
| name | Lorem ipsum
| desc | dolor sit amet
|===item.values gives an array of all values in the object item. It follows that item.values[1] gives you the second value.
An array with interpolated file names (for AsciiDoc consumption)
yaml2text blocks can be used for pre-processing document elements for AsciiDoc consumption.
EXAMPLE:
Given:
# strings.yaml
---
prefix: doc-
items:
- lorem
- ipsum
- dolorThe following block...
[yaml2text,strings.yaml,yaml]
------
[source,ruby]
----
\include::{yaml.prefix}{s.#}.rb[]
----
{EOS}
------\... will render as:
[source,ruby]
----
\include::doc-0.rb[]
----
[source,ruby]
----
\include::doc-1.rb[]
----
[source,ruby]
----
\include::doc-2.rb[]
----Putting it altogether -- Array of objects
EXAMPLE:
Given:
# array_of_objects.yaml
---
- name: Lorem
desc: ipsum
nums: [3, 5]
- name: dolor
desc: sit
nums: []
- name: amet
desc: lorem
nums: [2, 4, 6]The following block...
[yaml2text,array_of_objects.yaml,ar]
----
First array item of last array item is {ar[-1].nums[0]}.
Last array item of first array item is {ar[0].nums[-1]}.
{ar.*,item,EOF}
{item.name}:: {item.desc}
{item.nums.*,num,EON}
- {item.name}: index = {num.#}, index+1 = {num.# + 1},
{num} === {ar[num.#]}, prev = {ar[num.# - 1]}
{EON}
{EOF}
----\... will render as:
First array item of last array item is 2.
Last array item of first array item is 5.
Lorem:: ipsum
- Lorem: index = 0, index+1 = 1,
3 === 3, prev = 5
- Lorem: index = 1, index+1 = 2,
5 === 5, prev = 3
dolor:: sit
amet:: lorem
- amet: index = 0, index+1 = 1,
2 === 2, prev = 6
- amet: index = 1, index+1 = 2,
4 === 4, prev = 4
- amet: index = 2, index+1 = 3,
6 === 6, prev = 2Notice the various contexts and their corresponding scope delimiters (EOF for item, EON for num).
You might also have noticed that one can do simple arithmetics in interpolations and array indexing, like {num.# + 1} and {ar[num.# - 1]} in the example above.
Ending notes
In this blog post, we covered the most common use cases for including YAML data in a Metanorma document using the yaml2text block.
With the simple techniques shown in this article, you should be well equipped to handle any data structures YAML throws at you.
Happy authoring!