jinjyaml

An application-specific tag for Jinja2 templates within PyYAML.

This can be useful if you want to render only specially tagged nodes in the document, rather than treating the entire YAML string as a template.

Usage

Basic Example 1

1. Add Jinja2 template constructor for tag "!j2"

   import yaml
   import jinjyaml as jy
   
   ctor = jy.Constructor()
   yaml.add_constructor("!j2", ctor, yaml.SafeLoader)

2. create YAML file 1.yml, with such contents:

   array: !j2 |
     {% for i in range(n) %}
     - sub{{i}}: {{loop.index}}
     {% endfor %}

3. load and render the YAML file

   with open("1.yml") as fp:
       data = yaml.load(fp, Loader=yaml.SafeLoader)
       # or for the short:
       # data = yaml.safe_load(fp)
   
   jy.extract(data, context={"n": 3}, inplace=True)
   
   print(data)

We'll get:

{"array": [{"sub0": 1}, {"sub1": 2}, {"sub2": 3}]}

Advanced Usage

Include files

Jinja2's include filter function

We have such YAML files:

• sub-1.yml:

  "1.1": one
  "1.2": two

• sub-2.yml:

  "2.1":
    "2.1.1": three
    "2.1.2": four

• main.yml:

  foo: !j2 |
  
    {% filter indent %}
    {% include "sub-1.yml" %}
    {% endfilter %}
  
    {% filter indent %}
    {% include "sub-2.yml" %}
    {% endfilter %}

execute python code:

from pprint import pprint

import jinja2
import jinjyaml as jy
import yaml

env = jinja2.Environment(loader=jinja2.FileSystemLoader("."))

ctor = jy.Constructor()
yaml.add_constructor("!j2", ctor, yaml.SafeLoader)

with open("main.yml") as fp:
    doc = yaml.safe_load(fp)

obj = jy.extract(doc, env)
pprint(obj)

We'll get:

{"foo": {"1.1": "one",
         "1.2": "two",
         "2.1": {"2.1.1": "three", "2.1.2": "four"}}}

pyyaml-include

ℹ️ Note:
Jinja2's include and indent features do not handle indentation well in languages sensitive to it, such as Python or YAML. Therefore, using these features in complex cases is not recommended.

For such scenarios, consider using pyyaml-include. This package provides a PyYAML extension that allows you to include other YAML files while preserving proper indentation. Using this extension can help maintain the integrity of your YAML files more effectively.

1. install pyyaml-include:

   pip install pyyaml-include

2. add both pyyaml-include and jinjyaml's constructor:

   import yaml
   import jinjyaml as jy
   import pyyaml_include
   
   yaml.add_constructor("!j2", jy.Constructor)
   yaml.add_constructor("!inc", pyyaml_include.Constructor(base_dir="path_to_you_dir"))

3. Assume that we have YAML files same to previous example, the main.yml can be modified as below:

   foo: !j2 |
     {% for i in range(n) %}
     - !inc sub-{{loop.index}}.yml
     {% endfor %}

4. include and load other YAML files:

   Assume that we have YAML files same to previous example:

   with open("main.yml") as fp:
       doc = yaml.safe_load(fp)
   
   obj = jy.extract(doc, env)
   pprint(obj)

Then we'll get:

{
  "foo": [
    {"1.1": "one", "1.2": "two" },
    {"2.1": {"2.1.1": "three", "2.1.2": "four"}}
  ]
}

In this situation, there is no need to use jinja2.Environment and jinja2.FileSystemLoader to render the template, nor is it necessary to apply the indent filter within the template. This is because pyyaml-include has already parsed the included files into objects.

❇️ Conclusions:
You can use jinja2's include and indent to literally include other YAML files as raw text, or use pyyaml-include to include other YAML files as already-parsed objects.