Include YAML files from snakeyaml

Question

I would like to have YAML files with an include, similar to this question, but with Snakeyaml: How can I include an YAML file inside another?

For example:

%YAML 1.2
---
!include "load.yml"

!include "load2.yml"

I am having a lot of trouble with it. I have the Constructor defined, and I can make it import one document, but not two. The error I get is:

Exception in thread "main" expected '<document start>', but found Tag
 in 'reader', line 5, column 1:
    !include "load2.yml"
    ^

With one include, Snakeyaml is happy that it finds an EOF and processes the import. With two, it's not happy (above).

My java source is:

package yaml;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;

import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.AbstractConstruct;
import org.yaml.snakeyaml.constructor.Constructor;
import org.yaml.snakeyaml.nodes.Node;
import org.yaml.snakeyaml.nodes.ScalarNode;
import org.yaml.snakeyaml.nodes.Tag;

public class Main {
    final static Constructor constructor = new MyConstructor();

    private static class ImportConstruct extends AbstractConstruct {
        @Override
        public Object construct(Node node) {
            if (!(node instanceof ScalarNode)) {
                throw new IllegalArgumentException("Non-scalar !import: " + node.toString());
            }

            final ScalarNode scalarNode = (ScalarNode)node;
            final String value = scalarNode.getValue();

            File file = new File("src/imports/" + value);
            if (!file.exists()) {
                return null;
            }

            try {
                final InputStream input = new FileInputStream(new File("src/imports/" + value));
                final Yaml yaml = new Yaml(constructor);
                return yaml.loadAll(input);
            } catch (FileNotFoundException ex) {
                ex.printStackTrace();
            }
            return null;
        }
    }

    private static class MyConstructor extends Constructor {
        public MyConstructor() {
            yamlConstructors.put(new Tag("!include"), new ImportConstruct());
        }
    }

    public static void main(String[] args) {
        try {
            final InputStream input = new FileInputStream(new File("src/imports/example.yml"));
            final Yaml yaml = new Yaml(constructor);
            Object object = yaml.load(input);
            System.out.println("Loaded");

        } catch (FileNotFoundException ex) {
            ex.printStackTrace();
        }
        finally {
        }
    }
}

Question is, has anybody done a similar thing with Snakeyaml? Any thoughts as to what I might be doing wrong?

Answer 1

I see two issues:

final InputStream input = new FileInputStream(new File("src/imports/" + value));
            final Yaml yaml = new Yaml(constructor);
            return yaml.loadAll(input);

You should be using yaml.load(input), not yaml.loadAll(input). The loadAll() method returns multiple objects, but the construct() method expects to return a single object.

The other issue is that you may have some inconsistent expectations with the way that the YAML processing pipeline works:

If you think that your !include works like in C where the preprocessor sticks in the contents of the included file, the way to implement it would be to handle it in the Presentation stage (parsing) or Serialization stage (composing). But you have implemented it in the Representation stage (constructing), so !include returns an object, and the structure of your YAML file must be consistent with this.

Let's say that you have the following files:

test1a.yaml

activity: "herding cats"

test1b.yaml

33

test1.yaml

favorites: !include test1a.yaml
age:       !include test1b.yaml

This would work ok, and would be equivalent to

favorites:
  activity: "herding cats"
age: 33

But the following file would not work:

!include test1a.yaml
!include test1b.yaml

because there is nothing to say how to combine the two values in a larger hierarchy. You'd need to do this, if you want an array:

- !include test1a.yaml
- !include test1b.yaml

or, again, handle this custom logic in an earlier stage such as parsing or composing.

Alternatively, you need to tell the YAML library that you are starting a 2nd document (which is what the error is complaining about: expected '<document start>') since YAML supports multiple "documents" (top-level values) in a single .yaml file.