How can I prevent re-formating of a yaml file when ruamel.yaml is used?

Question

I am using solution in the related answer for How to auto-dump modified values in nested dictionaries using ruamel.yaml .

I have observe that when self.update(self.yaml.load(f) or {}) is called: it keeps reformatting the yaml file by sorting it based on the alphabetic order and changes its indention into 2.

=> Would it be possible to prevent reformatting?

I have also tried following line, which did not make any affect on it in order to keep indention as 4 space:

yaml = YAML(typ="safe")
yaml.indent(mapping=2, sequence=4, offset=2)
yaml.default_flow_style = False

config.yaml:

c:  # my comment
    b:  
      f: 5  
      e: 22  
a:
    z: 4
    b: 4  # my comment

code (exactly same code from How to auto-dump modified values in nested dictionaries using ruamel.yaml ):

#!/usr/bin/env python3

import sys
import os
from pathlib import Path
import ruamel.yaml


class SubConfig(dict):
    def __init__(self, parent):
        self.parent = parent

    def updated(self):
        self.parent.updated()

    def __setitem__(self, key, value):
        if isinstance(value, dict):
            v = SubConfig(self)
            v.update(value)
            value = v
        super().__setitem__(key, value)
        self.updated()

    def __getitem__(self, key):
        try:
            res = super().__getitem__(key)
        except KeyError:
            super().__setitem__(key, SubConfig(self))
            self.updated()
            return super().__getitem__(key)
        return res

    def __delitem__(self, key):
        res = super().__delitem__(key)
        self.updated()

    def update(self, *args, **kw):
        for arg in args:
            for k, v in arg.items():
                self[k] = v
        for k, v in kw.items():
            self[k] = v
        self.updated()
        return


_SR = ruamel.yaml.representer.SafeRepresenter
_SR.add_representer(SubConfig, _SR.represent_dict)


class Config(dict):
    def __init__(self, filename, auto_dump=True):
        self.filename = filename if hasattr(filename, "open") else Path(filename)
        self.auto_dump = auto_dump
        self.changed = False
        self.yaml = ruamel.yaml.YAML(typ="safe")
        self.yaml.default_flow_style = False
        if self.filename.exists():
            with open(filename) as f:
                self.update(self.yaml.load(f) or {})

    def updated(self):
        if self.auto_dump:
            self.dump(force=True)
        else:
            self.changed = True

    def dump(self, force=False):
        if not self.changed and not force:
            return
        with open(self.filename, "w") as f:
            self.yaml.dump(dict(self), f)
        self.changed = False

    def __setitem__(self, key, value):
        if isinstance(value, dict):
            v = SubConfig(self)
            v.update(value)
            value = v
        super().__setitem__(key, value)
        self.updated()

    def __getitem__(self, key):
        try:
            res = super().__getitem__(key)
        except KeyError:
            super().__setitem__(key, SubConfig(self))
            self.updated()
        return super().__getitem__(key)

    def __delitem__(self, key):
        res = super().__delitem__(key)
        self.updated()

    def update(self, *args, **kw):
        for arg in args:
            for k, v in arg.items():
                self[k] = v
        for k, v in kw.items():
            self[k] = v
        self.updated()


cfg = Config(Path("config.yaml"))

=> config.yaml file is updated as follows, where, its sorted and indention changed into 2, and comments are removed:

a:
  b: 4
  z: 4
c:
  b:
    e: 5
    f: 22

It would be good to have at least some example output, both how it is, and how you want it. Even better would be to have a functional program that generates the unwanted output. Without those it is a bit of guess to check if you are doing things wrong or if `ruamel.yaml` is malfunctioning. — Anthon, Nov 19 '21 at 11:57
Thank you for your quick response. I have updated my questions with example output and functional code. I have used your exact code on the link I mention on my question. Calling `cfg = Config(Path("config.yaml"))` line reorganizes yaml file's order based on alphabetic order and it also updated indention from 4 space to 2 space. If possible I just wanted to prevent this behavior. It also removes the comments — alper, Nov 19 '21 at 12:10
If you use `typ='safe'` you get the C-based loader/dumper, you'll need to use the (default) round-trip dumper with `self.yaml = ruamel.yaml.YAML()` — Anthon, Nov 19 '21 at 13:02
I have change it into `self.yaml = ruamel.yaml.YAML()` but now it gives `ruamel.yaml.representer.RepresenterError: cannot represent an object: {'b': {'f': 5, 'e': 22}}` error. Should I make change into dump() function as well? — alper, Nov 19 '21 at 13:35
no the dump function should be fine, as it uses `self.yaml` ( you do want to open the file as `"wb"` since the output is UTF-8). Have you updated the `add_represernter` call for Subconfig to use the `RoundTripRepresenter` ? — Anthon, Nov 19 '21 at 19:12
Ah I haven't updated `add_represernter` call into `RoundTripRepresenter `. I just have to do `_SR = representer.RoundTripRepresenter` right? — alper, Nov 19 '21 at 21:05
It works when I changed into `representer.RoundTripRepresenter ` except it still removes comments :-( — alper, Nov 19 '21 at 21:11

Anthon · Accepted Answer · 2021-11-19T13:03:49.900

It is not possible to prevent indentation "reformatting". ruamel.yaml is documented to normalize the formatting so that all (block style) mappings are indented the same, and all (block style) sequences are indented the same. You can however set each of these, which you don't seem to do for the mappings (the representation for Python dicts in YAML):

import sys
import ruamel.yaml

data = dict(a=dict(b=[42, 18]))

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=2)
yaml.dump(data, sys.stdout)

which gives indentation of four for both mappings and sequences:

Make sure to use the default (round-trip) loader/dumper to prevent sorting and to give you full control over indenting the output.

How can I prevent re-formating of a yaml file when ruamel.yaml is used?

1 Answers1