2

We just started getting container.yml "is not JSON serializable" from ansible-container 0.9.2 - which was the version we have been running for months and no errors have a occurred. Any ideas why this could just started?

Our stack trace is

Traceback (most recent call last):
  File "/home/jenkins/workspace/tylt/ansible-container/container/cli.py", line 299, in __call__
    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))
  File "/home/jenkins/workspace/tylt/ansible-container/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/home/jenkins/workspace/tylt/ansible-container/container/core.py", line 153, in hostcmd_build
    config = get_config(base_path, vars_files=vars_files, engine_name=engine_name, project_name=project_name)
  File "/home/jenkins/workspace/tylt/ansible-container/container/utils/__init__.py", line 49, in get_config
    project_name=project_name, vault_files=vault_files)
  File "/home/jenkins/workspace/tylt/ansible-container/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/home/jenkins/workspace/tylt/ansible-container/container/config.py", line 62, in __init__
    self.set_env('prod')
  File "/home/jenkins/workspace/tylt/ansible-container/container/docker/config.py", line 45, in set_env
    super(AnsibleContainerConfig, self).set_env(env, config=config)
  File "/home/jenkins/workspace/tylt/ansible-container/container/config.py", line 146, in set_env
    self._resolve_defaults(config)
  File "/home/jenkins/workspace/tylt/ansible-container/container/config.py", line 194, in _resolve_defaults
    logger.debug(u'Resolved template variables', template_vars=defaults)
  File "/home/jenkins/.local/lib/python2.7/site-packages/structlog/stdlib.py", line 61, in debug
    return self._proxy_to_logger("debug", event, *args, **kw)
  File "/home/jenkins/.local/lib/python2.7/site-packages/structlog/stdlib.py", line 119, in _proxy_to_logger
    **event_kw)
  File "/home/jenkins/.local/lib/python2.7/site-packages/structlog/_base.py", line 176, in _proxy_to_logger
    args, kw = self._process_event(method_name, event, event_kw)
  File "/home/jenkins/.local/lib/python2.7/site-packages/structlog/_base.py", line 136, in _process_event
    event_dict = proc(self._logger, method_name, event_dict)
  File "/home/jenkins/workspace/tylt/ansible-container/container/utils/visibility.py", line 42, in unorder_dict
    event_dict[key] = json.dumps(value)
  File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: ordereddict([('cron', ordereddict([('version', '3.0pl1-128ubuntu2')])), ('graphicsmagick', ordereddict([('version', '1.3.23-1build1')])), ('ruby2.4', ordereddict([('version', '2.4.4-1bbox1~xenial1')])), ('ruby2.4-dev', ordereddict([('version', '2.4.4-1bbox1~xenial1')]))]) is not JSON serializable

This is what our python environment looks like:

ansible-container (0.9.2)
awscli (1.14.70)
backports.ssl-match-hostname (3.5.0.1)
botocore (1.9.23)
certifi (2018.4.16)
chardet (3.0.4)
colorama (0.3.7)
docker (3.4.1)
docker-pycreds (0.3.0)
docutils (0.14)
futures (3.2.0)
idna (2.7)
ipaddress (1.0.22)
Jinja2 (2.10)
jmespath (0.9.3)
MarkupSafe (1.0)
pip (9.0.3)
pyasn1 (0.4.2)
python-dateutil (2.6.1)
PyYAML (3.12)
requests (2.19.1)
rsa (3.4.2)
ruamel.ordereddict (0.4.13)
ruamel.yaml (0.15.52)
s3transfer (0.1.13)
selenium (3.11.0)
setuptools (39.0.1)
six (1.11.0)
structlog (18.1.0)
urllib3 (1.23)
websocket-client (0.48.0)

This is an example of our container.yml

version: "2"
defaults:
  additional_packages:
    ruby2.4:
       version: '2.4.4-1bbox1~xenial1'
    ruby2.4-dev:
       version: '2.4.4-1bbox1~xenial1'
    graphicsmagick:
       version: '1.3.23-1build1'
    cron:
       version: '3.0pl1-128ubuntu2'
services:
  ruby:
    from: tylt
    working_dir: "/var/www"
    command:
    - /bin/bash
    - -exc
    - |
      RAILS_ENV=kube bundle exec unicorn -c config/unicorn.rb -E kube &
      RAILS_ENV=kube bundle exec sidekiq -C config/sidekiq.yml -c 10 -e kube &
      wait %1
    volumes:
    - ${PWD}:/tmp
    roles:
    - tylt.rails
    labels:
      com.ansible.container.commit: "{{ git_commit }}"
      com.ansible.container.pull-request: "{{ pull_request }}"
registries: {}
  • 1
    Interestingly, the OrderedDict from the error message is entirely json encodable: `json.dumps(OrderedDict([('cron', OrderedDict([('version', '3.0pl1-128ubuntu2')])), ('graphicsmagick', OrderedDict([('version', '1.3.23-1build1')])), ('ruby2.4', OrderedDict([('version', '2.4.4-1bbox1~xenial1')])), ('ruby2.4-dev', OrderedDict([('version', '2.4.4-1bbox1~xenial1')]))]))` – Danielle M. Aug 09 '18 at 17:58
  • 1
    ah, ansible-container uses a totally different ordereddict: https://github.com/ansible/ansible-container/blob/release-0.9.2/container/utils/ordereddict.py – Danielle M. Aug 09 '18 at 18:01
  • 1
    So while looking up what `ruamel.yaml (0.15.52)` was, it looks like your version was released today. Did you upgrade your env recently? – Danielle M. Aug 09 '18 at 18:07
  • 1
    Danielle genius! I'll have to add to our requirements.txt to lock it down. – Mark Solonkovich Aug 09 '18 at 18:15
  • 1
    I looked at `ruamel.yaml` release history and version 0.15.x has release at least once in a week, even daily. As stated on pypi web site _For production systems you should pin the version being used with `ruamel.yaml<=0.15`_ – Baptiste Mille-Mathias Aug 09 '18 at 18:26
  • 1
    @DanielleM. can you provide this as the answer would love to give you some karma – Mark Solonkovich Aug 09 '18 at 18:33
  • @baptistemm saw that thanks yeah strange that ansible-container doesn't lock it down in their requirements.txt https://github.com/ansible/ansible-container/blob/release-0.9.2/requirements.txt – Mark Solonkovich Aug 09 '18 at 18:34
  • Yes that would worth to report this issue on github I'd say – Baptiste Mille-Mathias Aug 09 '18 at 18:43

2 Answers2

1

Reposting from comments:

So while looking up what ruamel.yaml (0.15.52) was, it looks like your version was released today. Did you upgrade your env recently?

Danielle M.
  • 3,607
  • 1
  • 14
  • 31
0

Until new version of ansible-container is released we fixed this by adding the same line to ansible-requirements.txt

ruamel.yaml>=0.15.34,<=0.15.51

linkebon
  • 401
  • 1
  • 4
  • 12