How to make junos_config Ansible module to support non-ASCII characters?

Question

I'm using junos_config module in Ansible version 2.2.1.0 which in a nutshell takes a plain-text configuration file and uses it to configure a Juniper network device. It seems to use lxml during the process. The problem is, that this plain-text file contains a non-ASCII character and Ansible fails with following error:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
fatal: [10.10.10.111]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_FZ42iu/ansible_module_junos_config.py\", line 344, in <module>\n    main()\n  File \"/tmp/ansible_FZ42iu/ansible_module_junos_config.py\", line 335, in main\n    run(module, result)\n  File \"/tmp/ansible_FZ42iu/ansible_module_junos_config.py\", line 293, in run\n    return load_config(module, result)\n  File \"/tmp/ansible_FZ42iu/ansible_module_junos_config.py\", line 258, in load_config\n    diff = module.config.load_config(candidate, **kwargs)\n  File \"/tmp/ansible_FZ42iu/ansible_modlib.zip/ansible/module_utils/netcfg.py\", line 58, in load_config\n  File \"/tmp/ansible_FZ42iu/ansible_modlib.zip/ansible/module_utils/junos.py\", line 201, in load_config\n  File \"/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py\", line 388, in load\n    return try_load(rpc_contents, rpc_xattrs)\n  File \"/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py\", line 350, in try_load\n    got = self.rpc.load_config(rpc_contents, **rpc_xattrs)\n  File \"/usr/local/lib/python2.7/dist-packages/jnpr/junos/rpcmeta.py\", line 79, in load_config\n    rpc.append(E('configuration-text', contents))\n  File \"/usr/local/lib/python2.7/dist-packages/lxml/builder.py\", line 236, in __call__\n    v = t(elem, item)\n  File \"/usr/local/lib/python2.7/dist-packages/lxml/builder.py\", line 185, in add_text\n    elem.text = (elem.text or \"\") + item\n  File \"src/lxml/lxml.etree.pyx\", line 1031, in lxml.etree._Element.text.__set__ (src/lxml/lxml.etree.c:53225)\n  File \"src/lxml/apihelpers.pxi\", line 715, in lxml.etree._setNodeText (src/lxml/lxml.etree.c:24420)\n  File \"src/lxml/apihelpers.pxi\", line 703, in lxml.etree._createTextNode (src/lxml/lxml.etree.c:24283)\n  File \"src/lxml/apihelpers.pxi\", line 1443, in lxml.etree._utf8 (src/lxml/lxml.etree.c:31502)\nValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters\n", "module_stdout": "", "msg": "MODULE FAILURE"}

I tried to use # -*- coding: utf-8 -*- in junos_config module, but this did not help. I also tried to set the Python PYTHONIOENCODING to utf8.

Configuration file for junos_config module can be seen below:

groups {
    replace: classes {
        /* Configured by Ansible £ */
        system {
            login {
                class test {
                    idle-timeout 1;
                    permissions network;
                    allow-commands "show route.*|quit|exit";
                    deny-commands .*;
                }
            }
        }
    }
}

When it contains the £ character, then module fails with Unable to load config: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters error message.

How to make junos_config(or lxml) to support non-ASCII characters?

@KonstantinSuvorov Sample configuration file with "bad" character is added. If I remove the `£` character, then module runs correctly. — Martin, Jul 24 '17 at 15:33
I would try to force python3: http://docs.ansible.com/ansible/latest/python_3_support.html given this check: https://github.com/lxml/lxml/blob/572e10843774a5d6300125d89bdc423d53c92971/src/lxml/apihelpers.pxi#L1431 implies no python2 utf-8 support in lxml. — lossleader, Jul 24 '17 at 19:21
@lossleader That was it. Thanks! Could you post your comment as an answer so that I can accept this. — Martin, Jul 25 '17 at 13:24

Hoall · Answer 1 · 2017-07-24T16:03:41.120

Without inspecting the source code and not having your configuration file or the problematic string, I would say your problem is a non-ASCII and non-UTF-8 character. From the error message:

ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

I would say it's a control character.

After some research I came across this answer https://stackoverflow.com/a/28380315/3810217 Maybe it helps. With hexdump filename you can check for null bytes and so on.

Another solution would be to recreate/save the file with gedit. There you can set the wished encoding to utf-8 (at least for new files).

I added an example `junos_config` module configuration file to my initial post. — Martin, Jul 24 '17 at 15:38

score 1 · Accepted Answer · answered Jul 25 '17 at 17:06

1

Unfortunately, lxml is limited to ascii support in python2, but it has both ascii and utf-8 in python3. Luckily, Ansible-2.2 includes a tech preview of python-3 support, so the module should work if you enable the python3 interpreter.

answered Jul 25 '17 at 17:06

lossleader

13,182
2
31
45

How to make junos_config Ansible module to support non-ASCII characters?

2 Answers2