11

I am writing my doctests like this:

>>> some_function(a=1, b=2)
{u'id': u'123', u'name': u'abc'}

This works fine for Python version 2.5, 2.6 & 2.7 but fails for Python 3 with following error:

Expected:
    {u'id': u'123', u'name': u'abc'}
Got:
    {'id': '123', 'name': 'abc'}

Problem is that if I write my doctests like this:

>>> some_function(a=1, b=2)
{'id': '123', 'name': 'abc'}

They will work only for Python3 and fail on Python2 version. My question is how do I make it cross version compatible?

Irfan
  • 684
  • 5
  • 15
  • 2
    You'll have to use the same techniques as you'd use for making your code work in both Python 2.x and 3.x. That can get ugly, fast. Personally, I'd stick with unittests instead of doctests and use Sphinx for API documentation. – Martijn Pieters Nov 20 '12 at 13:14
  • 2
    I do use unittest for testing, this is just to validate the examples. – Irfan Nov 20 '12 at 13:15
  • Have you tried with Python 3.3? 3.3 reintroduces the u"" syntax (http://docs.python.org/3/whatsnew/3.3.html). Don't know if the change will affect doctests, but it is worth a try. – codeape Nov 20 '12 at 13:18
  • 5
    @codeape: You can use `u''` to *specify* a literal, but `repr(somestring)` will not use them, so the tests still will fail. Compare that to using `r''` raw python literals; the `r` will be dropped from the representation as well. – Martijn Pieters Nov 20 '12 at 13:19
  • OK, I see. I guess the only way to do this with doctests would be to rewrite the tests, then. For instance ``>>> result = some_function(a=1, b=2) >>> print(result["id"])`` etc. – codeape Nov 20 '12 at 13:28
  • Another approach that could work: ``>>> foo() == {"Hermione": "hippogryph", "Harry": "broomstick"}\n True`` (taken from the doctest documentation) – codeape Nov 20 '12 at 13:31
  • +1 I have the same problem, with large integers having an 'L' at the end in python 2.x and not in python3. I guess doctests are just fragile and broken for any code which needs to work across versions. – wim Apr 16 '13 at 04:09

3 Answers3

8

I ran into the same problem with doctests in IPython. There's no neat solution, but I wrapped all of the u' prefixes in {}, i.e. {u}', and made a little function that would include or exclude them as appropriate.

You can see the u_format() function and a doctest using it.

But that's rather messy, so I've moved many tests away from doctests.

Alternatively, you can test it like this:

>>> some_function(a=1, b=2) == {'id': '123', 'name': 'abc'}
True

If you need some unicode strings in the keys, you can use u'abþ', and use distribute to run 2to3 on the doctests. But that only works on input code, not output reprs.

Thomas K
  • 39,200
  • 7
  • 84
  • 86
4

If you use pytest, you can just do:

>>> some_function(a=1, b=2) # doctest: +ALLOW_UNICODE
{u'id': u'123', u'name': u'abc'}

And the u will be stripped if you are running Python 3, and kept in Python 2.

Bite code
  • 578,959
  • 113
  • 301
  • 329
  • This did not work for me, see my follow-up question here: https://stackoverflow.com/questions/42158733/unicode-literals-and-doctest-in-python-2-7-and-python-3-5 – matth Jul 31 '18 at 10:50
3

I run into the same issue with doctests in NLTK; it was solved by using a custom doctest output checker (that treats u'foo' and 'foo' the same) which is installed by a custom nose plugin: https://github.com/nltk/nltk/blob/develop/nltk/test/doctest_nose_plugin.py

This solution is not pretty, but it works quite well (there are about 0.5 megabytes of doctests in NLTK) and it doesn't make doctests less readable.

EDIT: found a simplified standalone version of this nose plugin: https://github.com/gnublade/doctest-ignore-unicode

Mikhail Korobov
  • 21,908
  • 8
  • 73
  • 65