Putting shlex in debug mode

Question

I wanted to see if shlex was a good choice for something I'm trying to build, so I thought I'd put it in debug mode to play around with it. Only, shlex's constructor has this weird thing it does: it sets self.debug to 0 and then immediately checks if it's true.

…
self.debug = 0
self.token = ''
self.filestack = deque()
self.source = None
if self.debug:
    print 'shlex: reading from %s, line %d' \
          % (self.instream, self.lineno)

I know Python has some powerful metaprogramming features, but I can't figure out how this is intended to work – even if I override the constructor, there's no programmatic way to get between the setting of the value and its use.

Is there supposed to be a way to output the statement in the if self.debug condition (and if so, how?), is it a bug, or is there some third possibility I haven't considered?

Programmatically surely there's a way: http://ideone.com/wcPVRg. But it looks like a really old code, [written in 1999](https://github.com/python/cpython/blob/2bdca80eb5d7757d4542e9e12e425a54a11fe300/Lib/shlex.py) something and no one has touched that part at all. o_O — Ashwini Chaudhary, May 01 '15 at 23:37
Last one was bit hardcoded, here's slightly different way: http://ideone.com/lDCKvu — Ashwini Chaudhary, May 01 '15 at 23:44
That seems like a bug. It is perfectly reasonable for `shlex` to require you to set `s.debug = 1` only after construction… but in that case, there's no good reason for it to check `self.debug` during the initializer. — abarnert, May 02 '15 at 00:30
That being said, it's not a _terrible_ bug. You're only missing that first message, which you can easily reproduce yourself, right? — abarnert, May 02 '15 at 00:31
@abarnert no, it's not a terrible bug. It's just baffling in a "surely I'm missing something obvious" kind of way. — kojiro, May 02 '15 at 00:48
@kojiro: IIRC, `shlex` was originally a standalone script; presumably when ESR was adding stuff to his script and merging it over to the not-quite-identical module he'd submitted to the stdlib, he didn't notice that this line didn't make sense once merged, and nobody else was paying attention. So, everyone involved missed something obvious. :) — abarnert, May 02 '15 at 00:50
@kojiro: Meanwhile, are you going to report the bug? Someone should; if you don't want to, I'll do it. — abarnert, May 02 '15 at 01:17
@abarnert eventually if no one else does, but not in the next few hours. — kojiro, May 02 '15 at 01:24

abarnert · Accepted Answer · 2015-05-02T00:42:30.030

First, I'm pretty sure you've found a bug, and you should go report it. (You should make sure that it's still present in the latest 3.x code, but I just checked, and it is.) I don't see anything unreasonable about shlex objects not allowing you to set debug=1 until after they're initialized… but in that case, they shouldn't be checking self.debug in the initializer, since there's no way it can be set.

But this isn't that hard of a bug to work around. The only thing you lose this way is the first message, which only prints out public attributes you can print yourself. So, for example:

class debugging_shlex(shlex):
    def __init__(self, *args, **kwargs):
        # shlex is an old-style class in 2.7, so no super
        shlex.__init__(self, *args, **kwargs)
        self.debug = 1
        print('shlex: reading from %s, line %d' \
              % (self.instream, self.lineno))

More information for the bug report:

The unreachable code was added in this 2000 change, and the only change since then was to fit it to 80 columns.
There are no unit tests for the debug argument (not surprising, given that it's barely documented and just says "if you want to use this, read the source"… but you might want to add some anyway).

Well, I'll open a bug, but [the list of unhandled open bug reports on `shlex`](http://bugs.python.org/issue?%40search_text=shlex&ignore=file%3Acontent&title=shlex&%40columns=title&id=&%40columns=id&stage=&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40group=&%40pagesize=50&%40startwith=0&%40sortdir=on&%40queryname=&%40old-queryname=&%40action=search) doesn't give me much hope. — kojiro, May 02 '15 at 13:36
@kojiro: My guess is if you submit a patch removing those 2 lines, there's a greater than 50/50 chance it'll get accepted; if you don't submit a patch, nobody else will bother, and eventually it'll be another 47-month-old `shlex` bug that nobody cares about. — abarnert, May 03 '15 at 21:34

Putting shlex in debug mode

1 Answers1