Testing Django commands file or stdin

Question

I am writing a django command that takes a bunch of input and processes it.

It seems natural that because of the volume of the data, the input should either come in as a file or as stdin.

I would like to easily test it, and by easily, I mean, without having to create a bunch of files in my test environment.

Now, I remember somewhere (can't find it properly documented, but I did find the "PR"), that the "-" is supposed to read from stdin, but I can't get it to work.

It seems the command should do something like this:

class Command(BaseCommand):
    def add_arguments(self, parser):
        parser.add_argument("foo", type=file)

    def handle(self, *args, **options):
        file = options["foo"]
        # Then read the file, and process it

But then when I run the command on the command line, it doesn't like the - parameter (says it isn't a file).

The command docs recommend writing to self.stdout for better testing. I tried something similar for self.stdin but couldn't get that to work either.

Assuredly this is a common pattern, but I couldn't find any good helps on how to do this best. It seems like "There should be one-- and preferably only one --obvious way to do it.", but I can't find it. Is there something I'm missing?

I would create a bunch of files as my test fixture, if that solves the problems. I don't think that way is bad — Ming, Feb 14 '17 at 00:56
@Ming I want my tests to be simple, so they'll mostly just be files with just one line in them, even a single word, and that seems annoying to have to go look in the file to see what's inside. A test that passes a single item seems much cleaner. — McKay, Feb 14 '17 at 00:59
@Ming my tests of the "-" were run on the command line (edited question to better reflect that) — McKay, Feb 14 '17 at 01:09

score 3 · Accepted Answer · answered Feb 14 '17 at 01:15

3

Having a look at the docs for the type= argument to add_argument, it says "the argparse module provides the factory FileType".

So I did the following and ./manage.py test_stdin - then worked as you'd expect.

import argparse

class Command(BaseCommand):
    def add_arguments(self, parser):
        parser.add_argument("foo", type=argparse.FileType('r'))

    def handle(self, *args, **options):
        input_file = options["foo"]
        while True:
            line = input_file.readline()
            if len(line.strip()) == 0:
                break
            else:
                self.stdout.write("I just read %d chars, line %s" % (len(line),line))

answered Feb 14 '17 at 01:15

neomanic

346
1
7

This is great. Thanks. `argparse.FileType('r')` isn't very intuitive. I was (obviously) hoping for something simpler like `file`. Thanks. – McKay Feb 14 '17 at 01:41
It's because `file` doesn't have special handling for '-', whereas looking at the source for argparse it has a check for `if string == '-'` and returns stdin if the mode is 'r'. Makes sense when you realise '-' is a convention in executable args and not elsewhere. But thanks for the accept, was fun working this out. – neomanic Feb 14 '17 at 02:26

score 3 · Answer 2 · edited May 23 '17 at 11:53

Django command argument parser is a wrapping of argparse module. According to this post: Optional stdin in Python with argparse, you can define an argument to accept stdin or a actual file as an input.

Sample command for your reference:

management/commands/stdintest.py

from django.core.management.base import BaseCommand
import sys
import argparse


class Command(BaseCommand):
    help = 'Test stdin as input file'

    def add_arguments(self, parser):
        parser.add_argument('foo', nargs='?', type=argparse.FileType('r'), default=sys.stdin)

    def handle(self, *args, **options):
        foo = options.get('foo')
        sys.stdout.write(foo.read())

You can call the command without specifying the input file, and the command will grab input from stdin

$ python manage.py stdintest
1234
abcd     <- press ctrl+d to end stdin input
1234
abcd

or specify the foo argument with an actual file

$ echo '12345' > test.txt
$ python manage.py stdintest test.txt

I am not sure is that what you need. If I have misunderstood, please leave a comment. Hope it would help.

This is very helpful, thanks, but the other answer is slightly more of what I asked for. — McKay, Feb 14 '17 at 01:40

Testing Django commands file or stdin

2 Answers2

management/commands/stdintest.py