1

I have a script that I'm building an interface for so people can execute after uploading a CSV file. I'm able to execute everything and have the script run properly, but how do I display a continuous output? What changes should I make to my code?

Below are my files:

scripts.html - Scripts are executed from here and executed on the server via an AJAX call. Output is placed into div#output once the script is done executing.

<div class="table_container">
    <form method="post" enctype="multipart/form-data" data-ajax="false">{% csrf_token %}
        <h4>Rebilling</h4>
        <div class="clear"></div>
        <img class="loading-gif" src="{{ STATIC_URL }}img/loading-clear.gif" alt="" />
        <table>
            <tbody>
            <tr>
                <td style="width: 180px;"><label>Upload the CSV</label></td>
                <td>
                    <input type="hidden" name="script_type" value="renew_subscriptions">
                    <input type="file" name="csv_file" />
                </td>
            </tr>
            <tr>
                <td style="width: 180px;"></td>
                <td>
                    <input type="submit" name="Execute" />
                </td>
            </tr>
            </tbody>
        </table>
    </form> 
</div>

<h2>Script Output</h2>
<div id="output">
    {% autoescape off %}

    {% endautoescape %}
</div>

<script type="text/javascript">
    // Variable to store your files
    var files;

    // Add events
    $('input[type=file]').on('change', prepareUpload);

    // Grab the files and set them to our variable
    function prepareUpload(event)
    {
      files = event.target.files;
    }

    $('form').on('submit', submitForm);

    // Catch the form submit and upload the files
    function submitForm(event)
    {
        event.stopPropagation(); // Stop stuff happening
        event.preventDefault(); // Totally stop stuff happening
        $("#output").html("");

        var form = $(this);
        form.find(".loading-gif").css("display", "block");
        form.find("input[type='submit']").prop('disabled', true);

        // Create a formdata object and add the files
        var data = new FormData(form.get(0));

        $.ajax({
            url: '/crm/scripts',
            type: 'POST',
            data: data,
            cache: false,
            dataType: 'html',
            processData: false,
            contentType: false,
            success: function(data)
            {
                // console.dir(data);
                $("#output").html(data);
            },
            error: function(jqXHR, textStatus, errorThrown)
            {
                // Handle errors here
                console.log('ERRORS: ' + textStatus);
            },
            complete: function()
            {
                form.find(".loading-gif").css("display", "none");
                form.find("input[type='submit']").prop('disabled', false);
            }
        });


        return false;
    }
</script>

views.py - The AJAX is sent here and the command is executed through Django Management

def all_scripts(request):    # Accounts page
    # c = {}
    script_type = None
    csv_file = None
    out = StringIO()

    if request.is_ajax and request.method == 'POST':
        csv_file = request.FILES.get('csv_file')

        if csv_file:
            # print "over here"
            ### write the csv_file to a temp file
            tup = tempfile.mkstemp() # make a tmp file
            f = os.fdopen(tup[0], 'w') # open the tmp file for writing
            f.write(csv_file.read()) # write the tmp file
            f.close()

            ### return the path of the file
            filepath = tup[1] # get the filepath
            # print filepath

            if 'script_type' in request.POST:
                script_type = request.POST['script_type']
                if script_type == "change_credit":
                    credit_amount = None

                    if 'credit_amount' in request.POST:
                        credit_amount = request.POST['credit_amount']

                    if 'function' in request.POST:
                        function = request.POST['function']

                        if function == "remove":
                            management.call_command(script_type, filepath, credit_amount, remove=[True], stdout=out)
                        else:
                            management.call_command(script_type, filepath, credit_amount, stdout=out)

                elif script_type == "renew_subscriptions":
                    management.call_command(script_type, filepath, verbosity=1, interactive=False, stdout=out)

                print out.getvalue()
                return HttpResponse(out.getvalue())

    return render_to_response('crm/scripts.html', context_instance=RequestContext(request))

Just need the output to display continuously line by line. Any help is much appreciated.

Cheers, Zee

zee
  • 656
  • 2
  • 7
  • 30
  • So what exactly is your question? What is wrong with the code you showed us? Does it not work? Does it produce unexpected behavior? – Cory Kramer Oct 10 '14 at 11:47
  • Sorry, just edited the post to include a question. My question is essentially, what do I change so I can have a continuous output to display? – zee Oct 10 '14 at 12:06
  • @Cyber does that clarify? – zee Oct 10 '14 at 12:33

1 Answers1

3

“The web request is a scary place, you want to get in and out as quick as you can” - Rick Branson

What you have done here is created an architectural issue. Basically you are creating additional disk IO when writing your CSV file. You are doing this in the web request. 'Not a good idea'.

However its also the crux of the issue you are describing.

Quick n dirty: you could get a return value from a django management command like so. Pass that back to the success method of jquery's ajax call as your data.

However: please don't to that!

You need an async task system to hand off the write of that csv file. Additionally you want to write the data somewhere ( dbms / nosql ) that your webpage can listen for via ( polling, streaming or websockets ) This is not a trivial undertaking but the end result is well worth the effort. Here are some proven django-stack choices to solve this type of issue.

Building a Asynchronous Tasking/Queuing System

Polling for that data

This pycon talk covers these technologies.

Community
  • 1
  • 1
Jeff Sheffield
  • 5,768
  • 3
  • 25
  • 32