Premature end of script headers for unknown reason

Question

Whenever I call the cgi script below by submitting a form to it I get an internal server error and the server log shows the line: Premature end of script headers: LoopFinderRetrieval.cgi, referer: http://loopfinder-prod-01.uit.tufts.edu/LoopFinderRetrieval.html

In short, the cgi file is meant to go to the folder indicated by a run-ID, given at the time of submission, open and read a file called JobStatus.txt, and then perform an action based on the result. This can either be returning a specific error to the user, or giving them their results. As I understand it, the error I'm seeing would be caused if I was, for example omitting the line:

"Content-type:text/html\r\n\r\n"

but the line is present, and another CGI script using the exact same PrintHeader() and PrintFooter() functions on the same server is running without error. Can anybody see any clear errors that might be causing this? If not, what I've read indicates that this might be a permissions issue. In that case I'll have to contact the administrators and have it fixed, but I don't want to do that unless I know it's the problem. Thanks.

#!/usr/bin/python2.6

# Import modules for CGI handling 
import cgi, cgitb
import os 
cgitb.enable()

#Functions to automatically print HTML headers and footers.
def PrintHeader():
    print "Content-type:text/html\r\n\r\n"
    print "<html>"
    print "<head>"
    print "<title>LoopFinder Running</title>"
    print "</head>"
    print "<body>"
def PrintFooter():
    print "</body>"
    print "</html>"

# Create instance of FieldStorage 
form = cgi.FieldStorage() 
ID_Number = form.getvalue('IDNum')

with open('/loopfinder_data/RunData/%s/JobStatus.txt' % ID_Number, 'r') as pre_textfile:
    textfile = pre_textfile.read()
    if textfile[0] == 'Running':
        PrintHeader()
        print '<h2>Your run is not complete. Please check back later.</h2>'
        PrintFooter()
    if textfile[0] == 'Stopped':
        PDBID = textfile[3]
        if textfile[1] == 'PDBError':
            PrintHeader()
            print '<h2>We were unable to download the PBDID you entered, which was %s</h2>' % PDBID
            print '<h2>Please check that this PDBID exists before trying again.</h2>'
            PrintFooter()
        elif textfile[1] == 'ChainCountError':
            PrintHeader()
            print '<h2>We were unable to either download or open the PBDID you entered, which was %s</h2>' % PDBID
            print '<h2>Please check that this PDBID exists before trying again.</h2>'
            PrintFooter()       
        elif textfile[1] == 'SingleChainError':
            PrintHeader()
            print '<h2>It appears that your PDB structure of interest contains only one chain.</h2>'
            print '<h2>LoopFinder requires a multi-chain interface from which to calculate energy values.</h2>'
            PrintFooter()
        elif textfile[1] == 'LoopFinderError':
            PrintHeader()
            print '<h2>LoopFinder experienced an unknown error while analyzing your PDB file.</h2>'
            print '<h2>Leave a comment including your run ID and we will try to solve this issue.</h2>'
            PrintFooter()   
        elif textfile[1] == 'PyRosettaError':
            PrintHeader()
            print '<h2>PyRosetta experienced an unknown error while analyzing your PDB file.</h2>'
            print '<h2>Leave a comment including your run ID and we will try to solve this issue.</h2>'
            PrintFooter()                   
    if textfile[0] == 'Completed':
        PrintHeader()
        print '<a href="http://<url_redacted>/loopfinder_data/RunData/%s/results/%s_Results.zip">\
Click here to download your results.</a>' % (ID_Number,ID_Number)
        PrintFooter()

Afraid not. I've submitted support tickets to the root level admins of the server (it's hosted by my university and I'm only an httpd admin). When/if I know what the cause was, I plan on coming back to say it so as to help the future googlers, but right now I'm still floating around in limbo. — Michael Bird, Jul 03 '15 at 12:28

André Laszlo · Answer 1 · 2015-07-01T07:55:38.843

The error message is vague enough, but it basically means that the output ended before the headers were ended. Just like you said.

What if your script crashes. Then no headers will be printed and this error may occur, I guess.

Add some logging to your script to get a better understanding of what is actually happening. You can wrap the whole script in a try block and log any exception, for example.

You have a lot of repeated code, maybe if you refactored your script a little, the bug will be easier to find. It seems to me like headers and footers are always printed, for example, maybe it can be done only once.

My guess is that the script crashes when your file is too short. When you hit a line that has textfile[1] == something on it and the file only has one line, you will get the following exception:

IndexError: list index out of range

But this is just a guess, proper logging will let you know.

Edit:

I see that you are using cgitb. Maybe use the logging from this module. Try disabling the browser output and send any exception to a log file instead. Change cgitb.enable() to:

cgitb.enable(display=0, logdir='/tmp/')

You won't need to wrap your program in a try block if you're using cgitb, in fact the two methods will probably interfere.

Thanks for the suggestions. I am fairly sure your guess isn't right here, because the program that creates JobStatus.txt won't change its status from 'Running' to 'Stopped' without adding two lines below it. I have tried to implement some basic logging as described in [this answer by nosklo](http://stackoverflow.com/questions/1508467/how-to-log-my-traceback-error) but am not seeing any traceback printed to the log, only the first Debug:root: line. I'm increasingly starting to think it's a server configuration more than a script issue. Thanks. — Michael Bird, Jun 30 '15 at 17:28
If you are seeing *any* output in the log, it means that your script is at least running. The permissions issue you mentioned before probably ends the cgi execution before the script even starts. — André Laszlo, Jul 01 '15 at 07:58

score 0 · Accepted Answer · edited Jun 20 '20 at 09:12

OK, so it turns out that we have code errors here, not a permissions error. Quite embarrassing ones in fact. The corrected code is below. The first issue is I was using file.read() to try and read a file line by line. I should have been using file.readlines(), and have additionally changed the line to [line.rstrip('\n') for line in pre_textfile] in order to remove newline characters. There was also an index error in line 33 which I have corrected. What remains unclear is why I couldn't get any sort of logging to work for me, which would have saved several people a fair amount of time.

!/usr/bin/python2.6

# Import modules for CGI handling 
import cgi, cgitb
import os 
cgitb.enable()

#Functions to automatically print HTML headers and footers.
def PrintHeader():
    print "Content-type:text/html\r\n\r\n"
    print "<html>"
    print "<head>"
    print "<title>LoopFinder Running</title>"
    print "</head>"
    print "<body>"
def PrintFooter():
    print "</body>"
    print "</html>"

# Create instance of FieldStorage 
form = cgi.FieldStorage() 
ID_Number = form.getvalue('IDNum')

with open('/loopfinder_data/RunData/%s/JobStatus.txt' % ID_Number, 'r') as pre_textfile:
    textfile = [line.rstrip('\n') for line in pre_textfile]
    if textfile[0] == 'Running':
        PrintHeader()
        print '<h2>Your run is not complete. Please check back later.</h2>'
        PrintFooter()
    if textfile[0] == 'Stopped':
        PDBID = textfile[2]
        if textfile[1] == 'PDBError':
            PrintHeader()
            print '<h2>We were unable to download the PBDID you entered, which was %s</h2>' % PDBID
            print '<h2>Please check that this PDBID exists before trying again.</h2>'
            PrintFooter()
        elif textfile[1] == 'ChainCountError':
            PrintHeader()
            print '<h2>We were unable to either download or open the PBDID you entered, which was %s</h2>' % PDBID
            print '<h2>Please check that this PDBID exists before trying again.</h2>'
            PrintFooter()       
        elif textfile[1] == 'SingleChainError':
            PrintHeader()
            print '<h2>It appears that your PDB structure of interest contains only one chain.</h2>'
            print '<h2>LoopFinder requires a multi-chain interface from which to calculate energy values.</h2>'
            PrintFooter()
        elif textfile[1] == 'LoopFinderError':
            PrintHeader()
            print '<h2>LoopFinder experienced an unknown error while analyzing your PDB file.</h2>'
            print '<h2>Leave a comment including your run ID and we will try to solve this issue.</h2>'
            PrintFooter()   
        elif textfile[1] == 'PyRosettaError':
            PrintHeader()
            print '<h2>PyRosetta experienced an unknown error while analyzing your PDB file.</h2>'
            print '<h2>Leave a comment including your run ID and we will try to solve this issue.</h2>'
            PrintFooter()                   
    if textfile[0] == 'Completed':
        PrintHeader()
        print '<a href="http://<url_redacted>/loopfinder_data/RunData/%s/results/%s_Results.zip">\
Click here to download your results.</a>' % (ID_Number,ID_Number)
        PrintFooter()

Oh! Missed the `read` instead of `readlines` :) At least I had the location of the bug right! :P Glad you found it. Logging is invaluable. — André Laszlo, Jul 08 '15 at 16:27

Premature end of script headers for unknown reason

2 Answers2

!/usr/bin/python2.6