Apache CGI in user directory "End of script output before headers"

Question

I know there are some questions about this topic, but none seems to solve my issue. See this or this or this.

I'm on Linux, Fedora21, and I'm trying to enable per user directory CGI script. I followed these instructions, but without success.

I get the error:

[cgi:error] End of script output before headers: test.cgi

test.cgi is an executable sh file, containing a very simple script:

#!/usr/bin/sh

echo "Content-type: text/plain"
echo ""
echo "Hello"

which has executable flag and runs without problems from shell. I also tried with Python: same result.

I also disabled selinux for good measure.

I also tried setting the debug level to Apache's ErrorLog, but all I get is only "granted" permissions before the error above.

I also configured the /etc/httpd/conf.d/userdir.conf file with

<Directory "/home/*/public_html">
    AllowOverride All
    Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
    Require all granted
</Directory>

<Directory /home/*/public_html/cgi-bin/>
    Options ExecCGI FollowSymLinks
    SetHandler cgi-script
    AddHandler cgi-script .cgi .exe .pl .py .vbs
    Require all granted
    AllowOverride All
</Directory>

and restarted the server. No success. Everything looks fine to me, I can't understand... What's wrong??

EDIT:

I forgot to add that the issue is just for per-user directory: if I move the same script to /var/www/cgi-bin directory, it works as expected.

EDIT 2:

The shell does exist:

$ ls /usr/bin/sh
/usr/bin/sh

Answer 1

Finally I solved that. Thanks to @JimB, because in their comment they pointed out SUEXEC, which I didn't know about (or simply ignored till now).

After reading a bit the suEXEC documentation, I understood the the problem had to be there. So, I took a look at the configuration:

# suexec -V
 -D AP_DOC_ROOT="/var/www"
 -D AP_GID_MIN=1000
 -D AP_HTTPD_USER="apache"
 -D AP_LOG_SYSLOG
 -D AP_SAFE_PATH="/usr/local/bin:/usr/bin:/bin"
 -D AP_UID_MIN=1000
 -D AP_USERDIR_SUFFIX="public_html"

and everything looked Ok (good uid/gid for my user, userdir_suffix is fine, etc). So I took a look at the system logs:

# journalctl -b | grep "suexec"
May 22 11:43:12 caladan suexec[5397]: uid: (1000/user) gid: (1000/user) cmd: test.cgi
May 22 11:43:12 caladan suexec[5397]: directory is writable by others: (/home/user/public_html/cgi-bin)

and that's the problem: my cgi-bin directory was writable by others.

I fixed by simply changing the permissions to 755.

Answer 2

For me, it worked when I changed the shebang line (#!/usr/bin/sh) to #!/usr/bin/env sh. I found that any shebang lines from What is the preferred Bash shebang? seemed to work (however note that sh is different from bash so if you want to use sh stick with it).

So this code worked for me:

#!/usr/bin/env sh
echo "Content-type: text/plain"
echo ""
echo "Hello"

Also, according to the post mentioned above, it seems /usr/bin/env sh seems preferred over /bin/sh. I have no idea about the per directory stuff.

Answer 3

This sometimes comes up when you try to call other Python module methods from your cgi where you might have left some 'print' statements (perhaps for debugging). So scan your code for any 'print' statement, sometimes this fixes the problem easily.

Answer 4

Let only the file owner have write permissions on the cgi script, but not the group, that is -rwxr-xr-x and not -rwxrwxr-x.

In user directories often the group will be a personal user group that only the user is member of anyway, but it seems like Apache gets nervous about seing the g+w bit but gives a somewhat bogus error message about this.

Answer 5

I had the same issue. I also needed this:

sudo setsebool -P httpd_read_user_content 1

which I found out by looking at the log (journalctl -b ) which told me...

SELinux is preventing /usr/bin/python2.7 from search access on the directory public_html.

Answer 6

I saw the message "End of script output before headers: myscript.py" for a Python 2.x CGI script that ran fine from the command line.

The problem turned out to be that it wasn't executing correctly by the web server, even though it was from the command-line. Whatever error message the system gave back to the server, it surely didn't pass for CGI headers (e.g., "Content-Type: text/html\r\n\r\n"). Hence, this failure message.

For me, correcting it meant changing the shebang from:

#!/usr/bin/env python

To a more system-specific (but verifiable):

#!/usr/local/bin/python

Perhaps you're encountering something similar.

(FreeBSD 9.x.)

Answer 7

Try running your script as the apache user.

There are a number of reasons this could be happening. @JimB mentions problem with the shebang. @premganz mentions problems with debug print statements that fire before your Content-Type string is written. In my case it was a db connection failure. It could be any other problem.

I found the best way to debug this is to run the script directly as the apache user on the command line, and see what errors it gives.

How to run your script as the Apache user

Assuming you're running apache2, and the apache user is www-data, and your cgi script is /myapp/myreport.py - you can do the following.

1. Allow logins as the www-data user, by changing its default shell. Edit /etc/passwd (temporarily)

   sudo su - 
   cd /etc
   cp -p passwd passwd.2017-10-22
   
   emacs passwd     # edit file - use your favorite editor
   

   Change:

   www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
   

   To:

   www-data:x:33:33:www-data:/var/www:/bin/bash
   

2. Log in as the www-data user

   sudo su - www-data
   

3. Set any environment vars. Convert your apache SetEnv statements to export statements. In my case, my apache config directory settings has these vars set

   SetEnv PYTHONPATH /myapp/lib
   SetEnv VCONF /myapp/conf/prod.yaml
   

   Run them as

   export PYTHONPATH=/myapp/lib
   export VCONF=/myapp/conf/prod.yaml
   

4. Then try your cgi script

   cd /myapp
   ./myreport.py
   

5. You should see whatever error Apache is experiencing. FIX THEM.

6. Set the www-data user's default shell back to /usr/sbin/nologin

Answer 8

This error occurs when we use print in older way.

print 'your test'

It should be for later version of Python.

print ('your test')

Answer 9

I was having a problem executing a python cgi in a userdir.

/var/log/httpd/error_log reveals:

[Fri Apr 26 13:09:41.840285 2019] [cgi:error] [pid 25421] [client 98.234.206.134:60837] End of script output before headers: index.cgi

The user being invoked in the URL was doug, user 42, group 42

I vaguely remembered suexec having a problem executing cgi scripts for a group or user below 1000.

So I created a new group called user1k as 1000. Then I did

adduser -g 1000 -u 1000 dwg
chmod 755 /home/dwg
mkdir /home/dwg/public_html

And I placed my python cgi script into this directory.

Now that the group and user were >= 1000 the script began executing correctly. I am not sure whether it was the user or the group which fixed the problem, but changing them both definitely solved it.

Note: It was the userid being below 1000 that was the problem. So make the the userID of a userdir cgi executee is > 999.

Here is an excerpt from the man suexec output.

Is the target userid ABOVE the minimum ID number? The minimum user ID number is specified during configuration. This allows you to set the lowest possible userid that will be allowed to execute CGI/SSI programs. This is useful to block out "system" accounts.

Is the target groupid ABOVE the minimum ID number? The minimum group ID number is specified during configuration. This allows you to set the lowest possible groupid that will be allowed to execute CGI/SSI programs. This is useful to block out "system" groups.