1

I have a legacy Perl CGI script written with CGI.pm, which replies its progress by printing some lines of text through the CGI interface. Its legacy client would then read the response line by line and do whatever.

I'm unable to setup nginx to make it work, because of buffering issues.

My setup:

  1. CentOS 7 + EPEL
  2. yum install fcgi fcgi-devel spawn-fcgi
  3. fcgiwrap recently needed a patch to support a NO_BUFFERING environment variable, to turn off buffering. This is described in this answer on ServerFault. However, it's puzzling:

    1. the debian bug report linked in that answer claims the patch has been applied upstream by fcgiwrap, but there is nothing related to it in the upstream github repository, excepting for an open issue which has never been replied to. Anyway, the patch has been applied to debian packages, but the EPEL package does not include it, so:
    2. git clone the repository
    3. applied debian_patches_add-environment-variable-NO_BUFFERING-to-disable-out.patch from the above debian bugreport.
    4. autoreconf -i && ./configure && make && make install
  4. Configured span-fcgi, this is \etc\sysconfig\spawn-fcgi:

OPTIONS="-u nginx -g nginx -a 127.0.0.1 -p 9001 -P /var/run/spawn-fcgi.pid -- /usr/local/sbin/fcgiwrap -f"
  1. Configured nginx for fastcgi:
location ~ .cgi$ {
  fastcgi_pass 127.0.0.1:9001;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  fastcgi_request_buffering off;
  fastcgi_read_timeout 900s;
  fastcgi_param NO_BUFFERING 1;
  gzip off;

  include /etc/nginx/fastcgi_params;
}

Other info:

  1. The perl script does $| = 1; so afaik the script itself should not be buffering its own output.
  2. The perl script also sends the X-Accel-Buffering: no HTTP header, which is often referred to as required to disable buffering with nginx, although afaik it should not be needed with fastcgi_buffering off in the configuration file.
  3. By listening on the 9001 port with netcat (nc -l 9001 > request.txt), I've checked that nginx correctly passes the NO_BUFFERING variable in the request along with all the other parameters.
  4. By doing cat request.txt | nc localhost 9001, I see that fcgiwrap is what is buffering the output: if I kill the spawned cgi process midway, I get all the output generated until then.
  5. The CGI script is launched and working fine otherwise, just the client is not receiving the data line by line (but I need the fastcgi_read_timeout 900s line above to avoid timeouts).
  6. I'm not sure I fully understand the fcgiwrap source code, but the patch is trivial: it just calls FCGI_fflush after every write if the NO_BUFFERING environment variable was set. However, just to be sure, I replaced unbuffered = !!getenv("NO_BUFFERING"); with unbuffered = 1;, with no success.

So can someone spot what I'm missing?

gigabytes
  • 3,104
  • 19
  • 35

0 Answers0