Is it necessary to check both '\r\n\r\n' and '\n\n' as HTTP Header/Content separator?

Question

The accepted answer on this page says we should check HTTP server responses for both \r\n\r\n and \n\n as the sequence that separates the headers from the content.

Like:

HTTP/1.1 200 Ok\r\n
Server: AAA\r\n
Cache-Control: no-cache\r\n
Date: Fri, 07 Nov 2014 23:20:27 GMT\r\n
Content-Type: text/html\r\n
Connection: close\r\n\r\n   <--------------

or:

HTTP/1.1 200 Ok\r\n
Server: AAA\r\n
Cache-Control: no-cache\r\n
Date: Fri, 07 Nov 2014 23:20:27 GMT\r\n
Content-Type: text/html\r\n
Connection: close\n\n   <--------------

In all the responses I've seen in Wireshark, servers use \r\n\r\n.

Is it really necessary to check for both? Which servers/protocol versions would use \n\n?

Answer 1

I started off with \r\n\r\n but soon found some sites that used \n\n. And looking at some professional libraries like curl, they also handle \n\n even if it's not according to the standard.

I don't really know the curl code, but see for example here: https://github.com/curl/curl/blob/7a33c4dff985313f60f39fcde2f89d5aa43381c8/lib/http.c#L1406-L1413

/* find the end of the header line */
  end = strchr(start, '\r'); /* lines end with CRLF */
  if(!end) {
    /* in case there's a non-standard compliant line here */
    end = strchr(start, '\n');

    if(!end)
      /* hm, there's no line ending here, use the zero byte! */
      end = strchr(start, '\0');
  }

Looking at that I think even \0\0 would be handled.

So:

• To handle "anything" out there, then yes.
• If you want to strictly follow the standard then no.

Answer 2

HTTP spec says:

The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR.

In practice I've never seen web server with CR line separator.