45

When a browser sends an HTTP request to a web server, what encoding is used to encode the HTTP protocol on the wire? Is it ASCII? UTF8? or UTF16? Or does it specify which encoding it uses in a predefined format (before any decoding takes place?)

P.S I'm not asking about the actual payload (e.g. HTML) of the request/response. I'm asking about the request line (i.e. GET /index.html HTTP/1.1) and headers (i.e. Host: google.com)

Gumbo
  • 643,351
  • 109
  • 780
  • 844

2 Answers2

38

HTTP 1.1 uses US-ASCII as basic character set for the request line in requests, the status line in responses (except the reason phrase) and the field names but allows any octet in the field values and the message body.

Gumbo
  • 643,351
  • 109
  • 780
  • 844
  • 2
    By "field names" you mean http headers? – Oleg Yablokov Jun 02 '21 at 09:23
  • 4
    @OlegYablokov might be late to the party, but yes. According to [RFC 7230](https://datatracker.ietf.org/doc/html/rfc7230#section-3.2), "header-field" (just http header) is a pair of "field-name" and "field-value" separated by ":" and single optional space between ":" character and "field-value". – Rafał Kopczyński Nov 20 '21 at 16:00
20

RFC 2616 includes this:

OCTET          = <any 8-bit sequence of data>
CHAR           = <any US-ASCII character (octets 0 - 127)>
UPALPHA        = <any US-ASCII uppercase letter "A".."Z">
LOALPHA        = <any US-ASCII lowercase letter "a".."z">
ALPHA          = UPALPHA | LOALPHA
DIGIT          = <any US-ASCII digit "0".."9">
CTL            = <any US-ASCII control character
                  (octets 0 - 31) and DEL (127)>
CR             = <US-ASCII CR, carriage return (13)>
LF             = <US-ASCII LF, linefeed (10)>
SP             = <US-ASCII SP, space (32)>
HT             = <US-ASCII HT, horizontal-tab (9)>
<">            = <US-ASCII double-quote mark (34)>

And then pretty much everything else in the document is defined in terms of those entities (OCTET, CHAR, etc.). So you could look through the RFC to find out which parts of an HTTP request/response can include OCTETs; all other parts must be ASCII. (I'd do it myself, but it'd take a long time)

For the request line specifically, the method name and HTTP version are going to be ASCII characters only, but it's possible that the URL itself could include non-ASCII characters. But if you look at RFC 2396, it says that.

A URI is a sequence of characters from a very limited set, i.e. the letters of the basic Latin alphabet, digits, and a few special characters.

Which I guess means that it'll consist of ASCII characters as well.

David Z
  • 128,184
  • 27
  • 255
  • 279