The comment by Alexander Farkas from 8/2 is a perfect, if succinct, answer. Range requests (also known as "Byte Serving") allow the client to request (any) part of the file.
The client makes (at least) three GET requests with HTTP 206 responses (provided the server is capable of handling range requests): one for the file headers (Content-Length is what matters, along with "Accept-Ranges: bytes"). Then the client requests the end of the file, usually less than the last MB of content (this seems to vary by browser); once the client has the moov atom from the end of the file, it requests the rest of the content. When you seek, the metadata allows the client to know how to map time to byte range, and issues a new request for partial content.
A reasonable transcript of what this looks like in practice is at Sample http range request session