How is HTTP ChunkSize chosen?

Asked Sep 10 '19 at 01:20

Active Sep 10 '19 at 01:20

Viewed 246 times

I've been tinkering in C to handle PUT requests that use Chunked Encoding, and this answer has been immensely helpful.

While handling incoming datastreams, I noticed curl seems to use 16372 as the max Chunklen,

--> ChunkSize string: 3ff4 Dec: 16372
--> ChunkSize string: 3ff4 Dec: 16372
--> ChunkSize string: 1e7d Dec: 7805
--> ChunkSize string: 0 Dec: 0

and Golang seems to use 32768 as max:

--> ChunkSize string: 8000 Dec: 32768
--> ChunkSize string: 8000 Dec: 32768
--> ChunkSize string: 1e1d Dec: 7709
--> ChunkSize string: 0 Dec: 0

How are ChunkLens chosen by say, curl? Is there a rhyme or reason to choose 16372, which btw is not divisible by 8:

>>> 16372 / 8
2046.5

When sending a large file, why shouldn't we use a larger chunksize? Anything else I should be aware of when it comes to chunksize?

thanks!

asked Sep 10 '19 at 01:20

"is not divisible by 8" --- what is so special in "8 bytes"? – zerkms Sep 10 '19 at 01:23
just an artifact that my char array sizes are multiples of 8. – struggling_learner Sep 10 '19 at 01:30
1

It is chosen ny way the sender likes. There is no restriction. – user207421 Sep 10 '19 at 04:20
1

when curl uploads using chunked encoding, it uses up to its entire upload buffer size for this purpose. That is 16K by default. – Daniel Stenberg Sep 10 '19 at 20:50
@DanielStenberg, thank you. I see https://curl.haxx.se/libcurl/c/CURLOPT_UPLOAD_BUFFERSIZE.html now! – struggling_learner Sep 10 '19 at 22:22

0 Answers0