66

After the --dump-header writes a file, how to read those headers back into the next request? I would like to read them from a file because there are a number of them.

I tried standard in: cat headers | curl -v -H - ...

I'm actually using the feature in Firebug to "Copy Request Headers" and then saving those to a file. This appears to be the same format.

jcalfee314
  • 4,642
  • 8
  • 43
  • 75

5 Answers5

103

Since curl 7.55.0

Easy:

$ curl -H @header_file https://example.com

... where the header file is a plain text file with an HTTP header on each line. Like this:

Color: red
Shoesize: 11
Secret: yes
User-Agent: foobar/3000
Name: "Joe Smith"

Before curl 7.55.0

curl had no way to bulk change headers like that from a file. They had to be modified one bye one with -H.

Your best approach with an old curl version is probably to instead write a shell script that gathers all the headers from the file and use them, like:

#!/bin/sh
while read line; do
  args="$args -H '$line'";
done
curl $args https://example.com

Invoke the script like this:

$ sh script.sh < header_file
Daniel Stenberg
  • 54,736
  • 17
  • 146
  • 222
  • 1
    can you add an example of the header file? i'd like to see the format. how to put in strings, null, bool, int, float; where to use quotes. i just keep getting this as a search result so I think it would help people – kirill_igum Aug 13 '20 at 23:50
  • 1
    Added. But really exactly how each header is supposed to look like or contain depends entirely on that specific header... – Daniel Stenberg Aug 14 '20 at 15:02
  • 1
    @DanielStenberg it would be really useful if you could add the above `header_file` example to the docs https://curl.se/docs/manpage.html#-H – icc97 Feb 09 '23 at 12:00
  • 1
    @icc97 coming... https://github.com/curl/curl/pull/10455 – Daniel Stenberg Feb 09 '23 at 13:24
  • 2
    Now visible at https://curl.se/docs/manpage.html#-H =) – Daniel Stenberg Feb 09 '23 at 18:51
62

how about this:

curl -v -H "$(cat headers.txt)" yourhost.com

where headers.txt looks like

Header1: bla
Header2: blupp

works in BASH.

Cpt. Senkfuss
  • 1,347
  • 3
  • 12
  • 20
  • 1
    Works in curl 7.22.0. I added a space so it would be 'bla bla' and I could not trip it up. This command was esier to test with `curl -v -H "$(cat headers.txt)" google.com` .. Lets see if anyone else comments to make sure this is a safe way to do it. It is not documented. – jcalfee314 Aug 06 '14 at 14:29
  • I explicitly stripped the `-v` before since I thought it's noise that's not adding to a correct answer, but you're right, it is easier. so I added it back – Cpt. Senkfuss Aug 08 '14 at 09:56
  • 3
    But it works more or less by coincidence since it feeds it in as a single header with embedded newlines. IMHO, A cleaner version would have a -H on each line in the headers.txt file too and leave it out from the command line. – Daniel Stenberg Mar 10 '15 at 14:44
  • I like this hack for its elegance, but there's one tiny problem: because headers are not explicitly listed individually, they don't override any default headers sent by curl. Result: duplicate headers. E.g. if you have a custom content-type header, combined with -d, you now send two, possibly conflicting, content-type headers. – hraban Jun 14 '18 at 14:19
  • 3
    This will work, HOWEVER, keep in mind that in a process listing (such as `ps aux`), the expanded variable may be seen by other users. *SO*, if you are providing credentials or auth-token in the header, this approach will _expose the credentials_ to other users - which would be unacceptable. – Yoshiya Jan 23 '20 at 13:55
28

Starting with curl 7.55.0 it can now read headers from a file:

curl -H @filename

It's that easy now.

Dmitrii Sutiagin
  • 571
  • 5
  • 8
  • 2
    Ok, but what's the syntax? 'header':'info' "header":"info" header:info or some other combination? I can't seem to find the right combination. – Chris Troutner Jul 23 '18 at 18:26
  • 1
    Chris, the syntax is the same as in CLI - "name: value", no quotes, 1 header per line. You can test what is sent by using -v flag: `curl -v -H @test.txt test.com` to see how it's actually sent and compare with `-H 'x: y' -H 'a: b'` - you should see identical output from curl as long as your `test.txt` contains 2 lines: `x: y` and `a: b`. – Dmitrii Sutiagin Jul 24 '18 at 22:11
14

As answered by @dmitry-sutyagin, if your curl is at least version 7.55.0 you can use the @ notation to read headers from a file:

curl -H @headerfile.txt https://www.google.com/  # requires curl 7.55.0

If your curl is NOT 7.55.0 or newer, there's a useful hack:

  • Use the option -K/--config <config file>, and put several -H/--header <header> lines in the text file.

For instance:

  1. curl --dump-header foo.txt https://www.google.com/
  2. If necessary, dos2unix foo.txt

  3. Convert the file to -H 'header' lines, manually or with a script:

    cat foo.txt |
  awk '$1 == "Set-Cookie:"' |
  perl -ne "chomp; next if /^\\s*\$/; if (/'/) { warn; next } print \"-H '\$_'\\n\";" |
  tee headerfile.txt

    This might output something like:

    -H 'Set-Cookie: 1P_JAR=2018-02-13-08; [...]'
-H 'Set-Cookie: NID=123=n7vY1W8IDElvf [...]'

  4. curl --config headerfile.txt https://www.google.com/

traal
  • 461
  • 3
  • 7
  • 3
    Using the default curl 7.54.0 on macOS Mojave, I had to use double quotes instead of single quotes in the config file, otherwise I got warnings about unescaped whitespace and the headers weren't set at all – CupawnTae Jun 05 '19 at 13:47
5
curl $(xargs -a headers.txt printf "-H '%s'") example.org
user2683246
  • 3,399
  • 29
  • 31