3

Everything ok with curl:

curl -v "http://user:password@localhost/_control.html" -d $'data1=1\r\n'

I tried this way in python:

url = "http://localhost/_control.html"

payload = {'data1': '1\r\n'}

headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}

r = requests.post(url, data=payload, headers=headers, auth=('user', 'password'))

But it doesn't work. Content-length in this case is 13 instead of 9 (with curl request)

Is it possible to send same data (with \r\n at the end) using python requests?

Dmitry
  • 334
  • 1
  • 10
  • I get only 9 bytes with curl, not 10. – Thomas Sep 10 '20 at 12:44
  • @Thomas you are right, I tried it with another data ("relay1" instead of "data1") and in this case it was just example and my misspelling. I corrected my answer, thank you! – Dmitry Sep 10 '20 at 12:51

1 Answers1

2

The \r and \n characters are being URL-encoded, as they should be, because application/x-www-form-urlencoded data cannot contain those characters directly:

Non-alphanumeric characters are replaced by %HH, a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., %0D%0A).

Logging the request sent from Python using this technique, we can see that it correctly sends these 13 bytes:

data1=1%0D%0A

In the case of curl, its manual page makes no mention of encoding of whatever you pass to -d/--data, so presumably you're expected to encode the string yourself before passing it to curl. We can confirm this with --trace-ascii -:

=> Send data, 9 bytes (0x9)
0000: data1=1

The \r\n pair doesn't show up clearly here, but we can infer that it's not encoded because of the byte count.

In short, the request you are sending with that curl command is not valid to begin with.

Thomas
  • 174,939
  • 50
  • 355
  • 478
  • could you please provide corrected payload for the request, it doesn't looks right: payload = {'data1' : '1%0D%0A'} – Dmitry Sep 10 '20 at 13:11
  • The `requests` will apply this encoding, you don't need to do it yourself, so what you originally wrote in Python is correct. If the server cannot handle it, then the problem is probably on the server side. – Thomas Sep 10 '20 at 13:13
  • But when I send the request with python the server does not respond correctly. With python in wireshark: `File Data: 14 bytes HTML Form URL Encoded: application/x-www-form-urlencoded Form item: "relay1" = "1 " Key: relay1 Value: 1\r\n` with curl: `File Data: 10 bytes HTML Form URL Encoded: application/x-www-form-urlencoded Form item: "relay1" = "1 " Key: relay1 Value: 1\r\n` Content length is not same. Sorry for ugly comment, cannot format it properly. – Dmitry Sep 10 '20 at 13:32