3

I need to send a request with query string containing square brackets: [, And it should not be encoded as %5b.

As the answer in this question <How to prevent python requests from percent encoding my URLs?> says, I have tried putting the query string in the URL instead of passing params argument to get(), but requests still decides to encode the [ character anyway.

Wireshark:

  1. Sent with python requests:
url = "http://localhost:2017/api/httpLatency?whiches=[%7B%22id%22:26,%22_type%22:%22subscriptionServer%22,%22sub%22:1%7D]"
r = requests.get(url)

enter image description here

  1. Another packet containing an unencoded [ in URL:

enter image description here

I also tried the solution in this GitHub issue of request, which is to prepare the Request object and send it yourself. It doesn't work either.

After doing some testing, I found out that some characters, like ( , : +, won't be URL-encoded if passed directly as part of URL, but characters like { [ will be URL-encoded anyway no matter passed as URL or as params arguments.

I wonder if this is a deliberate design of requests? If it is, why should requests force characters like [ { to be encoded, is it due to the safety issue?

And if it is not, is there a workaround for this problem? After hours of searching, I still couldn't find a way to make a request with an un-encoded [ character in the URL. Using urllib would be way too complex for me.

Any help would be appreciated!

oeter
  • 627
  • 2
  • 8
  • 23
  • 1
    According to RFC 1738, square brackets are considered to be "unsafe". I quote from the RFC "All unsafe characters must always be encoded within a URL". Therefore, *requests* appears to be fully compliant in this respect –  Nov 25 '21 at 15:29
  • @DarkKnight Thanks for the info, I'll read the RFC document you mentioned. The link is [URL Character Encoding Issues](https://datatracker.ietf.org/doc/html/rfc1738#section-2.2), just for convenience. – oeter Nov 25 '21 at 15:36
  • See [here](https://docs.python-requests.org/en/latest/user/advanced.html) under PreparedRequests. – CypherX Nov 26 '21 at 02:05
  • @CypherX Thanks for the link, but I don't think this part [PreparedRequests](https://docs.python-requests.org/en/latest/user/advanced/#prepared-requests) is really helping. – oeter Nov 26 '21 at 04:32

0 Answers0