Querying Elasticsearch with escaped quotes in python requests

Question

I'm trying to query elasticsearch using python requests. Following this post, I'm using the following process:

params = {
    "query": {
        "filtered": {
            "query": {
                "bool": {
                    "should": [
                        {
                            "query_string": {
                                "query": r'\"text in quotes\"'
                            }
                        }
                    ]
                }
            },
            "filter": {
                "bool": {
                    "must": [
                        {
                            "range": {
                                "@timestamp": {
                                    "from": 1458581389860,
                                    "to": 1458494989000
                                }
                            }
                        }
                    ]
                }
            }
        }
    },
    "size": 100,
}
response = requests.get(url, params=params)

Unfortunately, the quotation marks in the query don't appear to be properly escaped for elasticsearch. I've also tried:

'\\"text in quotes\\"'
response = requests.get(url, data=json.dumps(params))

The equivalent curl, which works, looks as follows:

curl -XGET 'MYURL/_search?pretty' -d '{
    "query": {
        "filtered": {
            "query": {
                "bool": {
                    "should": [
                        {
                            "query_string": {
                                "query": "\"test in quotes\""
                            }
                        }
                    ]
                }
            },
            "filter": {
                "bool": {
                    "must": [
                        {
                            "range": {
                                "@timestamp": {
                                    "from": 1458581389860,
                                    "to": 1458494989000
                                }
                            }
                        }
                    ]
                }
            }
        }
    },
    "size": 100,
}'

`'\\"text in quotes\\"'` is the same as `r'\"text in quotes\"'` — OneCricketeer, Mar 21 '16 at 18:21
That's what I thought. Could `requests` be doing something to the string after the value is passed? — Luke, Mar 21 '16 at 18:28
It might, but I don't know why it would. Have you tried to use the python elasticsearch library? — OneCricketeer, Mar 21 '16 at 18:30
Also, did you read the answer on the link you provided? The `params` parameter is not for data being sent. If you're trying to send data to the server you should specifically be using the `data` parameter — OneCricketeer, Mar 21 '16 at 18:34
I'm trying to send query parameters as outlined in the link, which is why I used the `params` parameter. I have also tried sending the query via a json dump of the above dict using the `data` parameter, but to no avail. — Luke, Mar 21 '16 at 18:42
I think the problem is the cURL is using `\"` to escape the quotes. In Python you only need to do `r'"text in quotes"'` — OneCricketeer, Mar 21 '16 at 18:52

score 1 · Answer 1 · answered Mar 21 '16 at 18:57

In cURL, you are escaping the quotes. "\"text in quotes\"", and this will become "text in quotes".

Your Python problem is that you don't need to escape anything if you use single quotes like you've done with r'\"text in quotes\"' that will print \"text in quotes\" because it is a raw string containing the slashes.

So, you have two options:

Use double quotes for the Python string and escape "\"text in quotes\""
Just use single quotes with un-escaped double quotes '"text in quotes"'

That's the first approach that I tried. Maybe the string isn't the problem. — Luke, Mar 21 '16 at 19:07

score 0 · Answer 2 · answered Mar 22 '16 at 14:28

0

If the string is not the problem, be careful of character encoding. Try to manage it and use UTF-8.

answered Mar 22 '16 at 14:28

Adrien Chaussende

341
2
6

score 0 · Accepted Answer · answered Mar 22 '16 at 19:26

Turns out that uri being used in the python example went to http whereas the uri for the curl example was https. It works with the following changes:

http -> https
the string as '"text in quotes"'
sent the query as data response = requests.get(url, data=json.dumps(params))

I don't understand why it was partially working before, 100 hits were being returned.

Querying Elasticsearch with escaped quotes in python requests

3 Answers3