Why is requests is adding a part of a list to repeated query URIs?

Question

I am trying to place a variable into a POST request using the Requests library. Here is my code:

import requests

message = "if i can\'t let it go out of my mind"

split_message = message.split()

initial_request = requests.get('http://ws.spotify.com/search/1/track?q='.join(split_message[:3]))
print initial_request.content

The outcome I am getting is this (which is an error, as it is messing up the URI):

No connection adapters were found for 'ifhttp://ws.spotify.com/search/1/track?q=ihttp://ws.spotify.com/search/1/track?q=can't'

I would like the request URI to look like this:

"http://ws.spotify.com/search/1/track?q=if i can\'t"

What am I missing here? Is there a better way to pass a variable to the request URI? I tried using a dictionary as the payload, but I can't place a variable into a dictionary.

score 3 · Accepted Answer · 2014-01-09T04:06:47.257

3

String.join is very counter intuitive, the string is something you join with not join to.

Examine the following code:

>>> x = range(5)
>>> x
[0, 1, 2, 3, 4]
>>> x = [str(c) for c in x]
>>> x
['0', '1', '2', '3', '4']
>>> "-".join(x)
'0-1-2-3-4'

Here the "joiner" (-) is inserted between every element in our array, which we've converted to strings using list comprehension.

Here is your code:

'http://ws.spotify.com/search/1/track?q='.join(split_message[:3])

Your joining string is 'http://ws.spotify.com/search/1/track?q=' and so this is inserted between every element in the array split_message[:3].

So lets examine whats going on:

>>> message = "if i can\'t let it go out of my mind"
>>> split_message = message.split()
>>> split_message
['if', 'i', "can't", 'let', 'it', 'go', 'out', 'of', 'my', 'mind']
>>> split_message[:3]
['if', 'i', "can't"]

Here the array is 3 items long, which explains why the output string is:

 ifhttp://ws.spotify.com/search/1/track?q=ihttp://ws.spotify.com/search/1/track?q=can't

But adding in some line breaks:

 if
   http://ws.spotify.com/search/1/track?q=
 i
   http://ws.spotify.com/search/1/track?q=
 can't'

Notice the joining string is inserted twice, which explains why at first glance it just looks like its mucking up the URI.

What you want instead is:

request = 'http://ws.spotify.com/search/1/track?q='+"%20".join(split_message[:3])

Notice, in the above we join using %20: like so: "%20".join(split_message[:3]) and add this the request prefix. Which gives the URL below, with the spaces correctly encoded:

"http://ws.spotify.com/search/1/track?q=if%20i%20can't"

edited Jan 09 '14 at 04:06

answered Jan 09 '14 at 04:00

thanks for the explanation on why this is happening. – metersk Jan 09 '14 at 04:02
@Barnaby If its helpful accepting an answer by clicking the green tick lets other users with similar problems know a particular answer help you solve your particular issue. – Jan 09 '14 at 04:08
yes, i plan on doing so. is the %20 a URL code for a space or +? – metersk Jan 09 '14 at 04:10
@Barnaby Both the `+` and `%20` are legal tokens for escaping a single space ina URI *in the query portion only*. [Generally `%20` is safer](http://stackoverflow.com/questions/2678551/when-to-encode-space-to-plus-or-20). – Jan 09 '14 at 04:13
Please note that you should let Requests encode the query string for you, as per @user2357112's answer. – Lukasa Jan 09 '14 at 09:30
@Lukasa I've not used the requests library before, you're probably right. I was just answering what I understood the question to be: "why isn't join working like I think it should" – Jan 09 '14 at 10:15
@LegoStormtroopr Sure. =) That comment was intended more for Barnaby than for you. – Lukasa Jan 09 '14 at 14:46

user2357112 · Answer 2 · 2014-01-09T04:06:44.850

0

initial_request = requests.get('http://ws.spotify.com/search/1/track?q=' +
                               ' '.join(split_message[:3]))

s.join(seq) sticks together the elements of seq with the string s stuck between them. For example, 'a'.join('bcd') == 'bacad'. Note that your desired string probably isn't what you really should be sending; you should probably use + instead of spaces. The requests module can handle url parameter encoding for you:

params = {'q': ' '.join(split_message[:3])}
initial_request = requests.get('http://ws.spotify.com/search/1/track',
                               params=params)

edited Jan 09 '14 at 04:06

answered Jan 09 '14 at 04:01

user2357112

260,549
28
431
505

`+` only works for encoding spaces in the query part, not the rest of the URI. `%20` is generally safer. – Jan 09 '14 at 04:15

Why is requests is adding a part of a list to repeated query URIs?

2 Answers2