1

I'm writing a Python script that gets an HTTP request as its input (namely, a url and some GET parameters). The response to the HTTP request is a piece of Python code representing a big Python dictionary. My script should evaluate the retrieved code and process the evaluated dictionary.

Had I been using Python 2.X, I would have done the follwoing:

import urllib
d = eval(urllib.request.urlopen(input_url).read())
process(d)

The problem is that I'm using Python 3.4, and the read method returns a bytes and not a string as in Python 2.X, so I can't use the eval function on it.

How do I evaluate the retrieved dictionary on Python 3.X?

snakile
  • 52,936
  • 62
  • 169
  • 241
  • possible duplicate of [Convert bytes to a Python string](http://stackoverflow.com/questions/606191/convert-bytes-to-a-python-string) – Aran-Fey Jan 20 '15 at 14:31
  • 1
    `eval` can be dangerous. You might prefer [`ast.literal_eval`](https://docs.python.org/2/library/ast.html#ast.literal_eval) instead. – jme Jan 20 '15 at 14:37
  • Does `json` format work in your case? If not; why? – jfs Jan 20 '15 at 19:51

2 Answers2

3

As @rawing said in the comments, you'll need to decode the bytes into a string. You'll also probably want to use ast.literal_eval over eval, as the latter executes arbitrary code, and is unsafe.

Here's an example where I've uploaded a dict of literals as a Github gist:

import urllib.request
import ast

url = ("https://gist.githubusercontent.com/eldridgejm/76c78b7d11a66162687b/" +
      "raw/60a76770970715f859d1e3d33c8e2afcac296a31/gistfile1.txt")

r = urllib.request.urlopen(url).read()
d = ast.literal_eval(r.decode())

print(d)

Running the above prints:

{'bar': 42, 'baz': 'apple', 'foo': 41}
Cristian Ciupitu
  • 20,270
  • 7
  • 50
  • 76
jme
  • 19,895
  • 6
  • 41
  • 39
  • Don't use `ast.literal_eval()` for json text. You could [get the character encoding from http headers](http://stackoverflow.com/q/14592762/4279). Or if the input is Python code then you could use `tokenizer.detect_encoding` module that recognize encoding declaration in Python source code. [Full example](https://gist.github.com/zed/dbdccd6d996b974e20cd) – jfs Jan 20 '15 at 20:20
  • @J.F.Sebastian Interesting, thanks. This answer assumes that the text is a `dict` of python literals. The only difference I am aware of in the case of literals is that `{"foo": (1,2,3)}` is not valid json. Do you know of others? – jme Jan 20 '15 at 20:58
  • Python literals and json text are different formats. You shouldn't pass Python dict as a literal if json is enough. – jfs Jan 20 '15 at 21:07
  • 1
    @J.F.Sebastian Oh, I totally agree. The OP says that what he has is Python, and not json. Maybe his data *is* valid json, in which case he could use the json parser, but if he were to ask "What can I represent in Python, but not in json?", tuples are the only thing that I know (if we're restricted to literals). – jme Jan 20 '15 at 21:39
1

Using jme's example and if it is valid json which I think it may well be you can just use requests and use .json:

import requests

r  = requests.get("https://gist.githubusercontent.com/eldridgejm/76c78b7d11a66162687b/raw/60a76770970715f859d1e3d33c8e2afcac296a31/gistfile1.txt")
print(r.json())
{u'baz': u'apple', u'foo': 41, u'bar': 42}
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321