Python Requests - downloading a .py file from GitHub

Question

I'm trying to use Requests to load a .py file from a GitHub directory. The .py file starts like this (I guess it's a dictionary?):

colors = {

    "1234": {
        "shade": [1, 2, 3]
        }
    }

I use the following code:

import requests
r = requests.get('https://raw.github.com/example/example1/master/scripts/colors.py')

I would like r to be a variable or a dictionary, so that I can easily look for some specific strings inside. However, when I print r I don't get a .py file, but there are also signs like \r\n etc. How can I read this file properly?

r.text

>>> 'colors = {\r\n\r\n    "1234": {\r\n        "shade": [1, 2, 3]}\r\n}\r\n'

At the end I would like to create a dictionary, where for "shade" I obtain all values. So first I would need to search r for the word "shade".

I will be thankful for any suggestions and solutions.

"I would like r to be a variable" - It *is* a variable. "However, when I print r I don't get a .py file" - because it's a string, not a file. — mkrieger1, Aug 01 '21 at 17:52
`ast.literal_eval(r.content)`? You'd have to hope that the repo's owner doesnt insert malicious code into that file — Sayse, Aug 01 '21 at 17:52
Does this answer your question? [Convert a String representation of a Dictionary to a dictionary?](https://stackoverflow.com/questions/988228/convert-a-string-representation-of-a-dictionary-to-a-dictionary) — mkrieger1, Aug 01 '21 at 17:54
Unfortunately it gives an error: >>> ast.literal_eval(r.content) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.9/ast.py", line 105, in literal_eval return _convert(node_or_string) File "/usr/lib/python3.9/ast.py", line 104, in _convert return _convert_signed_num(node) File "/usr/lib/python3.9/ast.py", line 78, in _convert_signed_num return _convert_num(node) File "/usr/lib/python3.9/ast.py", line 69, in _convert_num _raise_malformed_node(node) — enten77, Aug 01 '21 at 17:59
File "/usr/lib/python3.9/ast.py", line 66, in _raise_malformed_node raise ValueError(f'malformed node or string: {node!r}') ValueError: malformed node or string: b'colors = {\r\n\r\n "1234": {\r\n "shade": [1, 2, 3], — enten77, Aug 01 '21 at 18:01

Mark Tolonen · Accepted Answer · 2021-08-01T19:11:55.133

2

Since it is Python code, not just a string literal, use exec, but be aware if the code is malicious it will be executed:

text = 'colors = {\r\n\r\n    "1234": {\r\n        "shade": [1, 2, 3]}\r\n}\r\n'
exec(text)
print(colors['1234']['shade'])

Output:

[1, 2, 3]

edited Aug 01 '21 at 19:11

answered Aug 01 '21 at 18:14

Mark Tolonen

166,664
26
169
251

Traceback (most recent call last): File "", line 1, in TypeError: string indices must be integers – enten77 Aug 01 '21 at 18:31
@enten77 You ran the *exact* code above? It works. – Mark Tolonen Aug 01 '21 at 18:33
Yes, it works very well for "text = 'colors = ...", but when I try to apply it to my data (r.text) it doesn't work, unfortunately. No idea why is it so. – enten77 Aug 01 '21 at 18:59
@enten77 You would have to show the *exact* content of `r.text`. If it is a string like `text` is above, and is *valid* Python code, it should work. In fact it works with the `r.text` you display in your question (once I add the missing final `'` anyway). – Mark Tolonen Aug 01 '21 at 19:01
@enten77 The error indicates the line of code with the error is using a string to index a variable, but the variable is a list and requires integers `['123']` vs. `[123]` for example. If you show the code and the full error traceback, you can get help, perhaps in a new question. The answer to *this* question is "use `exec` when the string is Python code". :) – Mark Tolonen Aug 01 '21 at 19:17

Python Requests - downloading a .py file from GitHub

1 Answers1