Django REST - JSON gets returned with backslashes (serilized mutpiple times?)

Question

I have defined the following function in the backend for getting JSON responses

def create(self, request, *args, **kwargs):
    import requests
    import json
    url = request.POST.get('url')
    r = requests.get(url)
    return Response(json.loads(r.content.decode('utf-8')))

It works perfectly when used outside of a django view (if i remove all the function arguments and pass in a url)
This is the output when sending a sample request to the Twitter API

{
    "url": "https://twitter.com/elonmusk/status/1285830270709403648",
    "author_name": "Elon Musk",
    "author_url": "https://twitter.com/elonmusk",
    "html": "<blockquote class=\"twitter-tweet\"><p lang=\"en\" dir=\"ltr\">You don’t have a soul, you are a soul</p>&mdash; Elon Musk (@elonmusk) <a href=\"https://twitter.com/elonmusk/status/1285830270709403648?ref_src=twsrc%5Etfw\">July 22, 2020</a></blockquote>\n<script async src=\"https://platform.twitter.com/widgets.js\" charset=\"utf-8\"></script>\n",
    "width": 550,
    "height": null,
    "type": "rich",
    "cache_age": "3153600000",
    "provider_name": "Twitter",
    "provider_url": "https://twitter.com",
    "version": "1.0"
}

No only do the strings get wrapped in double quotes \ \ but also arguments like \n , blockquote are kept in the response

I know it has something to do with the response beeing serialized multiple times but I can't find exactly when this happens

Slightly Modifying the response to return the raw data returns this

return Response({
    "json_serialized" : json.loads(r.content.decode('utf-8')),
    "raw_content" : r.content.decode('utf-8')
})

Here is the output

{
    "json_serialized": {
        "url": "https://twitter.com/elonmusk/status/1285830270709403648",
        "author_name": "Elon Musk",
        "author_url": "https://twitter.com/elonmusk",
        "html": "<blockquote class=\"twitter-tweet\"><p lang=\"en\" dir=\"ltr\">You don’t have a soul, you are a soul</p>&mdash; Elon Musk (@elonmusk) <a href=\"https://twitter.com/elonmusk/status/1285830270709403648?ref_src=twsrc%5Etfw\">July 22, 2020</a></blockquote>\n<script async src=\"https://platform.twitter.com/widgets.js\" charset=\"utf-8\"></script>\n",
        "width": 550,
        "height": null,
        "type": "rich",
        "cache_age": "3153600000",
        "provider_name": "Twitter",
        "provider_url": "https://twitter.com",
        "version": "1.0"
    },
    "raw_content": "{\"url\":\"https:\\/\\/twitter.com\\/elonmusk\\/status\\/1285830270709403648\",\"author_name\":\"Elon Musk\",\"author_url\":\"https:\\/\\/twitter.com\\/elonmusk\",\"html\":\"\\u003Cblockquote class=\\\"twitter-tweet\\\"\\u003E\\u003Cp lang=\\\"en\\\" dir=\\\"ltr\\\"\\u003EYou don’t have a soul, you are a soul\\u003C\\/p\\u003E&mdash; Elon Musk (@elonmusk) \\u003Ca href=\\\"https:\\/\\/twitter.com\\/elonmusk\\/status\\/1285830270709403648?ref_src=twsrc%5Etfw\\\"\\u003EJuly 22, 2020\\u003C\\/a\\u003E\\u003C\\/blockquote\\u003E\\n\\u003Cscript async src=\\\"https:\\/\\/platform.twitter.com\\/widgets.js\\\" charset=\\\"utf-8\\\"\\u003E\\u003C\\/script\\u003E\\n\",\"width\":550,\"height\":null,\"type\":\"rich\",\"cache_age\":\"3153600000\",\"provider_name\":\"Twitter\",\"provider_url\":\"https:\\/\\/twitter.com\",\"version\":\"1.0\"}"
}

- The first one is the same of course,

This is not a problem with the Twitter API The same happens with any json response

Here is an example with the Facebook API

{
    "json_serialized": {
        "author_name": "Unboxholics",
        "author_url": "https://www.facebook.com/Unboxholics/",
        "provider_url": "https://www.facebook.com",
        "provider_name": "Facebook",
        "success": true,
        "height": null,
        "html": "<div id=\"fb-root\"></div>\n<script async=\"1\" defer=\"1\" crossorigin=\"anonymous\" src=\"https://connect.facebook.net/el_GR/sdk.js#xfbml=1&amp;version=v7.0\" nonce=\"OAmlb4oU\"></script><div class=\"fb-post\" data-href=\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\" data-width=\"552\"><blockquote cite=\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\" class=\"fb-xfbml-parse-ignore\"><p>8 ΧΡΟΝΙΑ UNBOXHOLICS\nΣήμερα γιορτάζουν τα κωλόμπουρα! Σήμερα γιορτάζουμε μαζί αυτήν την ημέρα! Σας ευχαριστούμε από...</p>Δημοσιεύτηκε από <a href=\"https://www.facebook.com/Unboxholics/\">Unboxholics</a> στις&nbsp;<a href=\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\">Τετάρτη, 22 Ιουλίου 2020</a></blockquote></div>",
        "type": "rich",
        "version": "1.0",
        "url": "https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3",
        "width": 552
    },
    "raw_content": "{\"author_name\":\"Unboxholics\",\"author_url\":\"https://www.facebook.com/Unboxholics/\",\"provider_url\":\"https://www.facebook.com\",\"provider_name\":\"Facebook\",\"success\":true,\"height\":null,\"html\":\"\\u003Cdiv id=\\\"fb-root\\\">\\u003C/div>\\n\\u003Cscript async=\\\"1\\\" defer=\\\"1\\\" crossorigin=\\\"anonymous\\\" src=\\\"https://connect.facebook.net/el_GR/sdk.js#xfbml=1&amp;version=v7.0\\\" nonce=\\\"OAmlb4oU\\\">\\u003C/script>\\u003Cdiv class=\\\"fb-post\\\" data-href=\\\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\\\" data-width=\\\"552\\\">\\u003Cblockquote cite=\\\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\\\" class=\\\"fb-xfbml-parse-ignore\\\">\\u003Cp>\\ud83e\\udd188 \\u03a7\\u03a1\\u039f\\u039d\\u0399\\u0391 UNBOXHOLICS\\ud83c\\udf82\\n\\u03a3\\u03ae\\u03bc\\u03b5\\u03c1\\u03b1 \\u03b3\\u03b9\\u03bf\\u03c1\\u03c4\\u03ac\\u03b6\\u03bf\\u03c5\\u03bd \\u03c4\\u03b1 \\u03ba\\u03c9\\u03bb\\u03cc\\u03bc\\u03c0\\u03bf\\u03c5\\u03c1\\u03b1! \\u03a3\\u03ae\\u03bc\\u03b5\\u03c1\\u03b1 \\u03b3\\u03b9\\u03bf\\u03c1\\u03c4\\u03ac\\u03b6\\u03bf\\u03c5\\u03bc\\u03b5 \\u03bc\\u03b1\\u03b6\\u03af \\u03b1\\u03c5\\u03c4\\u03ae\\u03bd \\u03c4\\u03b7\\u03bd \\u03b7\\u03bc\\u03ad\\u03c1\\u03b1! \\u03a3\\u03b1\\u03c2 \\u03b5\\u03c5\\u03c7\\u03b1\\u03c1\\u03b9\\u03c3\\u03c4\\u03bf\\u03cd\\u03bc\\u03b5 \\u03b1\\u03c0\\u03cc...\\u003C/p>\\u0394\\u03b7\\u03bc\\u03bf\\u03c3\\u03b9\\u03b5\\u03cd\\u03c4\\u03b7\\u03ba\\u03b5 \\u03b1\\u03c0\\u03cc \\u003Ca href=\\\"https://www.facebook.com/Unboxholics/\\\">Unboxholics\\u003C/a> \\u03c3\\u03c4\\u03b9\\u03c2&nbsp;\\u003Ca href=\\\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\\\">\\u03a4\\u03b5\\u03c4\\u03ac\\u03c1\\u03c4\\u03b7, 22 \\u0399\\u03bf\\u03c5\\u03bb\\u03af\\u03bf\\u03c5 2020\\u003C/a>\\u003C/blockquote>\\u003C/div>\",\"type\":\"rich\",\"version\":\"1.0\",\"url\":\"https://www.facebook.com/Unboxholics/photos/a.647580348625155/3017945248255308/?type=3\",\"width\":552}"
}

score 0 · Answer 1 · answered Jul 23 '20 at 17:20

It's not an issue of DRF. It's a very common issue when deserializing JSON data. The simplest way is to make your own deserializing method using json.loads method.

For example,

def deserialize(data: str) -> Any:
    return json.loads(data).replace('\\"', '\"').replace('\\n', '\n')

There are many articles and answers about this. See Python: json.loads chokes on escapes for more detailed answer.

Django REST - JSON gets returned with backslashes (serilized mutpiple times?)

1 Answers1