Get json payload then strip the html with python

Question

I have a json values that I need stripped of all html tags.

After using the following function:

def payloaded():
from urllib.request import urlopen
with urlopen("www.example.com/payload.json") as r:
    data = json.loads(r.read().decode(r.headers.get_content_charset("utf-8")))
text = (data["body"]["und"][0]["value"])
return(text)

This is the returned (text):

&lt;div class=&quot;blah&quot;&gt;'<p>This is the text.</p>\r\n'

This is the original (text):

<div class="blah"><p>This is the text.</p>

Note: I need all html tags stripped, and there is no real guidelines of what the tags I will be getting.

This is what I want the (text) to be:

This is the text.

This is the post function I am using:

def add_node_basic(text)
url = "www.example.com"
headers = {"content-type": "application/json"}
payload = {
    "auth_token": x,
    "docs":
        {
            "id": y,
            "fields": [
                {"name": "body", "value": text, "type": "text"},
            ]}
}

r = requests.post(url, data=json.dumps(payload), headers=headers)

Any suggestions on how to achieve this is much appreciated!

Answer 1

You can try slicing the string along with find method, like this:

>>> print text[text.find('<p>'):text.find('</p>')].strip('<p>')
This is the text.

If you are trying to extract text only from the HTML source, then you can use HTMLParser library in Python. Example:

from HTMLParser import HTMLParser

class MLStripper(HTMLParser):
    def __init__(self):
        self.reset()
        self.fed = []
    def handle_data(self, d):
        self.fed.append(d)
    def get_data(self):
        return ''.join(self.fed)

def strip_tags(html):
    s = MLStripper()
    s.feed(html)
    return s.get_data()