0

I want to delete all divs without classes (but not the content that is in the div).

My input

<h1>Test</h1>
<div>
    <div>
        <div class="test">
            <p>abc</p>
        </div>
    </div>
</div>

The output I want

<h1>Test</h1>
<div class="test">    
    <p>abc</p>
</div>

My try 1

Based on "Deleting a div with a particular class":

from bs4 import BeautifulSoup
soup = BeautifulSoup('<h1>Test</h1><div><div><div class="test"><p>abc</p></div></div></div>', 'html.parser')   
for div in soup.find_all("div", {'class':''}): 
    div.decompose()
print(soup)
# <h1>Test</h1>

My try 2

from htmllaundry import sanitize
myinput = '<h1>Test</h1><div><div><div class="test"><p>abc</p></div></div></div>'
myoutput = sanitize(myinput)
print myoutput
# <p>Test</p><p>abc</p> instead of <h1>Test</h1><div class="test"><p>abc</p></div>

My try 3

Based on "Clean up HTML in python"

from lxml.html.clean import Cleaner

def sanitize(dirty_html):
    cleaner = Cleaner(remove_tags=('font', 'div'))

    return cleaner.clean_html(dirty_html)


myhtml = '<h1>Test</h1><div><div><div class="test"><p>abc</p></div></div></div>'

print(sanitize(myhtml))
# <div><h1>Test</h1><p>abc</p></div>

My try 4

from html_sanitizer import Sanitizer
sanitizer = Sanitizer()  # default configuration
output = sanitizer.sanitize('<h1>Test</h1><div><div><div class="test"><p>abc</p></div></div></div>')
print(output)
# <h1>Test</h1><p>abc</p>

Problem: A div element is used to wrap the HTML fragment for the parser, therefore div tags are not allowed. (Source: Manual)

Sr. Schneider
  • 647
  • 10
  • 20

2 Answers2

1

If you want to exclude div without class, preserving its content:

from bs4 import BeautifulSoup
markup = '<h1>Test</h1><div><div><div class="test"><p>abc</p></div></div></div>'
soup = BeautifulSoup(markup,"html.parser")
for tag in soup.find_all():
    empty = tag.name == 'div' and not(tag.has_attr('class'))
    if not(empty):
        print(tag)

Output:

<h1>Test</h1>
<div class="test"><p>abc</p></div>
<p>abc</p>
0

Please checkout this.

from bs4 import BeautifulSoup

data="""
<div>
    <div>
        <div class="test">
            <p>abc</p>
        </div>
    </div>
</div>
"""
soup = BeautifulSoup(data, features="html5lib")

for div in soup.find_all("div", class_=True):
    print(div)

krishna
  • 1,029
  • 6
  • 12