Extract text from .html file, remove HTML, and write to text file using Python and Beautiful Soup

Question

I'm using Beautiful Soup 4 to extract text from HTML files, and using get_text() I can easily extract just the text, but now I'm attempting to write that text to a plain text file, and when I do, I get the message "416." Here's the code I'm using:

from bs4 import BeautifulSoup
markup = open("example1.html")
soup = BeautifulSoup(markup)
f = open("example.txt", "w")
f.write(soup.get_text())

And the output to the console is 416 but nothing gets written to the text file. Where have I gone wrong?

alternatively you can use, in 2.5+, the `with` statement to have that handled for you — mechanical_meat, Apr 26 '13 at 16:52
416 can be the returned value from `f.write()` (the number of bytes written). The writes are buffered by default; flush (application) buffers (`f.flush()`) or close the file (`f.close()` or use `with`-statement that does it for you) to be able to see something in the file outside the Python process. Note: it doesn't ensure that the data is actually saved (physically) to disk depending on your OS, filesystem, hdd it may take a while (usually it doesn't matter unless there is a power failure). `os.fsync()` might flush OS buffers ([usage example](http://stackoverflow.com/a/12012813/4279)). — jfs, Apr 26 '13 at 17:49

score 5 · Accepted Answer · answered Apr 26 '13 at 16:52

You need to send text to the BeautifulSoup class. Maybe try markup.read()

from bs4 import BeautifulSoup
markup = open("example1.html")
soup = BeautifulSoup(markup.read())
markup.close()
f = open("example.txt", "w")
f.write(soup.get_text())
f.close()

and in a more pythonic style

from bs4 import BeautifulSoup

with open("example1.html") as markup:
    soup = BeautifulSoup(markup.read())

with open("example.txt", "w") as f: 
    f.write(soup.get_text())

as @bernie suggested

Extract text from .html file, remove HTML, and write to text file using Python and Beautiful Soup

1 Answers1