2

I get a UnicodeEncodeError when using pdfminer (the latest version from git) installed via pip install git+https://github.com/pdfminer/pdfminer.six.git:

Traceback (most recent call last):
  File "pdfminer_sample3.py", line 34, in <module>
    print(convert_pdf_to_txt("samples/numbers-test-document.pdf"))
  File "pdfminer_sample3.py", line 27, in convert_pdf_to_txt
    text = retstr.getvalue()
  File "/usr/lib/python2.7/StringIO.py", line 271, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

How can I fix that?

Script

#!/usr/bin/env python

from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage
from StringIO import StringIO
import codecs

def convert_pdf_to_txt(path):
    rsrcmgr = PDFResourceManager()
    retstr = StringIO()
    codec = 'utf-8'
    laparams = LAParams()
    device = TextConverter(rsrcmgr, retstr, codec=codec, laparams=laparams)

    fp = file(path, 'rb')
    interpreter = PDFPageInterpreter(rsrcmgr, device)
    password = ""
    maxpages = 0
    caching = True
    pagenos = set()

    for page in PDFPage.get_pages(fp, pagenos,
                                  maxpages=maxpages,
                                  password=password,
                                  caching=caching,
                                  check_extractable=True):
        interpreter.process_page(page)

    text = retstr.getvalue()

    fp.close()
    device.close()
    retstr.close()
    return text

print(convert_pdf_to_txt("samples/numbers-test-document.pdf"))

Example pdf

https://www.dropbox.com/s/khjfr63o82fa5yn/numbers-test-document.pdf?dl=0

Himanshu
  • 2,384
  • 2
  • 24
  • 42
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
  • You can ignore the errors characters while reading file. [check this](https://stackoverflow.com/questions/12468179/unicodedecodeerror-utf8-codec-cant-decode-byte-0x9c) – pooya Jul 14 '17 at 11:58

1 Answers1

5

Replace from StringIO import StringIO by from io import BytesIO

and

replace retstr = StringIO() by retstr = BytesIO()

Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
  • this really depends wether the OP is using python3 or 2...hopefully 3, as you suggest with your answer ;-) – benzkji Jun 20 '19 at 08:41