How to use the Amazon Textract with PDF files

Question

I already can use the textract but with JPEG files. I would like to use it with PDF files.

I have the code bellow:

import boto3

# Document
documentName = "Path to document in JPEG"

# Read document content
with open(documentName, 'rb') as document:
    imageBytes = bytearray(document.read())

# Amazon Textract client
textract = boto3.client('textract')
documentText = ""

# Call Amazon Textract
response = textract.detect_document_text(Document={'Bytes': imageBytes})

#print(response)

# Print detected text
for item in response["Blocks"]:
    if item["BlockType"] == "LINE":
        documentText = documentText + item["Text"]

        # print('\033[94m' +  item["Text"] + '\033[0m')
        # # print(item["Text"])

# removing the quotation marks from the string, otherwise would cause problems to A.I
documentText = documentText.replace(chr(34), '')
documentText = documentText.replace(chr(39), '')
print(documentText)

As I said, it works fine. But I would like to use it passing a PDF file as in the web application for tests.

I know it possible to convert the PDF to JPEG in python but it would be nice to do it with PDF. I read the documentation and do not find the answer.

How can I do that?

EDIT 1: I forgot to mention that I do not intend to use de s3 bucket. I want to pass the PDF right in the script, without having to upload it into s3 bucket.

score 7 · Answer 1 · edited Nov 04 '21 at 22:12

As @syumaK mentioned, you need to upload the pdf to S3 first. However, doing this may be cheaper and easier than you think:

Create new S3 bucket in console and write down bucket name, then

import random
import boto3

bucket = 'YOUR_BUCKETNAME'
path = 'THE_PATH_FROM_WHERE_YOU_UPLOAD_INTO_S3'
filename = 'YOUR_FILENAME'

s3 = boto3.resource('s3')
print(f'uploading {filename} to s3')
s3.Bucket(bucket).upload_file(path+filename, filename)

client = boto3.client('textract')
response = client.start_document_text_detection(
                   DocumentLocation={'S3Object': {'Bucket': bucket, 'Name': filename} },
                   ClientRequestToken=random.randint(1,1e10))

jobid = response['JobId']
response = client.get_document_text_detection(JobId=jobid)

It may take 5-50 seconds, until the call to get_document_text_detection(...) returns a result. Before, it will say that it is still processing.

According to my understanding, for each token, exactly one paid API call will be performed - and a past one will be retrieved, if the token has appeared in the past.

Edit: I forgot to mention, that there is one intricacy if the document is large, in which case the result may need to be stitched together from multiple 'pages'. The kind of code you will need to add is


...
pages = [response]
while nextToken := response.get('NextToken'):
    response = client.get_document_text_detection(JobId=jobid, NextToken=nextToken)
    pages.append(response)

Thank you so much for the edit. I did not know about NextToken thing and never came across it while implementing... this is what happends when you don't read the entire documentation :'( I am googling this thing for past couple of days, as to why is textract not scanning my entire document when am using boto3 :3 — Malgo, Apr 27 '22 at 05:49

score 2 · Answer 2 · answered Nov 25 '19 at 19:01

2

As mentioned in the AWS Textract FAQ page https://aws.amazon.com/textract/faqs/. pdf files are supported and in Sdk as well https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/textract.html

Sample usage https://github.com/aws-samples/amazon-textract-code-samples/blob/master/python/12-pdf-text.py

answered Nov 25 '19 at 19:01

sash

1,124
2
15
32

I forgot to mention that I do not intend to use de s3 bucket. I want to pass the PDF right in the script, without having to upload it into s3 bucket. In that script you sended to me, I would have to use s3 bucket. Right? – ArthurS Nov 25 '19 at 19:05

score 1 · Answer 3 · answered Nov 28 '19 at 18:58

Since you want to work with PDF files meaning that you'll utilize Amazon Textract Asynchronous API (StartDocumentAnalysis, StartDocumentTextDetection) then currently it's not possible to directly parse in PDF files. This is because Amazon Textract Asynchronous APIs only support document location as S3 objects.

From AWS Textract doc:

Amazon Textract currently supports PNG, JPEG, and PDF formats. For synchronous APIs, you can submit images either as an S3 object or as a byte array. For asynchronous APIs, you can submit S3 objects.

score 0 · Answer 4 · answered Oct 17 '21 at 17:17

0

Upload the pdf to S3 bucket. After that, you can use easily use available functions startDocumentAnalysis to fetch pdf directly from s3 and do textract.

answered Oct 17 '21 at 17:17

Ashok Kumar Dhulipalla

199
3
10

score 0 · Answer 5 · answered Jun 28 '22 at 19:18

0

It works (almost), I had to make ClientRequestToken a string instead of an integer.

answered Jun 28 '22 at 19:18

Sergio Lucero

862
1
12
21

score 0 · Answer 6 · answered May 22 '23 at 07:20

You can textract pdf file page by page without using s3 bucket

For this task you need to cut the page from the pdf

I used ghostscript command example:

gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=jpeg -dJPEGQ=100 -dPDFFitPage -dDEVICEWIDTHPOINTS=3000 -dDEVICEHEIGHTPOINTS=3000 -dFirstPage=1 -dLastPage=1 -sOutputFile=[outputFilePath] [inputFileName]

this is the sample code for golang:

// GenerateImageByPDF gets a pdf file and a page number as intput, then generate an image file of that page

func GenerateImageByPDF(inputFileName string, outputFileName string, pageNumber int) error {
    dFirstPageFlag := fmt.Sprintf("-dFirstPage=%d", pageNumber)
        dLastPageFlag := fmt.Sprintf("-dLastPage=%d", pageNumber)
    outputFileFlag := fmt.Sprintf("-sOutputFile=%s", outputFileName)

    cmd := exec.Command("gs", "-dSAFER", "-dBATCH", "-dNOPAUSE", "-sDEVICE=jpeg", "-dJPEGQ=100", "-dPDFFitPage","-dDEVICEWIDTHPOINTS=3000", "-dDEVICEHEIGHTPOINTS=3000", dFirstPageFlag, dLastPageFlag, outputFileFlag, inputFileName)

    output, err := cmd.Output()
    if err != nil {
        return err
    }

    if strings.Contains(string(output), "Requested FirstPage is greater than the number of pages in the file") {
        return fmt.Errorf("requested page is greater than the number of pages in file")
    }
    // TODO (later development) : if the otuput size is more than 10MB we're gonna have problem with amazon textract service so you can use -dDownScaleFactor=2 to shorten the fize size and do it until it's less than 10MB
    return nil
}

How to use the Amazon Textract with PDF files

6 Answers6

Linked