Export all table data from PDF to Excel using Amazon textract

Question

Looking out to extract PDF data to Excel/CSV using Amazon Textract. How we can Insert the Input PDF data from the local folder.

Having PDF with multiple Tables, we need to extract all the tables from their respective pages and export the data to CSV/Excel files. which can be used for further analysis.

Piece of code received from AWS but could not understand how input pdf file can be taken up into the script.

import webbrowser, os
import json
import boto3
import io
from io import BytesIO
import sys
from pprint import pprint


def get_rows_columns_map(table_result, blocks_map):
    rows = {}
    for relationship in table_result['Relationships']:
        if relationship['Type'] == 'CHILD':
            for child_id in relationship['Ids']:
                cell = blocks_map[child_id]
                if cell['BlockType'] == 'CELL':
                    row_index = cell['RowIndex']
                    col_index = cell['ColumnIndex']
                    if row_index not in rows:
                        # create new row
                        rows[row_index] = {}
                        
                    # get the text value
                    rows[row_index][col_index] = get_text(cell, blocks_map)
    return rows


def get_text(result, blocks_map):
    text = ''
    if 'Relationships' in result:
        for relationship in result['Relationships']:
            if relationship['Type'] == 'CHILD':
                for child_id in relationship['Ids']:
                    word = blocks_map[child_id]
                    if word['BlockType'] == 'WORD':
                        text += word['Text'] + ' '
                    if word['BlockType'] == 'SELECTION_ELEMENT':
                        if word['SelectionStatus'] =='SELECTED':
                            text +=  'X '    
    return text


def get_table_csv_results(file_name):

    with open(file_name, 'rb') as file:
        img_test = file.read()
        bytes_test = bytearray(img_test)
        print('Image loaded', file_name)

    # process using image bytes
    # get the results
    client = boto3.client('textract')

    response = client.analyze_document(Document={'Bytes': bytes_test}, FeatureTypes=['TABLES'])

    # Get the text blocks
    blocks=response['Blocks']
    pprint(blocks)

    blocks_map = {}
    table_blocks = []
    for block in blocks:
        blocks_map[block['Id']] = block
        if block['BlockType'] == "TABLE":
            table_blocks.append(block)

    if len(table_blocks) <= 0:
        return "<b> NO Table FOUND </b>"

    csv = ''
    for index, table in enumerate(table_blocks):
        csv += generate_table_csv(table, blocks_map, index +1)
        csv += '\n\n'

    return csv

def generate_table_csv(table_result, blocks_map, table_index):
    rows = get_rows_columns_map(table_result, blocks_map)

    table_id = 'Table_' + str(table_index)
    
    # get cells.
    csv = 'Table: {0}\n\n'.format(table_id)

    for row_index, cols in rows.items():
        
        for col_index, text in cols.items():
            csv += '{}'.format(text) + ","
        csv += '\n'
        
    csv += '\n\n\n'
    return csv

def main(file_name):
    table_csv = get_table_csv_results(file_name)

    output_file = 'output.csv'

    # replace content
    with open(output_file, "wt") as fout:
        fout.write(table_csv)

    # show the results
    print('CSV OUTPUT FILE: ', output_file)


if __name__ == "__main__":
    file_name = sys.argv[1]
    main(file_name)

Sample PDF file Click Here

Not clear what your question is. The provided script accepts a filename on the command line, passes that to `get_table_csv_results`, which loads the file content into a buffer, sends that buffer to the Amazon Textract `analyze_document` API, then exports the extracted text to an output CSV files.. — jarmod, Mar 17 '21 at 19:21
@jarmod - How and in which piece of code the above script will accept the input file to process, that's what my question is. — Manz, Mar 18 '21 at 02:44
If you don’t know elementary Python or the boto3 SDK then this code isn’t going to make much sense to you. You might want to consider using the AWS console or the awscli if that’s the case. — jarmod, Mar 18 '21 at 02:48

score 0 · Answer 1 · answered Jul 06 '21 at 04:44

0

first you must generate the necessary environments in aws, install awscli and configure it with your aws credentials, having that, you only need to install the corresponding libraries and change the last line of the code:

if __name__ == "__main__": file_name = "name_image.png" main(file_name)

I recommend you to read this publication, to set up your aws environment:

https://medium.com/@victorjatoba10/extract-tables-and-forms-from-pdf-using-amazon-aws-textract-827c6e866453

answered Jul 06 '21 at 04:44

Diego Pavez Olave

1

1

Please describe the solution within you answer and not just link to a blog post. Online resources tend to go missing making your answer useless. – Aleksander Wons Jul 06 '21 at 07:48

score 0 · Answer 2 · answered May 04 '22 at 14:14

You can read the file yourself and pass the Bytes to Textract

import os

for filename in os.listdir('input'):
  if filename.endswith("jpg"): 
    with open('input/'+filename, 'rb') as img_file:
      img_bytes = img_file.read()
      response = client_Textract.analyze_document(Document={'Bytes': img_bytes}, FeatureTypes=["TABLES"])

Export all table data from PDF to Excel using Amazon textract

2 Answers2

Linked