read the text file in azure storage blob line by line using python

Question

I need to read text files from blob storage line by line and perform some operations and get a specific line to data frame. I have tried various ways to read file line by line. Is there any way to read a text file from blob line-line and perform operations and output specific line just like readlines() while data is in local storage?

candidate_resume = 'candidateresumetext'
block_blob_service = BlockBlobService(account_name='nam', account_key='key')
generator2 = block_blob_service.list_blobs(candidate_resume)
#for blob in generator2:
   #print(blob.name)
for blob in generator2:
    blob2 = block_blob_service.get_blob_to_text(candidate_resume,blob.name)
    #print(blob2)

    #blob_url=block_blob_service.make_blob_url(candidate_resume, blob.name)
    #print(blob_url)

    #blob3 = block_blob_service.get_blob_to_stream(candidate_resume,blob.name,range)
    blob3 = blob2.split('.')
    with open(blob.name,encoding = 'utf-8') as file:
        lines = file.readlines()
        for line in blob3:      
            if any(p in years_list for p in line ):
                if any(p in months_list for p in line):    
                    print(line)

score 2 · Accepted Answer · answered May 06 '19 at 02:18

2

The method get_blob_to_text is the right way, and you can follow the sample code below(you can make some changes if it does not meet your need). And you cannot use with open() as file since no real file is there.

#read the content of the blob(assume it's a .txt file)
str1 = block_blob_service.get_blob_to_text(container_name,blob_name)

#split the string str1 with newline.
arr1 = str1.content.splitlines()

#read the one line each time.
for a1 in arr1:
    print(a1)

answered May 06 '19 at 02:18

Ivan Glasenberg

29,865
2
44
60

Hello, if the answer works, please help mark it as an answer. Thanks. – Ivan Glasenberg May 07 '19 at 06:12
yea it worked for me . there are like 10000 text files in blob. can we get the file name assigned to each line i retreived ? – Nagasai Chowdary May 07 '19 at 07:08
do you mean that there are some files like a.txt,b.txt, and when you get each line from a.txt or b.txt, add the file name(a.txt or b.txt) to each line? – Ivan Glasenberg May 07 '19 at 07:15
If it's the case I mentioned, you can just directly add the blob name in the print(), like print(blob_name+":"+a1) – Ivan Glasenberg May 07 '19 at 07:22

read the text file in azure storage blob line by line using python

1 Answers1

Linked

Related