Running blob storage trigger over existing files multiple times

Question

I've been using a method to run my storage trigger over the same files multiple times by going into my storage account and deleting the blobs that are found in 'azure-webjobs-host' > 'blobreceipts' > '[Function App Name]' > '[Blob Trigger Name]'

I thought that this was all I needed to worry about to be able to run my function over the same files again, however, I must be missing something because this seems very inconsistent.

From what I can tell, whenever the trigger is ran over a file, another file should be created inside of the blobreceipts folder pertaining to the function that dealt with it. My guess was that this newly created file inside of 'blobreceipts' held all the data that kept the function from running over that same file again (like the file's E-Tag? etc.). So, my thinking was, if I just delete whatever is found inside of that folder I should be good to run my function again. However, sometimes after my function has run over thousands of files I'll go and check inside of 'blobreceipts' and find that there is only one file/blob in there. Or alternatively, there will be a lot of files that I'll have to delete from there, but even after doing so, it seems my function doesn't run over every single file again. Overall, the method seems to work, but only to an extent.

There must be something I'm missing. Something else that I need to get rid of in order, or do, to run my blob storage trigger again over all of the same files. Any ideas/suggestions?

Answer 1

Yes, Azure Function will run on the same files over multiple times. It will update the content in the same files if already exists.

Before updating the content on same files, it will check the file exists or not internally and if present, will fails the operation until we provide permission to overwrite through code level or in portal when we upload the Blob Files manually.

I have tried to run the Azure Function Blob Storage Trigger by uploading the same files. When uploading the same file from 2nd time, it asked the file exists so overwrite or do not upload options. If we select overwrite, then the function runs the trigger and updates the content.

For Multiple Files:

First Time - Uploaded 15 Text Files. Second Time - Uploaded 25 Text Files (15 were the same, 10 were New)

Overwrite option produced for the existing files uploading and accepted to overwrite.

function has run over thousands of files I'll go and check inside of 'blobreceipts' and find that there is only one file/blob in there. Or alternatively, there will be a lot of files that I'll have to delete from there, but even after doing so, it seems my function doesn't run over every single file again.

During the running of the Azure Function, when you upload a lot of files, you have to write the logic like:

If blob files exist, then overwrite else upload as new to the blob container and process the file data in the function.

For this logic, refer the Overwriting the Blob Storage in Azure Function Code, checking if a blob exists in Azure Storage.

Also, one of the workarounds is to upload the multiple JSON file in parallel to blob storage.

If you write the logic in Azure Function Code manually for overwriting the existing files, then the Blob Storage trigger function will run on all the files.