0

My Azure Function, invoked by a BlobTrigger, queries a SQL database and uploads the blob to Sharepoint using REST API. On average it takes about 5s to execute all the code. No issues here.

The problem is when many blobs get placed in the Blob Container at the same time and the Azure Function processes them in parallel. SQL queries may return wrong results, and Sharepoint uploads will fail often. To avoid this I want to process only one blob at a time.

I have no access to Event Grid (not part of our company's subscription). What other options are available to scale my project?

EDIT: As an example, I just uploaded 7 unique blobs (PDF-files) from which the Function extracts a Document number, queries a SQL db and awaits a response. When uploading one at a time, I ended up with 7 unique SQL responses. When uploading all blobs at the same time, I received the same SQL response 7 times.

AlexanderJ
  • 85
  • 2
  • 7
  • There is a contradiction here: *I want to process only one blob at a time.* vs *What other options are available to scale my project?*. Do you want to scale out or limit your processing? – Peter Bons Aug 28 '20 at 18:42
  • I mean scale as in: my solution works fine when I upload single blobs because only one blob is being processed. When I batch-upload blobs (e.g. >10) they're processed in parallel and several functionalities of my program (e.g. SQL-queries and Sharepoint uploads) don't work as expected anymore. Can I put blobs in a queue to avoid processing them in parallel? – AlexanderJ Aug 28 '20 at 18:52
  • you could do either: a) Limit the parallelism of your Function to 1. then they should all get processed sequentially. or b), and I would prefer this: the blob-triggered Function only puts the item into a queue. from there you have another Function which works sequentially on that queue to do the actual work – silent Aug 28 '20 at 19:42
  • I attempted to limit the scale out of my Function to 1 as described here (https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#limit-scale-out), but it didn't have the intended effect (blobs are still processed concurrently) – AlexanderJ Aug 28 '20 at 20:11
  • @silent, how would you do "a) Limit the parallelism of your Function to 1."? AFAIK it's not possible in this case because he's using a binding. – Kashyap Aug 28 '20 at 20:49
  • 1
    I answered to your other question (which is around the same problem) here https://stackoverflow.com/questions/63641197/instances-of-azure-functions-are-sharing-variables – krishg Aug 29 '20 at 09:17

0 Answers0