0

I'm trying to store only distinct rows in my output- Blob storage based on old data in that output-blob storage using azure stream analytics query language. Eg: On the first insert, I get some values to the streaming analytics input as below

Name, Score
saran, 100
raj,   90

and the above data is stored in blob as output.

On the second insert, I get the following table from stream analytics input

Name, Score
saran, 90
jeeva, 80

Now before storing this data to the blob, I have to compare the old data from a blob with new data and to insert only the distinct data to the blob and also I wanna update the value for already existing data. Thanks in Advance

saranraj kumar
  • 207
  • 1
  • 2
  • 10

1 Answers1

1

Your main requirement is implementing Upsert feature in the ASA blob storage output,and as i know , you can't get that feature in blob storage output.

However,you could consider loading the data into below output before blob storage(final destination) as workarounds:

1.Azure Cosmos DB SQL API:

Stream Analytics integration with Azure Cosmos DB allows you to insert or update records in your container based on a given Document ID column.More details,please refer to this official document.

2.SQL Database:

You could use stored procedure in the sql database to implement upsert,please refer to this thread:How to implement a conditional Upsert stored procedure?

After above process, you could load data into blob storage.For example,using copy activity in ADF.

Jay Gong
  • 23,163
  • 2
  • 27
  • 32
  • Thank you, How can I do the same in the question that I have asked, If I use Power BI as Output. – saranraj kumar Aug 02 '19 at 08:06
  • @saranrajkumar All i know about upsert in ASA are listed in my answer. You could connect PB to Azure Blob Storage:http://georgevwoods.com/2019/01/26/connect-settings-to-use-azure-blob-storage-as-a-data-source-for-the-power-bi-service/ – Jay Gong Aug 02 '19 at 08:26
  • @saranrajkumar So,any updates?Does my answer helps you? – Jay Gong Aug 12 '19 at 06:29