1

I have an existing system which uploads entities to Azure Table Storage in response to usage of our product. I am tasked with keeping a separate analytics system up to date with a copy of all entities stored in Azure Table Storage.

I have written a process to list all existing entities on Azure and import them into the analytics system. This allows me to run a once-off import of the Azure Table data into the analytics system. This is working and I have completed a once-off import.

I now need to keep the system up to date as new items are uploaded to Azure Table Storage.

I had hoped to run an Azure Function in response to a new item being added to Table Storage, but it appears that Azure Table Storage cannot be used as a trigger for Azure functions.

The next approach I was considering was to somehow list any new items on table storage. I have easy access to the most recent Partition Key, Row Key and Timestamp of the entity added to the analytics system, but I cannot find a way to any list entities with a timestamp greater than some give timestamp using the NodeJS Azure SDK. If I could list the new entities, then I would be able to periodically query for new entities and add them to the analytics system.

Is there a way to be notified of new uploads to Table Storage? Or to list any items created after a given timestamp?

Marc O'Morain
  • 3,699
  • 5
  • 28
  • 34

2 Answers2

1

This doesn't seem to be possible, but one option you may consider is migrating your solution to use CosmosDb Table API, where CosmosDb does support triggering Functions through the Change Feed feature.

From the documentation:

Specifically, the .NET Azure Tables SDK Azure.Data.Tables offers the latest features supported by the API for Table. The Azure Tables client library can seamlessly target either Azure Table storage or API for Table service endpoints with no code changes.

It ought to be possible to migrate such that the client code can't tell the difference.

To address your point about querying - why can't you do this? This would seem to be as straightforward as any other filter query using the SDK as described. I'm not familiar with the Node SDK, but I have no reason to think it wouldn't be supported based on my experience with the .NET SDK.

Tom W
  • 5,108
  • 4
  • 30
  • 52
1

If you can choose the PartitionKey & RowKey yourself you can use a timestamp (converted to ticks) to ensure new entries are being put at the very top of your table. Doing that also gives you the option to filter your for certain time ranges with greater & lower filters.

Thomas
  • 4,030
  • 4
  • 40
  • 79