1

I am using AWS QuickSight to display charts. The data source is an AWS IoT analytics dataset.

Is it possible to have AWS IoT analytics datastore data into the dataset in real time? I know that we can manually do run now and this will have data as of the time we press the run now. Is there any way to automatically have realtime data into the dataset?

Currently the minimum frequency for automating pulling of data from the datastore into the dataset is 1 minute. This means that the data in the dataset (and hence QuickSight charts) will always be 1 minute stale.

variable
  • 8,262
  • 9
  • 95
  • 215

1 Answers1

2

Scheduling your query to run regularly in order to refresh the dataset is the only automated option when you use AWS IoT Analytics datasets. The highest frequency is 1 minute (near real-time) which is perfectly fine for most IoT-enabled solutions.

An alternative solution that is worth investigating would be using AWS IoT Core and forwarding messages from the topic stream to Lambda through IoT Core rules, then putting records on an Amazon Kinesis Data Firehose delivery stream, and finally using Amazon QuickSight to visualize the data stored within an S3 bucket. This link is a tutorial about how to accomplish it.

kahveci
  • 1,429
  • 9
  • 23
  • Instead of S3 bucket, how about AWS DynamoDb or AWS Timestream db? – variable Nov 03 '20 at 09:33
  • It really depends on your use-case. As you know, S3 is not a database and good for storing static datasets while offering a significant cost advantage. If your business problem is time sensitive and relies on streaming data, I would suggest considering a timeseries database. In this case, you should also question whether Amazon QuickSight is good enough for the presentation layer. Hope this helps. – kahveci Nov 04 '20 at 10:14
  • For IoT what are other choices for presentation layer? – variable Nov 04 '20 at 10:17
  • Also, you mentioned "messages from the topic stream to Lambda through IoT Core rules, then putting records on an Amazon Kinesis Data Firehose delivery stream" - Instead of Lambda action why not directly to the 'Kinesis Data Firehose' action? – variable Nov 04 '20 at 10:19
  • For the presentation layer, I would recommend that you have a look at Grafana. Obviously, it is not a managed AWS service, but still can be easily deployed and scaled on AWS. Lambda gives you an extra layer of flexibility to process your payload, add some business logic, and route into different destinations. Again it really depends on your business problem, which I do not have much detail about. – kahveci Nov 04 '20 at 16:46
  • How is the near real-time dashboard accomplished in quicksight? Let's suppose I automate the AWS IoT analytics to the 1 minute frequency. Is there a way to refresh also in quicksight? I think it has a minimun of 1 hour for refreshing but I'm not sure. – jessica Feb 11 '21 at 18:16
  • I don't think QuickSight is designed for near real-time dashboarding. I would recommend that you have a look at Grafana for this purpose. Good news is that there is now a fully managed Grafana service on AWS: https://aws.amazon.com/grafana/ – kahveci Feb 11 '21 at 20:02