How can I get JSON data into Azure Time Series Insights through an Azure EventHub using Python?

Question

I am attempting to get some sample JSON data to show up in Azure Time Series Insights (TSI) using Python so I can visualize the data within their exploratory browser.

I have gone through the necessary prerequisites with regards to Azure EventHubs and Time Series Insights setup. These include:

1. Create a resource group in the Azure portal
2. Create an event hub namespace within the resource group
3. Create an event hub entity within that event hub namespace
4. Create a consumer group within the event hub entity
5. I set up an Azure TSI environment within the resource group.
6. And finally I added an Event Source to the Azure TSI environment using the different created sources as the details (resource group, event hub namespace, event hub name, etc.)

Beyond that I tested successfully sending event messages to my event hub (but not into the TSI environment) using python by following this documentation: https://learn.microsoft.com/en-us/azure/event-hubs/get-started-python-send-v2 and using this code (albeit con_str filled in and eventhub_name filled in:

import asyncio
from azure.eventhub.aio import EventHubProducerClient
from azure.eventhub import EventData

async def run():
    # Create a producer client to send messages to the event hub.
    # Specify a connection string to your event hubs namespace and
        # the event hub name.
    producer = EventHubProducerClient.from_connection_string(conn_str="EVENT HUBS NAMESPACE - CONNECTION STRING", eventhub_name="EVENT HUB NAME")
    async with producer:
        # Create a batch.
        event_data_batch = await producer.create_batch()

        # Add events to the batch.
        event_data_batch.add(EventData('First event '))
        event_data_batch.add(EventData('Second event'))
        event_data_batch.add(EventData('Third event'))

        # Send the batch of events to the event hub.
        await producer.send_batch(event_data_batch)

loop = asyncio.get_event_loop()
loop.run_until_complete(run())

I also tested successfully sending into my TSI environment Microsoft's Windmill Simulator data by following this documentation: https://learn.microsoft.com/en-us/azure/time-series-insights/time-series-insights-send-events

I am now at a loss on how to actually get sample JSON data into the Azure TSI environment using Python.

Any help would be appreciated. Thank you!

Answer 1

As the article mentions here, you will have to send the body of the EventData with the JSON formatted String.

Sharing you the modified snippet of the your above code :

import asyncio
import nest_asyncio
nest_asyncio.apply()
from azure.eventhub.aio import EventHubProducerClient
from azure.eventhub import EventData
import json

async def run():
    # Create a producer client to send messages to the event hub.
    # Specify a connection string to your event hubs namespace and
        # the event hub name.
    producer = EventHubProducerClient.from_connection_string("<>", eventhub_name="<>")
    async with producer:
        # Create a batch.
        event_data_batch = await producer.create_batch()

        # Add events to the batch.

        #Method 1 - You provide a JSON string 
        body1 = '{"id":"device2","timestamp":"2016-01-17T01:17:00Z"}' 
        event_data_batch.add(EventData(body1))

        #Method 2 - You get the JSON Object and convert to string
        json_obj = {"id":"device3","timestamp":"2016-01-18T01:17:00Z"}
        body2= json.dumps(json_obj)
        event_data_batch.add(EventData(body2))


        #This just sending the string which will not be captured by TSI
        event_data_batch.add(EventData('Third event'))

        # Send the batch of events to the event hub.
        await producer.send_batch(event_data_batch)


loop = asyncio.get_event_loop()
loop.run_until_complete(run())

Output :