4

I created a sample application to send events to the event hub, which subsequently sends data to the azure data explorer database. I can see the events appearing in the event hub, but the same is not getting ingested in the Azure Data Explorer database even after creating the data ingestion from the event hub.

I have already deleted and recreated the data explorer and along with the database and data ingestion from the event hub, but no success. BTW, I have also done metadata mapping.

The data should be inserted to the ADX database table, and I should be able to query the data, but the query to the table returns a blank.

Brock Adams
  • 90,639
  • 22
  • 233
  • 295
Neeraj
  • 158
  • 8
  • I was about to write the same question. One more update: last week Friday I could able to ingest data and able to view the data in adx table. I have no clue what am missing now . – Ada Pongaya Jul 22 '19 at 23:44
  • Will update this thread if found something useful – Ada Pongaya Jul 22 '19 at 23:48

3 Answers3

3

Thanks, Yoni for providing the command to debug. It helped in debugging. I finally found the solution by changing the type of table from JSON to MultiJSON as the data included nested JSON data.

If your JSON does not contain nested/multiline JSON, you can choose simple JSON, else for multiline JSON, choose MultiJSON

Neeraj
  • 158
  • 8
2

a few data points which may be helpful in troubleshooting this:

  1. does .show ingestion failures run against your ADX cluster show anything related?

  2. how does your ingestion mapping look like, and how does a sample event look like?

Yoni L.
  • 22,627
  • 2
  • 29
  • 48
  • It's json data. { name : "Ada", age: 28 } – Ada Pongaya Jul 22 '19 at 23:51
  • 1
    a valid JSON payload would have looked like: `{"name": "Ada", "age": 28}`. plus, what about the other questions? – Yoni L. Jul 23 '19 at 00:03
  • Yes, when I ran it, it shows records and says that the JSON was ill-formed. I will work on it now and will get back in case I face issues again. For now, I will mark it as the answer. – Neeraj Jul 23 '19 at 00:12
  • Stream_NoDataToIngest: The input stream produced 0 bytes. This usually means that the input JSON stream was ill formed. Ingestion Mapping: {"column":"name","datatype":"string","Path":"$.name","Transform":"None"},{"column":"age","datatype":"int","Path":"$.age","Transform":"None"} – Ada Pongaya Jul 23 '19 at 00:29
  • again, it's imperative to understand how your data looks like in order to answer your question. you mentioned you can see events appearing in event hub. how does a single sample message look like (exactly, including whitespaces, linebreaks, etc.)? Plus - did you setup format as `json` or `multijson` in your EventHub connection? – Yoni L. Jul 23 '19 at 00:42
  • This is how I am creating the JSON - – Neeraj Jul 23 '19 at 04:12
  • Yes, I did setup the data for the table as json. I want to share the details here but it is too long – Neeraj Jul 23 '19 at 04:23
  • you always have the (official) option of opening a support ticket via the Azure portal – Yoni L. Jul 23 '19 at 04:27
1

For me Multiline JSON has turned out to be the issue. Using Multiline JSON as the data format has fixed the issue for me.

Sampath Kumar K
  • 61
  • 1
  • 3