Arithmetic operation overflow in Azure Stream Analytics with reference data

Question

I have an Azure Stream Analytics job that enhances incoming data with some reference data. Some of them, e. g. physical addresses change not so often.

A generic script would look like this.

SELECT
    a.input1
    a.input2
    b.input1
INTO output
from a
LEFT JOIN metadata AS b ON a.input1 = b.input1

If I try to run this script, I get an error:

System Exception Arithmetic operation resulted in an overflow. at Microsoft.Streaming.StreamingNode.ReferenceData.DiscoveryCursor.get_NextDiscoveryDueTimeInMillisecond() at Microsoft.Streaming.StreamingNode.ReferenceData.ReferenceDataDiscoveryStrategyBase.Discover() at Microsoft.Streaming.StreamingNode.ReferenceData.ReferenceDataDiscoveryProcessor.Process(Boolean isInitialization) at Microsoft.Streaming.StreamingNode.ReferenceData.ReferenceDataDiscoveryProcessor.InitializeInput() at Microsoft.Streaming.StreamingNode.ProcessorScheduler.InitializeTopology(CancellationToken cToken) at Microsoft.Streaming.StreamingNode.ProcessorScheduler.ThreadProc()

I simplified the script step by step. Eliminating every notion of b in the select statement was not sufficient, I even had to delete the Join statement.

What troubles me is that there is no arithmetic operation, so I have no idea where the overflow should come from. And the reference data in my database are ridiculously simple.

Thanks a lot for the feedback. I'll be looking into it. Per your find below it looks like some kind of overflow on the time counter. I will follow-up. — Florian Eiden, Dec 13 '21 at 17:34
I reproduced the issue and created a bug on our side. I will keep you posted. — Florian Eiden, Dec 14 '21 at 04:52

score 1 · Answer 1 · answered Dec 13 '21 at 13:08

The solution to the above problem struck me by chance. Due to the rarely changing reference data I set the refresh-rate of the reference data to something like 30 days.

This leads to the overflow error. After reducing the refresh limit to 10 days I encountered no further issues and my ASA query worked.

Hope this helps somebody and saves some time.

Arithmetic operation overflow in Azure Stream Analytics with reference data

1 Answers1