0

We have a Azure WebJob that fires a Timer every 15 seconds to poll an external Xml data feed. The request returns a 2.4 MB XML file with traffic data. Before reading data the method checks the LastModified header value against the value in the last received data - this is in line with recommendations set by the data feed vendor.

The WebJob works for days/weeks without issue but occasionally it will hang whilst reading the XML data into a memory stream for processing. The last entry on the WebJob log is "Starting copy content to memory stream".

    HttpRequestMessage webRequest = new HttpRequestMessage(HttpMethod.Get, feedPath);

using (var _webApiHttpClient = new HttpClient())
{
    _webApiHttpClient.BaseAddress = new Uri(SettingsManager.Instance.BaseUrl);

    _webApiHttpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("*/*"));

    log.WriteLine(webRequest.RequestUri.ToString());
    using (HttpResponseMessage response = await _webApiHttpClient.SendAsync(webRequest, HttpCompletionOption.ResponseHeadersRead))
    {
        var dateReceived = DateTime.Now.ToUniversalTime();
        log.WriteLine("Request result: {0}", response.StatusCode);

        if (response.IsSuccessStatusCode)
        {
            log.WriteLine("Received feed data");
            DateTimeOffset lastRead = new DateTimeOffset();
            if (lastChangedBlob.Exists())
            {
                using (var stream = lastChangedBlob.OpenRead())
                {
                    BinaryFormatter f = new BinaryFormatter();
                    lastRead = (DateTimeOffset)f.Deserialize(stream);
                }
            }

            log.WriteLine("Content last modified {0} and last read {1}", response.Content.Headers.LastModified, lastRead);
            var modified = response.Content.Headers.LastModified;

            if (modified.HasValue && modified.Value > lastRead)
            {

                log.WriteLine("Starting copy content to memory stream");

                using (Stream stream = new MemoryStream())
                {
                    await response.Content.CopyToAsync(stream);
                    log.WriteLine("Copy completed: response.Content.CopyToAsync");
                    // Go back to start of stream to save file to BLOB
                    stream.Seek((long)0, SeekOrigin.Begin);
                    log.WriteLine("Seeked begining of stream ready to ProcessFeed");
                    await ProcessFeed(type, output, queue, log, dateReceived, stream);
                    log.WriteLine("ProcessFeed completed");
                }

....

user2486488
  • 91
  • 1
  • 8
  • Did you check you're not getting any timeout? Related [Q/A](https://stackoverflow.com/questions/26596968/azure-webjob-timeout-configuration-settings) – huysentruitw Jan 29 '18 at 12:10
  • The web job itself doesn't terminate or timeout. I'm using a C# web job sdk TimerTrigger and it appears to wait at CopyToAsync. The only way I can terminate (after 2 days) is to Abort Host on the WebJob dashboard. This restarts the webjob and temporarily fixes things. – user2486488 Jan 29 '18 at 12:48
  • Are there any blocking calls anywhere in the call stack during this operation? – Crowcoder Jan 29 '18 at 13:11
  • 1
    I don't like "using" statement because you can't capture the exceptions.messages. I would replace the using with a try/catch and display the full trace stack error to help find the issue. You may have a memory leak so I would display a new message indicating the length of the response which may help determine the root cause. – jdweng Jan 29 '18 at 20:51

0 Answers0