3

I'm trying to use Stackdriver tracing while running a Google Cloud Run instance.

However, when tracing a call from point A to the container instance, the trace parent_span_id is broken. This leads to a broken trace on the stackdriver view that looks like the following: enter image description here

The first line in the image is the call to my Cloud Run endpoint. The last two lines are the trace from that endpoint. Notice how the display fails to present them properly.

From my investigation, the parent_span_id in the span presented in the end is a span_id that is never reported to StackDriver, meaning the UI (or a human) can't put together the trace.

My theory is that the Google Cloud endpoint that does SSL/TLS termination replaces the span with it's own span (legitimate) but then never reports its own traffic to Stackdriver, breaking all traces that cross a GCR boundary.

This theory seems bolstered by the unofficial FAQ maintained by ahmetb (as of December 2019).

This seems to happen regardless of whether the container is using node.js or python or any other runtime.

Any ideas/suggestions or something I missed?

Daniel Goldberg
  • 19,908
  • 4
  • 21
  • 29
  • 2
    Have you seen this question https://stackoverflow.com/questions/59157906/stackdriver-trace-with-google-cloud-run ? – ahmet alp balkan Dec 17 '19 at 22:23
  • Yes. Infact I replied to you in that question. That question deals with flushing traces in GCR and seeing them. That's not my problem (I have set buffer size to 1, etc.). My problem, maybe I can state it better, is that the GCR runtime seems to create a new span before the request reaches me. That span seems to not be reported to stackdriver, preventing the stackdriver trace UI from properly displaying the order of operations. – Daniel Goldberg Dec 17 '19 at 23:00
  • 1
    My mistake, I replied to you in a different GCF stackdriver question – Daniel Goldberg Dec 17 '19 at 23:11
  • But the problem is not what you referred to – Daniel Goldberg Dec 18 '19 at 06:24
  • can you please explain how these spans are getting generated? Also, what does opencensus has to do with this? Are you using this library? https://opencensus.io/exporters/supported-exporters/java/stackdriver-trace/ – grimmjow_sms Dec 18 '19 at 20:51
  • I'm running in node.js (will edit question) and using the google trace-agent library which relies on the opencensus specification. – Daniel Goldberg Dec 19 '19 at 09:31
  • This seems to happen regardless of whether the container is using node.js or python or any other runtime. – Daniel Goldberg Dec 19 '19 at 09:32

0 Answers0