Stack Overflow
Stack Overflow

Questions tagged [observability]

monitoring the internal state of a system by looking at its output

Observability is the ability to answer any question about a business or application through the collection and analysis of data. Succinctly, it’s an approach to understanding the operation of a system by reviewing output from the system. In the software world, observability generally is framed in the context of the ‘three pillars’ or telemetry data types: metrics, traces, and logs. Combining these three types of data gives you the power to answer questions about your business/application that you may not have known you’d need answers to when you set it up..

83 questions
5
votes
2 answers

Where does micrometer store data?

As i am in doubt when we use micrometer and prometheus in production as prometheus pull data form micrometer and we just use remote data storage for prometheus but some data are also stored by micrometer.. now my question is if my server is running…
Pradeep Kumar
  • 59
  • 3
4
votes
1 answer

Filter the parts of a Request Path which match against a Static Segment in Servant

Supposing I'm running a Servant webserver, with two endpoints, with a type looking like this: type BookAPI = "books" :> Get '[JSON] (Map Text Text) :<|> "book" :> Capture "Name" Text :> ReqBody '[JSON] (Text) :> Post '[JSON]…
Joe
  • 1,479
  • 13
  • 22
3
votes
1 answer

Open-telemetry auto instrumentation does not work without sidecar

I work at a startup, and we recently migrated our workloads to use Kubernetes, specifically we are running inside a cluster in EKS (AWS). I'm currently trying to implement a observability stack on our cluster. I'm running Signoz on a separate EC2…
Victor Chacon
  • 31
  • 3
3
votes
1 answer

How to configure OpenTelemetry agent for an Akka application

I am trying to export metrics and traces from my Akka app written in Scala using OpenTelemetry agent with the purpose of consuming the data in OpenSearch. Technology stack for my application: Akka - 2.6.* RabbitMQ (amqp client 5.12.*) PostgreSQL…
vasigorc
  • 882
  • 11
  • 22
2
votes
0 answers

How to create monitor/alert based on transaction duration in DataDog?

DataDog logs can be grouped into transactions and DataDog calculates each transactions duration, as shown here. DataDog docs say we can surface transactions with high latency but I cannot find any way to do it. This is what I have so far: What I am…
Abubakar Mehmood
  • 938
  • 1
  • 10
  • 19
2
votes
0 answers

no matches for kind "OpenTelemetryCollector" in version "opentelemetry.io/v1alpha1"

I tried instrumenting the open telemetry by deploying the following script. apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: my-instrumentation spec: exporter: endpoint: http://simplest-collector-headless:14250 …
codeX
  • 4,842
  • 2
  • 31
  • 36
2
votes
2 answers

Aggregate same metric, different labels at scrape time

Given a metrics with various labels (i.e., same metric name with different labels), is it possible to aggregate these at scrape time into a single value, e.g., by summing all the values? I know this is possible at query time, but I'm asking about…
BeeOnRope
  • 60,350
  • 16
  • 207
  • 386
2
votes
1 answer

Using Opentelemetry to write logs for Python

We are using Python and we want to use Opentelemetry for logs. I see that Logs for Python are not supported yet(https://opentelemetry.io/docs/instrumentation/python/) and for C++/.NET it is in beta/experimental. My question is will it make any sense…
Yanir Raveh
  • 23
  • 3
2
votes
0 answers

How to implement observability (as described by Charity Majors) in an Asp.Net Core application?

In https://charity.wtf/tag/observability/ in the section HOW TO GATHER AND STRUCTURE YOUR DATA Charity Majors presents the following scheme: Early in the pipeline initialize a "blob" accessible everywhere in code with every request having its own…
mark
  • 59,016
  • 79
  • 296
  • 580
1
vote
1 answer

Error Loki distributed with Helm: no private IP address found, and explicit IP not provided

I'm configuring loki-distributed on a kubernetes cluster via helm, but I'm getting the following error: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided. I found only one…
SrCabra
  • 33
  • 6
1
vote
1 answer

Dependencies do not show with OpenTelemetry automatic instrumentation for PHP

I have a very simple stack of PHP + Otel auto instrumetation + Elastic APM + Prometheus. My stack gets tracing and metrics information from my php app. My tracings are forwarded to the elastic cloud, where I use APM. My metrics go to Prometheus. I…
Pablo Marques
  • 39
  • 7
1
vote
0 answers

Prometheus metrics error "was collected before with the same name and label values"

I have this service made in go that exposes RESTful API. I have a middleware that keeps track of requests via prometheus metrics (countervec and histogram) -> requests_total and request_duration. Every time a request comes in, the middleware calls 2…
rolldawg
  • 11
  • 2
1
vote
1 answer

how to create queries in prometheus with wildcard

my application emits counter metrics something like this: app_counters_some_client_response_code_201_count_value{} app_counters_some_client_response_code_400_count_value{} app_counters_some_client_response_code_404_count_value{} I want to create a…
Shahbaz Khan
  • 93
  • 7
1
vote
1 answer

What's the best practice for restoring the trace id when a 3rd party integration interrupts the trace?

When we receive an HTTP call, we create a new trace id, and pass it to all microservices involved in dealing with that request and the subrequests that result from it. This works fine until a certain flow involves calling an external service that…
Federico Fissore
  • 712
  • 4
  • 18
1
vote
1 answer

Combine Micrometer's @Observed and B3 Trace ID value

The goal is to combine traces of network calls and spans observed by Micrometer. Network calls are observed with B3-headers. Method calls are observed with @Observed annotation. I have a B3-Trace-ID value that I want to combine with observed methods…
Dmytro Nasyrov
  • 15
  • 2
1
2 3 4 5 6