Graphite whisper losing old data

Question

My graphite whisper databases don't store data older than 7 days old. Using the whisper-fetch.py tool I can only see data up to 1 week back (when there should be data for at least 3 weeks at this point). I'm using grafana, statsd, graphite-web, carbon and whisper.

An example of one of the metrics being truncated:

stats.counters.api.create_order.pc.chrome

This is my storage-schemas.conf:

[carbon]
pattern = ^carbon\.
retentions = 10s:6h,1min:90d

[default_1min_for_1day]
pattern = .*
retentions = 10s:6h,1min:6d,10min:5y

And here is my storage-aggregation.conf:

[min]
pattern = \.lower$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.upper(_\d+)?$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.sum$
xFilesFactor = 0
aggregationMethod = sum

[count]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[count_legacy]
pattern = ^stats_counts.*
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

can you use whisper-info to get the expected retention of data ? — kamaradclimber, May 30 '15 at 09:00

score 4 · Accepted Answer · answered May 31 '15 at 05:25

I found the issue. Using whisper-info.py, the retention times were shown to be 7 days on all the old metrics, as I was witnessing. Newer metrics were being retained just fine. The current storage-schemas.conf didn't have 7 days specified anywhere... but the previous storage-schemas.conf file could have had it.

It seems graphite does not automatically update the whisper files for new retention times specified in storage-schemas.conf I used the whisper-resize.py tool to manually resize all the old metric whisper files, and now whisper-info.py shows the right retention periods.

yes indeed, graphite does not change retention on the fly. – kamaradclimber Jun 10 '15 at 06:26 — kamaradclimber, Jun 10 '15 at 06:26

Graphite whisper losing old data

1 Answers1