6

I have had a node web service running successfully on an aws ubuntu server for over a month, with the requests cached using redis.

Yesterday I started getting the following error from some of my routes:

MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

I was able to stop the error occurring by using:

config set stop-writes-on-bgsave-error no

as suggested in the answers to this question, but it doesn't actually solve the underlying problem.

To find the underlying problem I checked the logs and found the following had started happening:

[1105] 09 Aug 13:17:14.800 - 0 clients connected (0 slaves), 797680 bytes in use
[1105] 09 Aug 13:17:15.101 * 1 changes in 900 seconds. Saving...
[1105] 09 Aug 13:17:15.101 * Background saving started by pid 28090
[28090] 09 Aug 13:17:15.101 # Failed opening .rdb for saving: Permission denied
[1105] 09 Aug 13:17:15.201 # Background saving error

Over the weekend no one had been using the server, but before the weekend the logs were fine, and we were getting no errors:

[12521] 06 Aug 04:49:27.308 - 0 clients connected (0 slaves), 803352 bytes in use
[12521] 06 Aug 04:49:29.012 * 1 changes in 900 seconds. Saving...
[12521] 06 Aug 04:49:29.012 * Background saving started by pid 26663
[26663] 06 Aug 04:49:29.014 * DB saved on disk
[26663] 06 Aug 04:49:29.014 * RDB: 2 MB of memory used by copy-on-write
[12521] 06 Aug 04:49:29.112 * Background saving terminated with success

As I said, no one has touched this server in the intervening time.

Looking around for people having the same problem I found this question. I checked the ownership and permissions on the directory and db file as suggested in the answers there:

drwxr-xr-x 2 redis   redis     26 Aug  6 06:55 redis
-rw-r--r-- 1 redis   redis     18 Aug  6 06:55 dump-6379.rdb

The permissions and ownership both look ok to me, but I have noticed that the date on the file and folder is between the last time I saw the service working and the first time it failed. Unfortunately that hasn't really helped me with what to do next and I am at a bit of a loss.

I am looking for suggestions for next steps to find the cause of the problem, or at least a way of making redis able to write again.

Community
  • 1
  • 1
Carasel
  • 2,740
  • 5
  • 32
  • 51

0 Answers0