3

I have a node_exporter web server running which serves files from a directory. These files have host metrics at a particular point-in-time. I have a daemon running that scrapes metrics from other exporters every 15 seconds and writes to this directory. When it writes files, I think it removes the existing file and writes a new file. Because of this, once in a while, for a split second node_exporter serves no metrics from an exporter. How do I make sure the previous metrics exist until the new metrics are written and there's no empty/duplicate metrics?

pdna
  • 541
  • 1
  • 8
  • 17
  • Added duplicates describing how to do atomic updates both from node and from native bash, since the question doesn't include explicit code to be unambiguous. – Charles Duffy Aug 05 '18 at 23:40
  • So in bash there's no way to fix this? I don't see a solution in the duplicate you added for bash. Only says its not atomic – pdna Aug 05 '18 at 23:48
  • Renames within a single filesystem *are* atomic. I believe that's covered in the dupes -- I'll check, and if not, add more. – Charles Duffy Aug 06 '18 at 00:23
  • Ahh -- the node.js duplicate covers the create-and-rename pattern, the bash one didn't. Added an additional bash duplicate covering that ground. – Charles Duffy Aug 06 '18 at 00:25
  • To speak to the mechanism -- `foo > bar` opens the file `bar` with `O_TRUNC` *before* the command `foo` is even started, so it can connect `foo`'s stdout (FD 1) to that file. – Charles Duffy Aug 06 '18 at 00:50

1 Answers1

1
  1. write to a temporary file in the same directory

  2. when done, rename the temporary file to the result file

    mv tempfile outputfile
Benjamin W.
  • 46,058
  • 19
  • 106
  • 116
anneb
  • 5,510
  • 2
  • 26
  • 26
  • The question was tagged with "bash". If renamed from Node, Node should have the same directory and file access permissions. This might be somewhat less secure if Node is also a webserver. – anneb Aug 05 '18 at 23:42
  • so move command is atomic but redirection isn't? – pdna Aug 05 '18 at 23:49
  • yeah looks like its atomic. thanks! – pdna Aug 05 '18 at 23:52
  • 1
    redirection is not atomic. mv is 'mostly' atomic, see discussions about atomicity mv on stackoverflow – anneb Aug 05 '18 at 23:53