1

I am trying to update a simple JSON file (consists of one object with several key/value pairs) and I am using the same command yet getting different results (sometimes even having the whole json wiped with the 2nd command). The command I am trying is:

cat ~/Desktop/config.json | jq '.Option = "klay 10"' | tee ~/Desktop/config.json

This command perfectly replaces the value of the minerOptions key with "klay 10", my intended output.

Then, I try to run the same process on the newly updated file (just value is changed for that one key) and only get interactive terminal with no result. ps unfortunately isn't helpful in showing what's going on. This is what I do after getting that first command to perfectly change the value of the key: 

cat ~/Desktop/config.json | jq ‘.othOptions = "-epool etc-eu1.nanopool.org:14324 -ewal 0xc63c1e59c54ca935bd491ac68fe9a7f1139bdbc0 -mode 1"' | tee ~/Desktop/config.json

which I would have expected would replace the othOptions key value with the assigned result, just as the last did. I tried directly sending the stdout to the file, but no result there either. I even tried piping one more time and creating a temp file and then moving it to change to original, all of these, as opposed to the same identical command, just return > and absolutely zero output; when I quit the process, it is the same value as before, not the new one.

What am I missing here that is causing the same command with just different inputs (the key in second comes right after first and has identical structure, it's not creating an object or anything, just key-val pair like first. I thought it could be tee but any other implementation like a passing of stdout to file produces the same constant > waiting for a command, no user.

I genuinely looked everywhere I could online for why this could be happening before resorting to SE, it's giving me such a headache for what I thought should be simple.

Cristian Ramon-Cortes
  • 1,838
  • 1
  • 19
  • 32
Michael W
  • 343
  • 2
  • 11
  • First, you're missing a call to `cat`, the file name alone won't do anything, second, don't use `cat`, third, you used a left quote for your command... use a regular single quote. It's barely the "same command" as the one that works. – Jeff Mercado Dec 19 '17 at 07:43
  • Actually you want to avoid the [useless use of `cat`](https://stackoverflow.com/questions/11710552/useless-use-of-cat). – tripleee Dec 19 '17 at 08:26
  • Reading and writing the same file in a pipeline is not safe. All commands in the pipeline run more-or-less simultaneously, so `cat` might or might not finish reading the original contents before `tail` blanks it to get it ready for new contents. Have `jq` output to a temp file, then check for errors (did `jq` exit successfully?), and then (if there were no errors) replace the original with the temp file. – Gordon Davisson Dec 19 '17 at 08:57

1 Answers1

1

As @GordonDavisson pointed out, using tee to overwrite the input file is a (well-known - see e.g. the jq FAQ) recipe for disaster. If you absolutely positively want to overwrite the file unconditionally, then you might want to consider using sponge, as in

jq ... config.json | sponge config.json

or more safely:

cp -p config.json config.json.bak && jq ... config.json | sponge config.json

For further details about this and other options, search for ‘sponge’ in the FAQ.

peak
  • 105,803
  • 17
  • 152
  • 177
  • Thank you for this reply :) sponge is super useful, but it seems that sometimes the file will get wiped completely after a few jq pipelines to change values. All I'm trying to do is alter the values for a few keys within that file, not necessarily overwrite it (unless it's the only want to do so). Is there a better, maybe safer way to just change the values and have that become the new file? – Michael W Dec 20 '17 at 07:43
  • Write to another file and then when all the substitutions are done, replace the original file with your replacement. Running multiple processes to perform multiple replacements also sounds like a bad idea, though not data-loss bad, just inefficient and ugly bad. – tripleee Dec 20 '17 at 08:55