Sending jq with argjson via ssh

Question

I'm trying to run jq command via ssh for this JSON:

{
  "nodes": {
    "app": {
      "nodes": 1,
      "is_manager": true,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "data": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "analysis": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_1": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_2": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_3": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "master": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    }
  }
}

This is what I'm trying to run:

ssh -o StrictHostKeyChecking=no -i key.pem user@"172.13.1.23"
"jq -Rn --argjson original_doc \"\$(<nodes.json)\" '
  input | split(\"\u0000\") as \$ips
  | \$original_doc
  | .nodes.app.ip = \$ips[0]
  | .nodes.data.ip = \$ips[1]
  | .nodes.analysis.ip = \$ips[2]
  | .nodes.elastic_kafka_1.ip = \$ips[3]
  | .nodes.elastic_kafka_2.ip = \$ips[4]
  | .nodes.elastic_kafka_3.ip = \$ips[5]
  | .nodes.master.ip = \$ips[6]
' < <(printf '%s\0' \"\${GCP_INSTANCES[@]}\") > test.json && mv test.json nodes.json"

And this is an output:

{
  "nodes": {
    "app": {
      "nodes": 1,
      "is_manager": true,
      "ip": "",
      "cpus": 16,
      "memory": 64
    },
    "data": {
      "nodes": 1,
      "ip": "",
      "cpus": 16,
      "memory": 64
    },
    "analysis": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_1": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_2": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_3": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "master": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    }
  }
}

As you can see jq doesn't work properly due to some syntax issue with ssh or something.

I tested this command locally, without ssh, and it works properly.

I think the problem is with printf '%s\0', but couldn't figure out what exatly I'm doing wrong.

Generally speaking, don't try to quote shell commands to run remotely -- instead, let the shell itself do that work. — Charles Duffy, Oct 17 '19 at 13:07
Is `nodes.json` supposed to be on the local machine or the remote one? — Charles Duffy, Oct 17 '19 at 13:10
BTW, I'd argue that this question as a whole is duplicative of https://stackoverflow.com/questions/47558482/escaping-quotes-in-bash-embedded-awk -- it's awk in that case and jq in this one, but the underlying issue is the same. — Charles Duffy, Oct 17 '19 at 13:15

score 2 · Accepted Answer · answered Oct 17 '19 at 13:12

The easiest way to ensure that all the quoting is done right is to have the shell do it for you, by using declare -f to generate a textual representation of an already-locally-defined function, and declare -p to generate textual representations of any local variables that function needs access to. Thus:

doRemoteWork() {
  jq -Rn --argjson original_doc "$(<nodes.json)" '
    input | split("\u0000") as $ips
    | $original_doc
    | .nodes.app.ip = $ips[0]
    | .nodes.data.ip = $ips[1]
    | .nodes.analysis.ip = $ips[2]
    | .nodes.elastic_kafka_1.ip = $ips[3]
    | .nodes.elastic_kafka_2.ip = $ips[4]
    | .nodes.elastic_kafka_3.ip = $ips[5]
    | .nodes.master.ip = $ips[6]
  ' < <(printf '%s\0' "${GCP_INSTANCES[@]}") >"nodes.json.$$" \
  && mv "nodes.json.$$" nodes.json
}

ssh -o StrictHostKeyChecking=no -i key.pem user@172.13.1.23 \
  "$(declare -p GCP_INSTANCES; declare -f doRemoteWork); doRemoteWork"

Sending jq with argjson via ssh

1 Answers1

Linked