Add a string in front of a string and at the end of another string in alternating lines

Question

Here's my file format:

{"index":{}}
{"LAST_MODIFIED_BY":"john","CREATED_BY":"david","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a63"}
{"index":{}}
{"LAST_MODIFIED_BY":"joe","CREATED_BY":"brian","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test2","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a62"}
...
{"index":{}}
{"LAST_MODIFIED_BY":"sarah","CREATED_BY":"hannah","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test8","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a61"}

I need to add "TAGS":{} to encompass these two fields: BATCH_TAG and TASK_TYPE_NAME, so the desired output looks like this:

{"index":{}}
{"LAST_MODIFIED_BY":"john","CREATED_BY":"david","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a63"}
{"index":{}}
{"LAST_MODIFIED_BY":"joe","CREATED_BY":"brian","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test2"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a62"}
...
{"index":{}}
{"LAST_MODIFIED_BY":"sarah","CREATED_BY":"hannah","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test8"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a61"}

The closest that I could find is this one: Add a prefix string to beginning of each line but it adds a string in the front of each line, which is not really what I wanted.

Could anyone shed any light onto this please?

Thanks!

peak · Accepted Answer · 2021-06-22T20:45:43.410

2

The tricky bit here is ensuring the new TAGS key is inserted in the desired position. Using jq, this can be done with with_entries:

if has("BATCH_TAG")
then .BATCH_TAG = {BATCH_TAG, TASK_TYPE_NAME}
| with_entries(.key |= if . == "BATCH_TAG" 
                       then "TAGS"
                       else . end)
| del(.TASK_TYPE_NAME)
else .
end

edited Jun 22 '21 at 20:45

answered Jun 22 '21 at 20:35

peak

105,803
17
152
177

thanks for the response, but how can I run this using `jq`? I tried to run `jq if has("BATCH_TAG") then .BATCH_TAG = {BATCH_TAG, TASK_TYPE_NAME} | with_entries(.key |= if . == "BATCH_TAG" then "TAGS" else . end) | del(.TASK_TYPE_NAME) else . end input.txt > output.txt`, but it didn't work. – Fisher Coder Jun 22 '21 at 21:12
Perhaps you would find it simplest to put the jq program in a file, say program.jq, and then run something like: `jq -cf program.jq input.json`. – peak Jun 22 '21 at 21:15
Thank you so much, this did the job! – Fisher Coder Jun 22 '21 at 21:18

Ed Morton · Answer 2 · 2021-06-22T18:13:02.513

Since your question was Could anyone shed any light onto this please?, here is some light:

The file format you're working with appears to be JSON so use a tool that understands JSON to edit it, e.g. jq.

For example, when run on your sample input (after the ... line cluttering it up and making it untestable was removed) here is jq pretty-printing it:

$ cat file
{"index":{}}
{"LAST_MODIFIED_BY":"john","CREATED_BY":"david","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a63"}
{"index":{}}
{"LAST_MODIFIED_BY":"joe","CREATED_BY":"brian","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test2","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a62"}
{"index":{}}
{"LAST_MODIFIED_BY":"sarah","CREATED_BY":"hannah","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","BATCH_TAG":"","TASK_TYPE_NAME":"Test8","PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a61"}

$ jq . < file
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "john",
  "CREATED_BY": "david",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "BATCH_TAG": "",
  "TASK_TYPE_NAME": "Test",
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a63"
}
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "joe",
  "CREATED_BY": "brian",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "BATCH_TAG": "",
  "TASK_TYPE_NAME": "Test2",
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a62"
}
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "sarah",
  "CREATED_BY": "hannah",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "BATCH_TAG": "",
  "TASK_TYPE_NAME": "Test8",
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a61"
}

and with your desired output:

$ cat file2
{"index":{}}
{"LAST_MODIFIED_BY":"john","CREATED_BY":"david","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a63"}
{"index":{}}
{"LAST_MODIFIED_BY":"joe","CREATED_BY":"brian","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test2"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a62"}
{"index":{}}
{"LAST_MODIFIED_BY":"joe","CREATED_BY":"brian","LAST_MODIFIED_DATE":"2018-01-11T06:39:11.000Z","CREATION_DATE":"2018-01-11T06:39:11.000Z","INGESTION_DATE":"2018-01-11T06:39:11.000Z","DECISION":"AGREE","TAGS":{"BATCH_TAG":"","TASK_TYPE_NAME":"Test2"},"PUBLISHER_ID":"abc123","ID":"700adca2-c4d4-4245-8174-0479f7123a62"}

$ jq . < file2
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "john",
  "CREATED_BY": "david",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "TAGS": {
    "BATCH_TAG": "",
    "TASK_TYPE_NAME": "Test"
  },
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a63"
}
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "joe",
  "CREATED_BY": "brian",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "TAGS": {
    "BATCH_TAG": "",
    "TASK_TYPE_NAME": "Test2"
  },
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a62"
}
{
  "index": {}
}
{
  "LAST_MODIFIED_BY": "joe",
  "CREATED_BY": "brian",
  "LAST_MODIFIED_DATE": "2018-01-11T06:39:11.000Z",
  "CREATION_DATE": "2018-01-11T06:39:11.000Z",
  "INGESTION_DATE": "2018-01-11T06:39:11.000Z",
  "DECISION": "AGREE",
  "TAGS": {
    "BATCH_TAG": "",
    "TASK_TYPE_NAME": "Test2"
  },
  "PUBLISHER_ID": "abc123",
  "ID": "700adca2-c4d4-4245-8174-0479f7123a62"
}

Read the jq man page and/or google it for more information on how to apply it to your problem.

Thanks Ed, I went through man page of `jq` and found `with_entries` examples which helped me better understand this powerful `jq` utility. Thanks! — Fisher Coder, Jun 22 '21 at 21:20

Add a string in front of a string and at the end of another string in alternating lines

2 Answers2