-3

I have a file that has a list of my job API codes, and the status of that API but in the next line.

The api_file:

"id": "3b96db40-507e-4dd8-b2fd-e0e237fb928f",
"enabled": true,
"id": "3c07b7a7-0165-4017-84c6-b7e7383105a4",
"enabled": true,
"id": "41b2ed8f-8a21-4b3b-a690-2d7f24e577f6",
"enabled": true,
"id": "42767867-0a93-4cc4-93f8-c3893fe63edf",
"enabled": true,
"id": "4278314e-71ec-4e80-ae35-1dd67ceef2bd",
"enabled": false,
"id": "45c0c8c8-d0b6-4d82-87c3-1c66ec08cf1d",
"enabled": true,
"id": "492f0775-f6b8-4cf6-b355-02260513bd5a",
"enabled": true,
.....

I want to move the status line to the API code line (pair the every two lines), and I used sed 'N;s/[\n]//' command, and the output should look like this:

"id": "3b96db40-507e-4dd8-b2fd-e0e237fb928f","enabled": true,
"id": "3c07b7a7-0165-4017-84c6-b7e7383105a4","enabled": true,
"id": "41b2ed8f-8a21-4b3b-a690-2d7f24e577f6","enabled": true,
"id": "42767867-0a93-4cc4-93f8-c3893fe63edf","enabled": true,
"id": "4278314e-71ec-4e80-ae35-1dd67ceef2bd","enabled": false,
"id": "45c0c8c8-d0b6-4d82-87c3-1c66ec08cf1d","enabled": true,
"id": "492f0775-f6b8-4cf6-b355-02260513bd5a","enabled": true,

This works if I run it in shell command prompt, but when I use it in the Ansible playbook, I got an error:

The playbook:

- name: retrieve the IDs from the integration APIs
  shell: "cat api_file | sed 's/^ *//'  > merged_api_file"

And the error output:

TASK [retrieve the IDs from the integration APIs] *************************************************************************************************************************
  Friday 07 July 2023  16:12:23 +0000 (0:00:00.805)       0:00:04.707 *********** 
  fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/bin/cat /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-2023-07-06-22-15-46.conf | egrep \"\"id\"|\"enabled\"\" | sed 's/^ *//' | sed -e 'N;s/\n//' > /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-id-2023-07-06-22-15-46.conf", "delta": "0:00:00.005732", "end": "2023-07-07 16:12:23.294963", "msg": "non-zero return code", "rc": 1, "start": "2023-07-07 16:12:23.289231", "stderr": "sed: -e expression #1, char 4: unterminated `s' command", "stderr_lines": ["sed: -e expression #1, char 4: unterminated `s' command"], "stdout": "", "stdout_lines": []}

What is wrong with the sed command and how should I correct it?

Thanks!

U880D
  • 8,601
  • 6
  • 24
  • 40
Philip Shangguan
  • 449
  • 5
  • 19
  • 2
    Is there a reason why the command within the description highly differs from the command within the error message? – U880D Jul 07 '23 at 16:37

2 Answers2

1

The error message is:

sed: -e expression #1, char 4: unterminated `s' command

The command you show in your question is not the command that's producing the error, which actually looks like:

/usr/bin/cat /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-2023-07-06-22-15-46.conf |
 egrep \"\"id\"|\"enabled\"\" |
 sed 's/^ *//' |
 sed -e 'N;s/\n//' > /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-id-2023-07-06-22-15-46.conf

The problem here is that the \n in the command line is being interpreted as a carriage return, so the actual shell script looks like:

/usr/bin/cat /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-2023-07-06-22-15-46.conf | egrep ""id"|"enabled"" | sed 's/^ *//' | sed -e 'N;s/
//' > /mnt/stl-server-actions/opsgenie/disable/opsgenie-integrations-id-2023-07-06-22-15-46.conf

Look at the end of the first line:

...sed 's/^ *//' | sed -e 'N;s/

That embedded newline results in an unterminated s/// command in your sed expression. You're going to need to rewrite your command to properly escape the \n.

You haven't shown us your actual playbook, so it's hard to suggest an appropriate fix.

But even after fixing that, you're going to have additional problems, since you're apparently trying to read from and write to the same file in your pipeline. That is, you have, effectively:

cat somefile | sed > somefile

That's not going to do anything other than truncate somefile.

larsks
  • 277,717
  • 41
  • 399
  • 399
  • 1
    Sorry I did eliminate some from the command, and the whole command in the playbook was like this: shell: "cat api_file | egrep \"\"id\"|\"enabled\"\" | sed 's/^ *//' | sed 'N;s/[\n]//' . The egrep and the first sed worked as I tested the them earlier, and the results were like what I posted. It was the second sed that caused the error. I tried the suggestion from U880D below and it worked in the playbook. – Philip Shangguan Jul 07 '23 at 17:49
1

Even if I do not understand what you try to achieve finally and why, I understand that you ask

How to remove every second line break?

I want to move the status line to the API code line (pair the every two lines)

An approach could be as shown in the minimal example

---
- hosts: localhost
  become: false
  gather_facts: false

  tasks:

  - shell:
      cmd: |
        cat api_file |  # print file content to STDOUT
        paste -d '' - - # merge lines of files from STDIN
    register: result

  - debug:
      msg: "{{ result.stdout_lines }}"

which will result into the requested output of

TASK [debug] ******************************************************
ok: [localhost] =>
  msg:
  - '"id": "3b96db40-507e-4dd8-b2fd-e0e237fb928f","enabled": true,'
  - '"id": "3c07b7a7-0165-4017-84c6-b7e7383105a4","enabled": true,'
  - '"id": "41b2ed8f-8a21-4b3b-a690-2d7f24e577f6","enabled": true,'
  - '"id": "42767867-0a93-4cc4-93f8-c3893fe63edf","enabled": true,'
  - '"id": "4278314e-71ec-4e80-ae35-1dd67ceef2bd","enabled": false,'
  - '"id": "45c0c8c8-d0b6-4d82-87c3-1c66ec08cf1d","enabled": true,'
  - '"id": "492f0775-f6b8-4cf6-b355-02260513bd5a","enabled": true'

Similar Q&A

Further Documentation

Within the example are used Ansible multiline Shell script, as well Inline comments for Bash to get a better understanding what the commands are for and doing. And mainly man paste.

U880D
  • 8,601
  • 6
  • 24
  • 40