-1

I have a requirement where I need to replace line feed characters appearing as part of a data field in a CSV files. Fortunately all the unnecessary linefeeds are followed by an '_' character. So I decided to used sed to preprocess the file. The following command in sed works if run it interactively.

sed -i -e ':a;N;$!ba;s/\n_/_/g' file

But in server, the command will get executed with sh -c "<command>"

To test it in local, I ran the same command with sh and it is not working. The command looks as follows.

sh -c "sed -i -e ':a;N;$!ba;s/\n_/_/g' file"

Not sure what I'm missing. Please help.

oguz ismail
  • 1
  • 16
  • 47
  • 69
deena102
  • 41
  • 4
  • 1
    `$` has a special meaning between doublequotes, you're losing `$!` to expansion – oguz ismail May 27 '20 at 13:46
  • probably something not escaped/quoted correctly or different sed running on server? – alecxs May 27 '20 at 13:46
  • 1
    might be interesting to issue `set -vx` prior to running the `sh -c ...` command, objective being to see how the content between the double quotes is being processed; I'm guessing (at a minimum) the `$!ba` is being processed as some sort of variable (one possible solution would be to escape the `$`, eg, `...:a;N;\$!ba;...`) – markp-fuso May 27 '20 at 13:47
  • 1
    Thanks @markp-fuso.. Escaping $ worked. :) – deena102 May 27 '20 at 14:04

1 Answers1

2

Between double-quotes, $! expands to the PID of most recent background command (or the empty string if there is not one). Pass sed script as a positional parameter to sh to avoid dealing with quotation issues/escaping every special character:

sh -c 'sed -i -e "$1" file' _ ':a;N;$!ba;s/\n_/_/g'

See also: Difference between single and double quotes in Bash

oguz ismail
  • 1
  • 16
  • 47
  • 69
  • Thanks @oguz_ismail.. But I don't have control over the quotes in server side. I can only mention the command.. The server automatically adds sh -c to the command with double quotes by default around it. Unfortunately even escaping $ is working only in local and not working in server. – deena102 May 27 '20 at 15:31