2

Let's say I have a file.txt file like this:

some words
from here
blah blah blah
that begins
this is this

to here
other content

and another file called *config.conf" like this:

name1:value1
name2:value2    
expr:sed -re "s/this/that/g" -ne "/from here/,/to here/ p"
name3:value3
name4:value4

In my script.sh I need to get the whole sed command that is written after "expr:" in the config.conf and execute it in a pipe like this :

#!/bin/bash

pipecommand=$(cat info | grep -e "^expr:" | sed -re "s/^expr://g")
cat file.txt | $pipecommand > output.file

but I get this error:

sed: -e expression #1, char 1: unknown command: `"'

I've read about many similar questions here and the solution was using an array like this:

pipecommand=($(cat info | grep -e "^expr:" | sed -re "s/^expr://g"))
cat file.txt | ${pipecommand[@]} > output.file

Unfortunately this works only with less complex commands and only if I assign the "sed...blah blah blah" command directly to the variable, without reading it from a file.

Do some of you know a working solution to this?

P.S.: I can change both the script.sh and the config.conf files.

mugnozzo
  • 210
  • 1
  • 15

3 Answers3

3

Interpreting this as a question about how to apply the advice from Reading quoted/escaped arguments correctly from a string to your use case:

#!/usr/bin/env bash

# the sed expression here is modified to work with BSD sed, not only GNU sed
pipecommand=$(sed -ne 's/^expr://p' <info)

array=( )
while IFS= read -r -d ''; do
  array+=( "$REPLY" )
done < <(xargs printf '%s\0' <<<"$pipecommand")

<file.txt "${array[@]}" > output.file

This safer than eval, insofar as the words inside your expr: can be treated only as literal arguments, and can't be parsed as redirections, substitutions, parameter expansions, or other shell syntax. Of course, an expr of sh -c '...' can be used to enable shell syntax inside the ... section: Constraining the command (the first element of the array) is necessary if you genuinely wanted to sandbox, control or constrain the invoked command.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • The `sed` command seems excessively complicated. What is wrong with `sed -n 's/^expr://p' info`? – pjh Aug 23 '18 at 18:15
  • @pjh, that's indeed much better -- the idioms in my head are GNUisms, and I ended up with that overcomplicated version when trying to build something that would work on BSD. – Charles Duffy Aug 23 '18 at 18:17
  • Note that `bash` and `xargs` don't handle quotes in the same way, so this solution fails in some situations where an `eval`-based solution may succeed. Try, for example, a configuration file containing `expr:sed "s/\"/'/g"` (which tries to replace double quotes with single quotes). – pjh Aug 23 '18 at 19:09
2

Sadly, you need eval.

pipecommand=$(grep "^expr:" info | cut -d: -f2-)
eval "$pipecommand" <file.txt > output.file

Try to avoid eval.

tripleee
  • 175,061
  • 34
  • 275
  • 318
KamilCuk
  • 120,984
  • 8
  • 59
  • 111
  • 3
    ... and `cat` on a single file, and `grep` when you are using `sed` anyway. These are only [useless.](http://www.iki.fi/era/unix/award.html) though, not evil like `eval`. – tripleee Aug 23 '18 at 14:20
  • @Kamil Cuk: I'd like to avoid eval but at the moment your solution is the only one I tried that works... – mugnozzo Aug 23 '18 at 14:33
  • You don't "need" `eval` (see other solutions), but it provides a simple and effective solution. Given that the program is intended to run arbitrary code read from a file, using `eval` doesn't make the (very bad) situation much worse than it is already. – pjh Aug 23 '18 at 18:25
2

Turn your config file into a plugin with a well-defined interface. Here, your script expects a function named sed_wrapper, so you provide a definition with that name in your "config file" (renamed to lib.sh here).

# This is lib.sh
sed_wrapper () {
    sed -re "s/this/that/g" -ne "/from here/,/to here/ p"
}

Then, in your script, call the named function.

. lib.sh

sed_wrapper < file.txt > output.file
chepner
  • 497,756
  • 71
  • 530
  • 681
  • Thanks man! At the end I needed to completely change everything due to client's needs, but I'm using this kind of approach in the new version. – mugnozzo Aug 24 '18 at 10:26