How to concatenate parts of a commandline

Question

I've got a problem : an existing tool is calling ffmpeg -i path to file -f wav -ar 16000 ... but it fails to escape the path. (It should be ffmpeg -i path\ to\ file -f ... It's not trivial to fix the tool or the path it produces.

So my idea was to create a small script named ffmpeg, put it on the $PATH before the real ffmpeg, and forward the arguments after fixing them. "Fixing them" would mean concatenating ${2} to ${n-1} where ${n} is the -f argument which follows the path.

But how do I write this loop ? This answer loops over all arguments, and I could probably detect the -f parameter in that loop, but how do I then reassemble the command line for the real ffmpeg process?

This is [Bash FAQ 50](http://mywiki.wooledge.org/BashFAQ/050). Pass the arguments as an array. — miken32, Mar 10 '16 at 17:42
The problem you are going to face here is that once the quoting has been messed up you *can't* reliably restore it. You can guess that each "word" had one space between them but they could have had more. The original word could also have had globbing characters which got expanded and dropped out and you can't reconstruct that. Fixing the original tool is the only safe and reliable solution. — Etan Reisner, Mar 10 '16 at 17:50
As well as what Etan said, if there are multiple consecutive blanks (or white space generally) in the name — let alone newlines — you cannot identify the original name reliably. If you can't get the tool fixed, your only choice is to ensure you never provide a 'path to file' containing spaces, etc. Or file a bug report, or switch to an alternative tool the works correctly. — Jonathan Leffler, Mar 10 '16 at 17:59
If `path to file` is always between `-i` and `-f` quoting or escaping it should be possible if you can pass it through awk, etc. within a function I would imagine. — l'L'l, Mar 10 '16 at 18:11
@EtanReisner: I just realized that I could insert `*` and then count the number of matches. I'd be screwed if I have two matches, but it should be pretty reliable if it matches just one. — MSalters, Mar 10 '16 at 23:03
Insert what? Where? And do what? If you can control the filename (but not that it is used unquoted) you could write your own ffmpeg and mangle the filename on input in a reversible way in your `ffmpeg` wrapper script. But that's fairly ugly. Fixing the tool is *inifinitely* better. — Etan Reisner, Mar 10 '16 at 23:59
@EtanReisner: If I get `\foo bar.txt` as 2 independent arguments, I can insert `*` between the two to create `\foo*bar.txt` which will match any number of whitespace characters. — MSalters, Mar 11 '16 at 09:06
You insert that where? In your wrapper script? Between every "word" in the argument you assemble? Yes, I suppose you could do that to try to glob the original filename back from the pieces, that's true, but yes that will work badly with multiple matching files and is still an awful hack. — Etan Reisner, Mar 11 '16 at 14:15

that other guy · Answer 1 · 2016-03-11T00:25:22.813

1

You really need to report that as a bug, it's an embarrassing beginner's mistake and not hard to fix.

In any case, here's a kludge:

#!/bin/bash
array=()

start="-i"
stop="-f"

fileArg=0
for param
do
  if [[ $fileArg = 1 ]]
  then
    if [[ $param = $stop ]]
    then
      fileArg=0
      array+=("${file% }" "$param")
    fi
    file+="$param "
  else
    [[ $param = $start ]] && fileArg=1
    array+=("$param")
    file=""
  fi
done

printf "Your broken tool provided these argument:\n"
printf "%q " "$@"
printf "\n\n"
printf "Tried to salvage it by turning it into:\n"
printf "%q " "${array[@]}"
printf "\n"
/path/to/real/ffmpeg "${array[@]}"

Here's example output:

$ ./ffmpeg  -i path to file -f wav -ar 16000
Your broken tool provided these argument:
-i path to file -f wav -ar 16000

Tried to salvage it by turning it into:
-i path\ to\ file -f wav -ar 16000

This does not work for all cases -- there is no good way to reassemble a filename that has been split up into shell words. This method breaks for multiple spaces in a row, globs, brace expansions, quotes, dollars, backticks and others. The tool will most likely also be unable to even call ffmpeg (or this replacement) if there are mismatched quotes or parentheses in the filename.

As a nice feature, if the tool is ever fixed, this script will be a no-op (provided the filename is still between -i and -f)

edited Mar 11 '16 at 00:25

answered Mar 10 '16 at 18:17

that other guy

116,971
11
170
194

2

Running the correct `ffmpeg` is not as simple as calling `ffmpeg` if the script is called `ffmpeg` itself (you'll need to find it in the `$PATH` yourself, etc. and then use the full path). – Etan Reisner Mar 11 '16 at 00:01
@EtanReisner: Or just hardcode the real location in the script. Don't over-engineer hacks. – MSalters Mar 11 '16 at 09:38
@MSalters Assuming you can assert a single "real" location. That's very often not the case, but yes. A default path could certainly be tried first before falling back to a `$PATH` search, etc. – Etan Reisner Mar 11 '16 at 14:13
@EtanReisner: I can, there's exactly one server with this problem and it has ffmpeg installed solely for this purpose. I just run `which ffmpeg` and `echo $PATH`. – MSalters Mar 11 '16 at 15:45
Sure, in any specific given scenario you can hard-code a path. But answers are supposed to be generic in general. – Etan Reisner Mar 11 '16 at 15:51

l'L'l · Answer 2 · 2016-03-11T02:53:12.507

If you're able to pass your unquoted command to a function using awk, sed, or something similar it could be possible to fix the quoting or escape it if the path is always within the same two options:

#!/bin/env bash
# quote_path.sh
# wraps quotes or escapes an unquoted path in a command

# takes command with unquoted path and wraps in quotes 
quoted_cmd()
{
    quoted=$(echo $cmd | awk '{ sub(/ -f/,"\" -f"); sub(/-i /,"-i \""); print }')
    # eval $quoted
    escape_cmd # comment out if using eval $quoted
}

# takes quoted_cmd() and escapes it (as an alternative to wrapping in quotes)
escape_cmd() 
{   
    escape=$(echo $quoted | awk 'BEGIN { FS="-i \"|\" -f" } ; { print $2 }')
    concat=$(echo $escape | sed 's! !\\\\ !g')
    cmdesc=$(echo $quoted | awk '{ sub(/".*" -f/," -f"); $2="-i '"${concat}"'"; print }')
    eval $cmdesc
}

cmd="$@"
quoted_cmd

So with this you could pass in your command as argument (as a string or variable):

$ ./quote_path.sh ffmpeg -i /path to file /path to file -f wav -ar 16000 "foobar" ...

As Etan Reisner and that other guy mentioned, it should likely be reported as a bug, in the meantime it is what it is... give it a try!

Update: I was able to revise the function a bit more so that it is less likely to fail in the event another part of the command has an argument with quotes, contains additional slashes, etc. The command shouldn't require being quoted when passing it as an argument (optional) and will produce a quoted or an escaped path in the command. After invoking the function the entire path was returned, concatenated with the rest of the command.

Result:

ffmpeg -i "/path to file /path to file" -f wav -ar 16000 ... # quoted_cmd()
ffmpeg -i /path\ to\ file\ /path\ to\ file -f wav -ar 16000 ... # escape_cmd()
... 
ffmpeg version 2.5.3-tessus Copyright (c) 2000-2015 the FFmpeg developers
...

How to concatenate parts of a commandline

2 Answers2