Bash script call to imagemagick-convert fails, however syntax works when pasted to console

Question

Good test content is always hard to come by, so I often have to make my own.

I'm working on an issue where closed-caption/sub-titles are not displaying in alignment to spoken language. So on back-and-forth scenes the sub-titles appear to be in obvious mis-alignment.

So my plan was to use a video of a millisecond "accurate" counter and then display timed-text images (PNG) over the video through the application of which I don't have source for. Should my test prove that there are timing issues then I'll be able to push the vendor for source and/or have them address their timing issue(s).

So with a millisecond "accurate" counter movie in the background I intend to display imagemagick-convert created PNGs which will have sufficient embedded text to allow a high-speed camera to then show any mis-alignments during decode/rendering from the timed-text source file (Scenartist DVD SST format).

The issue I've run into is that my call to 'convert' works from the console, but doesn't work from my bash script. I've placed all necessary break-characters around single and double quotes, and have also tried passing the arguments via 'xargs' and achieved the same failure. it seems that bash's interpretation of the string gets construed before its passed to 'convert'. With 'set -x' enabled at the top of the script I see the string that's passed, and interestingly if that string is pasted to the console it works. Only it doesn't work directly from the script.

Please advise if my approach is incorrect and/or if I'm missing a parameter/option to make this work.

Here are a few rows of the SST file for reference:

1613    02:29:08:20 02:29:09:14 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX1595.png
1614    02:29:56:10 02:29:57:15 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX1596.png
1615    02:29:57:20 02:29:59:05 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX1597.png

Format: Enumeration, display-start time hh:mm:ss:frame display-stop time hh:mm:ss:frame, and image file name.

Note: I manually remove the SST file header and leave only the timed-text information for now, for those that are familiar with the format.

Bash-script:

#! /bin/sh
set -x

echo "SST File name passed in <$1>"  

COUNT=0
GFX_FILENAME="test"

while IFS= read -r LINE; do
    COUNT=`expr $COUNT + 1`; # Debug not needed
    echo "Converting entry <$COUNT>... " # Debug not needed

    echo "Line read is <$LINE>" # Debug not needed

    GFX_FILENAME=$(printf "%s" "$LINE" | cut -f4 -d'    ') # works
    GFX_INLINE_TEXT=$(printf "%s" "$LINE" | tr \\t ' ') # works

    GFX_FILENAME="gfx_filename_here.png" # debug, put in place to isolate any issue introduced from parsed value
    GFX_INLINE_TEXT="gfx text here" # debug, put in place to isolate any issue introduced from parsed value

    echo "Filename parsed from line is <$GFX_FILENAME>" # debug to check value before calling convert

    # Method 1, call convert directly:    
    convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw \"text 20,55 \'$GFX_INLINE_TEXT\'\" $GFX_FILENAME

    # Method 2, echo arguments to xargs then call convert (same result) commented out for now.  Note:  xargs hides 'convert's error output.
#   echo -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw \"text 20,55 \'$GFX_INLINE_TEXT\'\" $GFX_FILENAME | xargs

    echo "Converted an image..." # debug, when i find this in console output from script execution I know what's above (from run with set -x enabled) is what I can paste to the console to double-check my syntax

done < $1 # method used to pass file to while loop

echo "Done!!!" # not true, since it doesn't work

Output:

./sst_to_png.sh NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_v05.sst
+ echo File name passed in is <NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_v05.sst>
File name passed in is <NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_v05.sst>
+ COUNT=0
+ GFX_FILENAME=test
+ IFS= read -r LINE
+ expr 0 + 1
+ COUNT=1
+ echo Converting entry <1>... 
Converting entry <1>... 
> echo Line read is <0003   01:01:11:22 01:01:14:25 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
>ine read is <0003  01:01:11:22 01:01:14:25 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
+ printf %s 0003    01:01:11:22 01:01:14:25 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
+ cut -f4 -d    
+ GFX_FILENAME=NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
+ printf %s 0003    01:01:11:22 01:01:14:25 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
+ tr \t  
+ GFX_INLINE_TEXT=0003 01:01:11:22 01:01:14:25 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0000.png
+ GFX_FILENAME=gfx_filename_here.png
+ GFX_INLINE_TEXT=gfx text here
+ echo Filename parsed from line is <gfx_filename_here.png>
Filename parsed from line is <gfx_filename_here.png>
+ convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 'gfx text here'" gfx_filename_here.png
convert.im6: non-conforming drawing primitive definition `text' @ error/draw.c/DrawImage/3158.
convert.im6: unable to open image `20,55': No such file or directory @ error/blob.c/OpenBlob/2638.
convert.im6: no decode delegate for this image format `20,55' @ error/constitute.c/ReadImage/544.
convert.im6: unable to open image `'gfx': No such file or directory @ error/blob.c/OpenBlob/2638.
convert.im6: no decode delegate for this image format `'gfx' @ error/constitute.c/ReadImage/544.
convert.im6: unable to open image `text': No such file or directory @ error/blob.c/OpenBlob/2638.
convert.im6: no decode delegate for this image format `text' @ error/constitute.c/ReadImage/544.
convert.im6: unable to open image `here'"': No such file or directory @ error/blob.c/OpenBlob/2638.
convert.im6: no decode delegate for this image format `here'"' @ error/constitute.c/ReadImage/544.
convert.im6: non-conforming drawing primitive definition `text' @ error/draw.c/DrawImage/3158.
+ echo Converted an image...
Converted an image...

Then I paste:

convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 'gfx text here'" gfx_filename_here.png

To the console, copied from the script's output and it works.

Thanks in advance, hopefully the 'convert' errors mean something to someone. Its odd to me that the verbose output, with set -x enabled at top of script, produces something that works when pasted to the console but not when executed directly from script.

While very interesting, that's a lot to digest. Did you try making a standalone script with just one `convert` cmd? Also, are you using an uptodate `bash` and `convert`? (Maybe you want to edit Q and include the small test case in AND include info on your versions). Finally, it seems like it should work. Weird. Good luck. — shellter, Dec 04 '15 at 23:23
Separate Q, what is the source of the files that you're having sync troubles with? I never expect `.wtv` files to be perfect, as often the sync on a live viewing of a broadcast are out-of sync. (Sometimes everything is fine). Good luck again! ;-) — shellter, Dec 04 '15 at 23:25
It would seem to me that when you add the comment to the console manually, you're actually passing a string. However, in the script, bash interprets the line as several strings separated by spaces, the first being `"text`, the next `20,55` etc. Have you tried using two unescaped strings in the script, with the bash variable in between them, i.e. `convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 '"$GFX_INLINE_TEXT"'" $GFX_FILENAME` ? — Andras Deak -- Слава Україні, Dec 04 '15 at 23:40
Ubuntu 14.04 LTS GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc. — TheHairyOne, Dec 04 '15 at 23:41
Or better yet, simply `convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 '$GFX_INLINE_TEXT'" $GFX_FILENAME` ? — Andras Deak -- Слава Україні, Dec 04 '15 at 23:42
convert --version Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC Features: OpenMP — TheHairyOne, Dec 04 '15 at 23:43
Work-around: I put an echo in front of convert, and then redirected that output to a file. Then sh -xf that file and it works. I'll add this as a solution if I can't find a better one. — TheHairyOne, Dec 04 '15 at 23:44
Andras' agreed. There's something weird with argv here. I'm looking into either a.) printing the parameter list within a bash sub-function or b.) writing a simple c-program to print the argv list. Unfortunately I don't see a way to have convert print verbosely the entire argv list un-parsed so we can know what bash is passing to it. I'm getting thirsty though, so I might have to go get a pitcher and look into this further on Monday :) — TheHairyOne, Dec 04 '15 at 23:48
OK then. You should also try constructing a single string variable which you can pass to the `-draw` switch of `convert`, should be the least dirty. Cheers then;) — Andras Deak -- Слава Україні, Dec 04 '15 at 23:50
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/97029/discussion-between-thehairyone-and-andras-deak). — TheHairyOne, Dec 04 '15 at 23:53

score 1 · Answer 1 · answered Dec 05 '15 at 02:42

1

The correct line in a script is simply:

convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 '$GFX_INLINE_TEXT'" "$GFX_FILENAME"

I added quotes around $GFX_FILENAME in case the filename includes wierd characters.

You don't escape the quotes in the console command line. Why would you need to do so in a script?

answered Dec 05 '15 at 02:42

rici

234,347
28
237
341

Did you try it? This syntax results in dropping the quotes that convert requires for the draw field around: "text vs. text <-- double quote missing. With: convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw "text 20,55 '$GFX_INLINE_TEXT'" "$GFX_FILENAME" You'll get: convert -size 720x480 xc:transparent -font Palatino-Bold -pointsize 72 -fill black -draw text 20,55 '0004 01:01:15 NightAtTheMuseum_Th_16x9_EnS_DYN_B5-01608_FOX0001.png08_FOX0001.png error/constitute.c/WriteImage/1209. image format – TheHairyOne Dec 08 '15 at 16:49
Since this does work with "clean" variables, no trailing \n\t\r I'm working on trying to "clean" the variables before passing them in. Playing with tr now to attempt to clean the variables as per: http://stackoverflow.com/questions/19345872/dirty-variable-remove-carriage-return – TheHairyOne Dec 08 '15 at 16:54
@thehairyone: i did try it and it worked as expected. You are apparently not using bash, so i also tried it with dash and it worked fine there, too. – rici Dec 08 '15 at 17:09
Confirmed, my variable was 'dirty'. I added the breaks in an attempt to fix what I saw as a result of the variable being dirty, since it was causing the quotes (single and double) to disappear. – TheHairyOne Dec 08 '15 at 18:24
@thehairyone: it is important to understand the difference between the *syntax* for writing strings and the content of the string. In `utility "an argument" `, the quotes are bash syntax and `utility` never sees them. In `utility \"an argument\"`, the backslashes are syntax, but they make the quotes regular characters and `utility` does see them. It also sees two arguments, unlike the first example. See also http://stackoverflow.com/a/28653412/1566221 – rici Dec 08 '15 at 18:53

score 0 · Answer 2 · edited May 23 '17 at 11:59

Root cause answer:

Needed to clean-up dirty variables before attempting to use them by convert. All attempts at using break characters on quotes were futile. The trailing \r on the variables made the quotes disappear. The breaks put them back but caused convert to report numerous errors with the syntax. Ultimately "clean" variables are needed when passing variables as arguments, especially when there are other subsequent arguments in the string.

I tried the syntax found in:

How to remove carriage return from a string in Bash

But I couldn't get :

echo "${GFX_INLINE_TEXT//[$'\t\r\n ']}"

./sst_to_png.sh: 36: ./sst_to_png.sh: Bad substitution

to work for me.

So I went with:

GFX_INLINE_TEXT=$(printf "%s" "$LINE" | tr \\t ' ' | tr -d '\n' | tr -d '\r')

This allowed me to swap tabs for spaces, and then remove trailing \r (return) instances from the variable before passing it in as an argument to convert.

If `convert` doesn't like tabs and newlines, then you shouldn't put them in your string, but there is no general principle about "clean" and "dirty" variables. — rici, Dec 08 '15 at 18:56

Bash script call to imagemagick-convert fails, however syntax works when pasted to console

2 Answers2