0

I am trying to package some videos on an ubuntu-ec2 machine using shaka packager and following official tutorial.

So I have a list of multi resolution files i.e. original=x.mp4, converted are x_480p.mp4, x_360p.mp4 and so on. My lowest resolution is 360p. My bash script automatically detects the height and converts lower than that automatically. Using ffmpeg it's done nicely. Now the problem is, I need to automatically package the files in converted directory (all of them) using shaka.

If I run the script in a single line it works.

sudo packager in=dpnd_comp.mp4,stream=video,out=test/video.mp4 in=dpnd_comp.mp4,stream=audio,out=test/audio.mp4

For automatic process I am saving the paths in the vairable inputs. when I run this using variable, it just processes the last video, here 360p only.

This is the part -

# using a for loop here

        inputs="$inputs   in="$output_path"/"$content_id"_"$height"p.mp4,stream=video,output="$packaged_out"/"$content_id"_"$height"p.mp4  "
done


echo "$inputs"

sudo packager "$inputs" 

Note, `echo "$inputs" returns this

in=../bin/converted/0001_720p.mp4,stream=video,output=../bin/packaged/0001_720p.mp4     in=../bin/converted/0001_480p.mp4,stream=video,output=../bin/packaged/0001_480p.mp4     in=../bin/converted/0001_360p.mp4,stream=video,output=../bin/packaged/0001_360p.mp4

Any kind of help would be highly appreciated. If anyone ever worked with shaka and made the process automatic, please help.

Edit: Need to add more arguments after the inputs like this -

sudo packager "$inputs" \
    --enable_widevine_encryption \
    --key_server_url "$key_server" \
    --content_id "$content_id" \
    --signer "$signer_uname" \
    --aes_signing_key "$signing_key" \
    --aes_signing_iv "$signing_iv" \
    --mpd_output "$packaged_out"/"$content_id".mpd \
    --hls_master_playlist_output "$packaged_out"/"$content_id".m3u8"
ashraf minhaj
  • 504
  • 2
  • 14

2 Answers2

1

Your first command (the one you tried manually and that works) pass 2 arguments to "packager" (both starting with in=...). Your second, packager "$input" passes only one argument (containg a space, in=... in=... in a signle string).

So either you just drop the double quotes around $input when calling, in the last line: sudo packager $input. It will work if you are sure that none of your in=... strings ever contain spaces. But that is not advisable. Because if ever one of your file contains a space, such as "vid 720.mp4", then input will look like in=vid 720.mp4,output=packaged/vid 720.mp4 in=other.mp4,output=pack.mp4

And then running with quotes (sudo packager "$input") is wong, because pakager will be run with a single argument in=vid 720.mp4,output=packaged/vid 720.mp4 in=other.mp4,output=pack.mp4 that means nothing coherent to it.

Running without quotes (sudo packager $input) is also wrong, because then packager will be run with 4 arguments, in=vid, 720.mp4,output=packaged/vid, 720.mp4 and int=other.mp4,output=pack.mp4. Which is not what you want neither.

So again, if you are positive that never any space can find its way in your file names, go ahead, run without double quotes around input, and stop reading :-)

Otherwise, I would use arrays.

inputs=()
# using a for loop here

        inputs+=("in="$output_path"/"$content_id"_"$height"p.mp4,stream=video,output="$packaged_out"/"$content_id"_"$height"p.mp4  ")
done


echo "${inputs[@]}"

sudo packager "${inputs[@]}" 

Note1: if you want to see how args ars split, use printf rather that echo. Because you can't see difference between echo "one two" and echo one two. Whereas printf "(%s) " one two and printf "(%s) " "one two" shows what are the args.

Note2: You don't need to end double quotes each time you want to add a variable. Variables are expanded inside double quotes So

inputs=()
# using a for loop here

        inputs+=("in=${output_path}/${content_id}_${height}p.mp4,stream=video,output=${packaged_out}/${content_id}_${height}p.mp4")
done

printf "(%s) " "${inputs[@]}"

sudo packager "${inputs[@]}" 

(Not all ${...} are necessary here. But taking the habit of using ${...} each times avoids problems, for example for your ${height}p.mp4: it avoids p to be taken as part of the variable name)

Addendum: some experiments

To understand what happen here, see

printf "(%s) "  in=1  in=2
#(in=1) (in=2)

input=""
input="$input in=1"
input="$input in=2"
printf "(%s) " "$input"
#( in=1 in=2)

input=""
input="$input in=1"
input="$input in=2"
printf "(%s) " $input
#(in=1) (in=2)
# So, victory? No! see below

input=""
input="$input in=one 1"
input="$input in='two 2'" # in case you expect something in nested quotes
printf "(%s) " "$input" #Nope: ( in=one 1 in='two 2')
printf "(%s) " $input # Even worse: (in=one) (1) (in='two) (2')

# Now arrays
input=()
input+=("in=1")
input+=("in=two 2")
input+=("in='three 3'")
avar="four 4"
input+=("in=${avar}") # Just to check what I say about variable inside double quotes
printf "(%s) " "${input[@]}" # Victory: (in=1) (in=two 2) (in='three 3') (in=four 4)
printf "(%s) " ${input[@]} # Don't forget quotes. (in=1) (in=two) (2) (in='three) (3') (in=four) (4)

Edit

To take into account your edit to your question: it should really change nothing. What is the failure? Are you sure it was not there before, but couldn't see it because of previous problem now solved.

Just to experiment (without your "packager" program, but replacing it, again, with printf)

inputs=()
for i in {1..5}
do
   content="content $i"
   height=$((i*100))
   inputs+=("in=indir/${content}_${height}p.mp4,out=out/${content}_${height}p.mp4")
done

key_server=aServer
content_id=123
signer_uname="John Doe"
packaged_out=/home/me/out
printf '(%s) ' sudo packager "${inputs[@]}" \
    --enable_widevine_encryption \
    --key_server_url "${key_server}" \
    --content_id "${content_id}" \
    --signer "${signer_uname}" \
    --mpd_output "${packaged_out}/${content_id}.mpd" \
    --hls_master_playlist_output "${packaged_out}/${content_id}.m3u8"

Displays

(sudo) (packager) (in=indir/content 1_100p.mp4,out=out/content 1_100p.mp4) (in=indir/content 2_200p.mp4,out=out/content 2_200p.mp4) (in=indir/content 3_300p.mp4,out=out/content 3_300p.mp4) (in=indir/content 4_400p.mp4,out=out/content 4_400p.mp4) (in=indir/content 5_500p.mp4,out=out/content 5_500p.mp4) (--enable_widevine_encryption) (--key_server_url) (aServer) (--content_id) (123) (--signer) (John Doe) (--mpd_output) (/home/me/out/123.mpd) (--hls_master_playlist_output) (/home/me/out/123.m3u8)

Which is exactly what is expected.

chrslg
  • 9,023
  • 5
  • 17
  • 31
  • Thank your very very much, this works half way, if I use more arguments it fails - ``` I need to add more arguments to it and then it fails - ``` sudo packager "${inputs[@]}" \ --enable_widevine_encryption \ --key_server_url "$key_server" \ --content_id "$content_id" \ --signer "$signer_uname" \ --aes_signing_key "$signing_key" \ --aes_signing_iv "$signing_iv" \ --mpd_output "$packaged_out"/"$content_id".mpd \ --hls_master_playlist_output "$packaged_out"/"$content_id".m3u8" ``` (added on post now) – ashraf minhaj Nov 04 '22 at 21:17
  • You can't use ``` in comments (on SO there is 1 markup language for posts, another for comments, another for tag wiki, ...). So not very clear what you did. But why are you using double quotes inside what you add? Why not simply `inputs+=("in=${output_path}/${content_id}_${height}p.mp4,stream=video,output=${packaged_out}/${content_id}_${height}p.mp4")` – chrslg Nov 04 '22 at 21:22
  • In your edit, you have an stray double quote at the end of your new command. – chrslg Nov 04 '22 at 21:32
  • Faced some issues, to be safe I am using quote. Apologies. Edited at the end of the question too. So, my end goal is to add inputs in a command like this one - `sudo packager "$inputs" \ --enable_widevine_encryption \ --key_server_url "$key_server" \ --content_id "$content_id" \ --signer "$signer_uname" \ --aes_signing_key "$signing_key" \ --aes_signing_iv "$signing_iv" \ --mpd_output "$packaged_out"/"$content_id".mpd \ --hls_master_playlist_output "$packaged_out"/"$content_id".m3u8"` But this causes error with your array solution. any idea, sir? – ashraf minhaj Nov 04 '22 at 21:39
  • No the last one does not work, now nothing gets processed. If you want to download and check with shaka - `sudo wget https://github.com/shaka-project/shaka-packager/releases/download/v2.6.1/packager-linux-x64 -O /bin/packager sudo chmod +x /bin/packager` – ashraf minhaj Nov 04 '22 at 22:26
  • Thanks for all your help, but this did not help with shaka packager. I made it work by running this using python and subprocess. – ashraf minhaj Nov 05 '22 at 15:17
0

Ok I solved this problem (workaround) by running bash commands from python.

cmd_prefix = 'sudo packager..'
cmd_postfix = '--mpd_output "${packaged_out}/${content_id}.mpd" --hls_master_playlist_output "${packaged_out}/${content_id}.m3u8"

for inp in inputs
    cmd_prefix += inp

drm_command = cmd_perfix + cmd_postfix


drm_processor_response = Popen(f"({drm_command})", stderr=PIPE, stdout=PIPE, shell=True)
output, errors = drm_processor_response.communicate()
log_update([drm_processor_response.returncode, errors.decode("utf-8"), output.decode("utf-8")])

This worked and with python I have more control.

ashraf minhaj
  • 504
  • 2
  • 14