0

I'm creating a tool to parse an input file to re-construct a mailx command (server that creates the data for mailx cannot send emails, so I need to store data into a file so another server can rebuild the command and send it). I could output the whole command to a file and execute the file on the other server, but that's hardly secure / safe.... anyone could intercept the file and insert malicious stuff that would be run as root - this parsing tool is checking every minute for any files to parse and email using a systemd timer and service.

I have created the file, using 'markers / separators' with this format:

-------MESSAGE START-------
Email Body Text
Goes Here
-------MESSAGE END-------
-------SUBJECT START-------
Email Subject
-------SUBJECT END-------
-------ATTACHEMENT START-------
path to file to attach if supplied
-------ATTACHEMENT END-------
-------S OPTS START-------
list of mailx '-S' options eg from=EMAILNAME <email@b.c> or sendwait etc each one on a new line
-------S OPTS END-------
-------EMAIL LIST START-------
string of recipient emails comma separated eg. email1,email2,email3 etc..
-------EMAIL LIST END-------

And I have a program to parse this file and rebuild, and run the mailx command:

#!/bin/bash

## Using systemd logging to journal for this as its now being called as part of a service
## See: https://serverfault.com/questions/573946/how-can-i-send-a-message-to-the-systemd-journal-froma-the-command-line (kkm answer)

start_time="$(date +[%c])"
exec 4>&2 2> >(while read -r REPLY; do printf >&4 '<3>%s\n' "$REPLY"; done)
echo >&4 "<5>$start_time  -- Started gpfs_flag_email.sh"

trap_exit(){
    exec >2&
}
trap 'trap_exit' EXIT 

email_flag_path="<PATH TO LOCATION>/email_flags/"
mailx_message_start="-------MESSAGE START-------"
mailx_message_end="-------MESSAGE END-------"
mailx_subject_start="-------SUBJECT START-------"
mailx_subject_end="-------SUBJECT END-------"
mailx_attachement_start="-------ATTACHEMENT START-------"
mailx_attachement_end="-------ATTACHEMENT END-------"
mailx_s_opts_start="-------S OPTS START-------"
mailx_s_opts_end="-------S OPTS END-------"
mailx_to_email_start="-------EMAIL LIST START-------"
mailx_to_email_end="-------EMAIL LIST END-------"
no_attachment=false
no_additional_opts=false
additional_args_switch="-S "


num_files_in_flag_path="$(find $email_flag_path -type f | wc -l)"
if [[ $num_files_in_flag_path -gt 0 ]]; then
    for file in $email_flag_path*; do
        email_message="$(awk "/$mailx_message_start/,/$mailx_message_end/" $file | egrep -v -- "$mailx_message_start|$mailx_message_end")"
        email_subject="$(awk "/$mailx_subject_start/,/$mailx_subject_end/" $file | egrep -v -- "$mailx_subject_start|$mailx_subject_end")"
        email_attachment="$(awk "/$mailx_attachement_start/,/$mailx_attachement_end/" $file | egrep -v -- "$mailx_attachement_start|$mailx_attachement_end")"
        email_additional_opts="$(awk "/$mailx_s_opts_start/,/$mailx_s_opts_end/" $file | egrep -v -- "$mailx_s_opts_start|$mailx_s_opts_end")"
        email_addresses="$(awk "/$mailx_to_email_start/,/$mailx_to_email_end/" $file | egrep -v -- "$mailx_to_email_start|$mailx_to_email_end" | tr -d '\n')"
        if [[ -z "$email_message" || -z "$email_subject" || -z "$email_addresses" ]]; then
            echo >&2 "MISSING DETAILS IN INPUT FILE $file.... Exiting With Error"
            exit 1
        fi

        if [[ -z "$email_attachment" ]]; then
            no_attachment=true
        fi
        if [[ -z "$email_additional_opts" ]]; then
            no_additional_opts=true
        else
            additional_opts_string=""
            while read -r line; do 
                if [[ ! $line =~ [^[:space:]] ]]; then
                    continue
                else
                    additional_opts_string="$additional_opts_string \"${additional_args_switch} '$line'\""
                fi
            done <<<"$(echo "$email_additional_opts")"
            additional_opts_string="$(echo ${additional_opts_string:1} | tr -d '\n')"
        fi
        if [[ $no_attachment = true ]]; then
            if [[ $no_additional_opts = true ]]; then
                echo "$email_message" | mailx -s "$email_subject" $email_addresses
            else
                echo "$email_message" | mailx -s "$email_subject" $additional_opts_string $email_addresses
            fi
        else
            if [[ $no_additional_opts = true ]]; then
                echo "$email_message" | mailx -s "$email_subject" -a $email_attachment $email_addresses
            else
                echo "$email_message" | mailx -s "$email_subject" -a $email_attachment $additional_opts_string $email_addresses
            fi
        fi          
    done
fi
find $email_flag_path -type f -delete
exit 0

There is however an issue with the above that I just can work out..... the -S opts completely screw up the email headers and I end up with emails being sent to the wrong people (I have set a reply-to and from options, but the email header is jumbled and the reply-to email ends up in the to: field) like this:

To: Me <a@b.com>, sendwait@a.lan , -S@a.lan <-s@a.lan>, another-email <another@b.com> 

All I'm trying to do is rebuild the command as if I'd typed it in the CLI:

echo "EMAIL BODY MESSAGE" | mailx -s "EMAIL SUBJECT" -S "from=EMAILNAME <email@b.c>" -S "replyto=EMAILNAME <email@b.c>" -S sendwait my.email@b.com

I've tried quoting in ' ' and " " quoting the other mailx parameters around it etc etc... I have written other tools that pass variables as input arguments so I just cannot understand how I'm screwing this up.....

Any help would be appreciated...

EDIT

Thanks to Gordon Davisson's really helpful comments I was able to not only fix it but understand the fix as well using an array and appropriately quoting the variables... the tip about using printf was really really helpful in helping me understand what I was doing wrong and how to correct it :P

declare -a mailx_args_array

...
num_files_in_flag_path="$(find $email_flag_path -type f | wc -l)"
if [[ $num_files_in_flag_path -gt 0 ]]; then
    for file in $email_flag_path*; do
        ....
        mailx_args_array+=( -s "$email_subject" )
                
        if [[ ! -z "$email_attachment" ]]; then
            mailx_args_array+=( -a "$email_attachment" )
        fi
        if [[ ! -z "$email_additional_s_opts" ]]; then
            while read -r s_opt_line; do
                mailx_args_array+=( -S "$s_opt_line" )
            done < <(echo "$email_additional_s_opts")
        fi
        mailx_args_array+=( "$email_addresses" )
        
        echo "$email_message" | mailx "${mailx_args_array[@]}"
    done
fi
Owen Morgan
  • 301
  • 3
  • 9
  • 1
    Storing multiple complex arguments in a regular variable doesn't work; either use an array, or keep the different arguments separate until you actually use them. See ["Why does shell ignore quoting characters in arguments passed to it through variables?"](https://stackoverflow.com/questions/12136948/why-does-shell-ignore-quoting-characters-in-arguments-passed-to-it-through-varia) [BashFAQ #50: "I'm trying to put a command in a variable, but the complex cases always fail!"](http://mywiki.wooledge.org/BashFAQ/050) And avoid `eval` -- it's a massive bug magnet. – Gordon Davisson Apr 25 '21 at 19:12
  • @cyrus one reason I didn't minify the code more was I was unsure if the issue was also related to the way I was parsing the file and extracting the variables in the first place. Whilst I was sure, I couldn't 100% rule it out.. I had cut a lot of code out that was unrelated to the question, but agree maybe I could have done a tad more.. – Owen Morgan Apr 26 '21 at 11:47
  • @Gordon Davisson. Thanks :) Ironically I did originally write code to read the lines into an array, but then wasn't sure if I could add them to the command as your link shows and re-wrote it into a string! So it seems I inadvertently created the issue! I will look at this problem today and see if that sorts it!! – Owen Morgan Apr 26 '21 at 11:49
  • @GordonDavisson I tried putting all args into a bash array (easily edited the above program to add `arr+=(-S \"$variable\")` (I'm manually adding " " after to -S to match when I wrote them out manually originally) but now heirloom mailx (12.5) isn't setting the headers to the values they need to be.... frustratingly, when I echo the string `echo mailx "${arr[@]}"` and copy and paste the output into CLI it works..... I feel like I'm going crazy and am being tooooootally dumb and noob. – Owen Morgan Apr 26 '21 at 19:43
  • 1
    @OwenMorgan Don't escape the double-quotes -- that tells the shell to treat them as part of the data, rather than shell syntax. Also, don't use `echo` to view commands, since it shows the command *after* shell parsing (including quote and escape processing *and removal*), which is not at all the same as what would be executed without the `echo`. Try `printf "%q " mailx "${arr[@]}"; printf '\n'` to get a better idea what'll be executed. – Gordon Davisson Apr 26 '21 at 20:10
  • @GordonDavisson thanks for the tip about using printf in this instance! Really useful tip! Sorry for confusion but I did intend to escape the " " so it would match me doing it manually so I hoped it would show as ` -S "replyto=Email " ` the bit after replyto includes spaces which I didnt want it to split. – Owen Morgan Apr 26 '21 at 20:31
  • 1
    @OwenMorgan One way to clarify what's going on is to distinguish *syntactic* quotes and escapes (which are part of the shell syntax, and do things like protect whitespace in arguments) from *literal* quotes and escapes (which are just part of the data being passed, and don't have syntactic function). Here (as in most situations) you want your quotes to be *syntactic*, not literal. Quotes stored in variables (or arrays) are literal, not syntactic. Similarly, `echo` prints only literal quotes (syntactic ones are removed before being passed to `echo`). So if `echo` prints it, it's wrong. – Gordon Davisson Apr 26 '21 at 20:57
  • @GordonDavisson Thanks I was able to work out the issues and fix! I'll update post. It was me getting confused and trying to the command that was being passed programmatically LOOK like the command being type by me manually. Once I saw the output of the `printf` I totally saw what was happening and fixed it. As you no doubt suspected it was to do with splitting and inappropriate use of escaping and not using " " to stop string splitting etc.. – Owen Morgan Apr 27 '21 at 20:49

0 Answers0