assign variable from file with string literal newline

Question

This question may sound like a duplicate, but I think it's different than the pages that I've found. I'm trying to assign the contents of a text file to a bash variable, but I want the "\n" character to be included as a string rather than actually seeing it on a new line. For example, the contents of the file look something like this:

Here is the content of the text file

There are multiple lines

blah blah blah

I want the variable "text_file" below to be assigned the contents of the file so when I use it in my script it looks like so:

Here is the content of the text file\nThere are multiple lines\nblah blah blah

I'm using this variable in the following script, and I'm getting this error, which I believe is a result of the newline characters in "hello.txt" file that I'm assigning to the variable.

Error parsing parameter '--message': Invalid JSON: Invalid control character u'\n' at:

subject="Test Email Sent Via AWS"
message="here is the message to the user...\n\n"
text_file=`cat hello.txt`
full_message="$message$text_file"

cat <<EOF > generated_message.json
{

   "Subject": {
       "Data": "$subject",
       "Charset": "UTF-8"
   },
   "Body": {
       "Text": {
           "Data": "$full_message",
           "Charset": "UTF-8"
       }
    }
}
EOF
aws ses send-email --profile sendmail --from blah@email.com --destination file://destination.json --message file://generated_message.json

I think I'm missing something basic, but I can't figure it out. Thanks in advance for the help.

I don't think that's necessarily a duplicate question, although it does give me some additional information about what may be causing the issue. So if I'm understanding the responses correctly in that post, you can't have a "\n" character in JSON at all and it needs to be escaped with an additional backslash? If that is the case, does that mean I need to replace all my newline characters in the original "hello.txt" file with "\\n" to solve? — syang, Jul 15 '16 at 05:38

chepner · Accepted Answer · 2016-07-15T12:43:45.003

Don't try to hobble together valid JSON using traditional Unix tools; use a tool designed for JSON, like jq.

subject="Test Email Sent Via AWS"
message="here is the message to the user..."

jq -R --slurp \
   --arg message "$message" \
   --arg subject "$subject" '{
       "Subject": {
           "Data": $subject,
           "Charset": "UTF-8"
       },
       "Body": {
           "Text": {
               "Data": ($message + "\n\n" + @text),
               "Charset": "UTF-8"
           }
       }
   }' < hello.txt > generated_message.json

-R and --slurp ensure that the contents of hello.txt are passed directly to the @text function, which ensures the text is properly quoted as a JSON string. Passing the message and subject as variables, instead of embedding them directly into the filter argument, ensures they are JSON-encoded properly as well.

score 0 · Answer 2 · answered Jul 15 '16 at 05:46

Contents of text_file:

Here is the content of the text file
There are multiple lines
blah blah blah

Expected ouptut

Here is the content of the text file\nThere are multiple lines\nblah blah blah

You might do

declare -a file_as_array
while read line
do
file_as_array+=("${line/%/\\n}")
done<text_file
file_as_text="$(sed 's/\\n /\\n/g' <<<"${file_as_array[@]}")"
unset file_as_array
echo "$file_as_text"

Actual output

Here is the content of the text file\nThere are multiple lines\nblah blah blah\n

What's with the indirection via an array? Just `file_as_text=$(perl -pe 's/\n/\\n/' text_file)` should get you that. — tripleee, Jul 27 '16 at 08:33

assign variable from file with string literal newline

2 Answers2