Use newline with jq

Question

I've seen a number of posts on this but can't figure out what I need exactly. I've tried -r and argjson among other things.

I need the newlines to remain as \n and not be escaped to \\n.

I'd also like to be able to use ``` for code blocks but it ignores that section of the string.

FALLBACK_MESSAGE="TEST MESSAGE - $HOSTNAME"
MARKDOWN_MESSAGE="TEST MESSAGE - $HOSTNAME \(0x0a) \(\n) Hi <@U12345789>\n```Can we do a\nmultiline code block```"
JSON_STRING=$( jq -nr \
    --arg fallbackMessage "$FALLBACK_MESSAGE" \
    --arg sectionType "section" \
    --arg markdownType "mrkdwn" \
    --arg textMessage "$MARKDOWN_MESSAGE" \
    '{
        text: $fallbackMessage, 
        blocks: [
            {
                type: $sectionType,
                text: {
                    type: $markdownType, 
                    text: $textMessage
                }
            }
        ]
    }')
echo $JSON_STRING

Outputs:

{ "text": "TEST MESSAGE - devDebug", "blocks": [ { "type": "section", "text": { "type": "mrkdwn", "text": "TEST MESSAGE - devDebug \\(0x0a) \\(\\n) Hi <@U12345789>\\n" } } ] }

Put a newline in your string _as a newline literal_, not the two characters backslash and `n`. — Charles Duffy, May 04 '21 at 19:44
That is to say, `newline=$'\n'`, or `printf -v newline '%b' '\n'` will make a shell variable `newline` with an actual newline in it. — Charles Duffy, May 04 '21 at 19:45
BTW, don't use all-caps names -- they're reserved for variables built into (or meaningful to) the shell itself. — Charles Duffy, May 04 '21 at 19:45
...remember, you're telling jq to convert literal data into JSON. The JSON form of the string `\n` is `"\\n"`, whereas the JSON form of a string with only a newline is `"\n"`. So you want the data you're putting _into_ jq to have the literal newline that's ready to be converted, not the two-character `\n` sequence that isn't. — Charles Duffy, May 04 '21 at 19:54
Also, in double quotes, backticks are treated as command substitution syntax, so you need to escape them to stop their contents from being run as code. — Charles Duffy, May 04 '21 at 20:05
Note for zsh users: I'm trying to do the same but zsh behaviors are inconsistent: https://stackoverflow.com/questions/73899110/jq-discrepancy-in-newline-treatment-between-bash-and-zsh-and-zsh-subshell — jakub.g, Sep 29 '22 at 16:52

Charles Duffy · Accepted Answer · 2021-05-04T20:03:59.237

Make sure your shell variables contain actual newlines, not \n sequences.

If you want bash to convert escape sequences in a string into the characters they refer to, printf %b can be used for this purpose.

#!/usr/bin/env bash

fallback_message="TEST MESSAGE - $HOSTNAME"
markdown_message="TEST MESSAGE - $HOSTNAME \(0x0a) \(\n) Hi <@U12345789>\n\`\`\`Can we do a\nmultiline code block\`\`\`"

# create markdown_message_unescaped with an unescaped version of markdown_message
printf -v markdown_message_unescaped %b "$markdown_message"

jq -n \
  --arg textMessage "$markdown_message_unescaped" \
  --arg fallbackMessage "$fallback_message" \
  --arg sectionType section --arg markdownType markdown '
    {
      text: $fallbackMessage, 
      blocks: [
        {
          type: $sectionType,
          text: {
                    type: $markdownType, 
                    text: $textMessage
                }
            }
        ]
    }'

...properly emits as output:

{
  "text": "TEST MESSAGE - YOUR_HOSTNAME",
  "blocks": [
    {
      "type": "section",
      "text": {
        "type": "markdown",
        "text": "TEST MESSAGE - YOUR_HOSTNAME (0x0a) (\n)\nHi <@U12345789>\n```\nCan we do a multiline code block\n```"
      }
    }
  ]
}

This seems to work with `bash` but I'm seeing a weird discrepancy when using `zsh`: https://stackoverflow.com/questions/73899110/jq-discrepancy-in-newline-treatment-between-bash-and-zsh-and-zsh-subshell (same without the `printf %b` actually) — jakub.g, Sep 29 '22 at 16:56

Use newline with jq

1 Answers1

Linked