0

I just discovered mapfile in bash when shellcheck recommended it to my code to capture two shell parameters from a space-delimited string of two words. But, mapfile does not seem to function consistently. For example, this works as expected:

mapfile arr < myfile

The array arr is populated with entries corresponding to the lines of myfile. However, this does not work:

echo -e "hello\nworld" | mapfile arr

The array arr is not populated at all. And, this doesn't work either:

echo "hello world" | mapfile -d ' ' arr

I am not sure why it would make a difference where the standard input for the command comes from. I didn't know it would be possible to distinguish what the input came from, a file or a pipeline.

Any clues?

Note to moderator: It was suggested my question was a duplicate to Read values into a shell variable from a pipe . I do not agree. Nowhere is mapfile mentioned in that question, nor was there any other useful Q/A found in a SO search. In addition, that referenced question did not deal with shell parameter assignments. Therefore, this question and answers are valuable.

Kevin Buchs
  • 2,520
  • 4
  • 36
  • 55
  • This is [BashFAQ #24](https://mywiki.wooledge.org/BashFAQ/024) (`read` vs `readarray` is immaterial; it's the same limitation). – Charles Duffy Mar 07 '22 at 02:43
  • (to be clear, `mapfile` works the same way regardless, but using the pipe operator sets up a temporary scope, such that variables set in that scope are lost before the next command starts). – Charles Duffy Mar 07 '22 at 02:48
  • 1
    Note, I strongly disagree with moderator's suggestion this question is a duplicate to https://stackoverflow.com/questions/2746553/read-values-into-a-shell-variable-from-a-pipe . Nowhere does that other question mention mapfile and it is not even dealing with shell parameter assignments for later use! – Kevin Buchs Mar 07 '22 at 15:20
  • it doesn't specify mapfile, but it's _the exact same underlying problem with the exact same solution_. Part of the point of there being human expertise in establishing duplicates is so we can mark them when they aren't obvious -- and the nonobvious cases is where duplicates add the most value, by directing people to answers that will help them even when those answers might not be in their search terms. – Charles Duffy Mar 09 '22 at 13:25
  • To emphasize something there: "It's a duplicate" doesn't mean "it's not valuable". Closed is not deleted, and unlike other forms of closure, questions closed _as duplicates_ can still be upvoted. But if the other question's answers suffice to answer this question in full -- and they do -- it's better for everyone to have a single pool of answers that's getting as much attention as possible, rather than dividing into two lower-quality pools. – Charles Duffy Mar 09 '22 at 13:26
  • (I also disagree with you on stated facts; the linked duplicate _is_ dealing with shell variable assignments for later use, plainly and on its face). – Charles Duffy Mar 09 '22 at 13:28
  • Anyhow, let's take a look at the answer you accepted. It suggests fixing `... | mapfile` by replacing it with `mapfile < <(...)` -- just as the linked duplicate suggests fixing `... | read` by replacing it with `read < <(...)`; and it describes shell semantics (with pipelines creating implicit subshells) that the linked duplicates' answers _also_ describe, because those semantics are the cause of both instances of what is in fact the same problem. Other than the mere semantics of which _specific_ shell builtin is being used to populate a variable, I don't see any pertinent difference at all. – Charles Duffy Mar 09 '22 at 13:32
  • Similarly, the duplicate has answers that teach other approaches: `read <<<$(...)`, or `... | { read var; ...code that uses var...; }`, or using the shell option `shopt -s lastpipe` (in the specific set of conditions where it works); and _all_ of those approaches work with `mapfile` just as well as they do with `read`. By ignoring any answer that doesn't explicitly mention `mapfile` in its text you're needlessly constraining your options. – Charles Duffy Mar 09 '22 at 13:38

1 Answers1

1

Technically the array is being populated; the issue is that mapfile is called in a sub-process which, when it exits (back to the command line) the array assignment is lost, ie, you can't pass assignments 'up' from a sub-process to a parent/calling process.

Try these instead:

$ mapfile -d ' ' arr < <(echo -e "hello\nworld")
$ typeset -p arr
declare -a arr=([0]=$'hello\nworld\n')

$ mapfile -d ' ' arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello " [1]=$'world\n')

While these will populate the array ... you'll have to decide if this is what you were expecting to show up in the array. Perhaps the following are a bit closer to what's desired?

$ mapfile -t arr < <(echo -e "hello\nworld")
$ typeset -p arr
declare -a arr=([0]="hello" [1]="world")

$ mapfile -t arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello world")

On the 2nd command set, if the intention is to parse each word into an array then perhaps switch out mapfile with read?

$ read -ra arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello" [1]="world")
markp-fuso
  • 28,790
  • 4
  • 16
  • 36