0
$ ls . > /tmp/tfile.txt
$ flist=`cat /tmp/tfile.txt`
$ echo $flist
a.txt b.txt c.txt
$ echo ${flist#* }
a.txt b.txt c.txt

Why ${flist#* } cannot output b.txt c.txt? How to handle variables contain line breaks?

lufy
  • 33
  • 5
  • Your output shows ´b.txt´ and ´c.txt´. But it shouldn't show, as in question, because of how ´#*´ works. – j23 Feb 18 '22 at 10:27
  • 3
    Your current question aside, parsing the output of `ls` is almost always a bad idea. (Almost? Always!) There are various better alternatives, like `for file in *` or working with `find` (both of which will completely remove the need to substitute). – DevSolar Feb 18 '22 at 10:53
  • 1
    What is actually in the file (and variable)? `echo $flist` does *not* give an accurate indication of what's there (see ["I just assigned a variable, but echo $variable shows something else"](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else)). – Gordon Davisson Feb 18 '22 at 18:20
  • Note that in the real world you would want to use an array: `flist=( * )`, and then to print all but the first file, `printf '%s\n' "${flist[@]:1}"`. This also lets you refer to the first file as `"${flist[0]}"`, the second as `"${flist[1]}"`, etc. – Charles Duffy Feb 21 '22 at 02:46
  • ...if for some reason you _really do_ need to use `ls`, you could use `readarray -t flist < <(ls .)` in bash 4.0 or newer, or `IFS=$'\n' read -r -d '' -a flist < <(ls . && printf '\0')` to support bash 3.x as well. – Charles Duffy Feb 21 '22 at 02:51

3 Answers3

0

${flist#* } expands like $flist but removes the smallest portion matched by * (asterisk+space, where asterisk matches zero or more characters and space matches space character) from the beginning of it.

So, it removes characters up to and including the first space from the expansion. So in your case, it prints a.txt and then sees a space and exits, thereby not printing b.txt and c.txt.

if you put echo "${line#*.txt}", then the output will be:

b.txt c.txt

See this reference for understanding #* in bash.

j23
  • 3,139
  • 1
  • 6
  • 13
  • Thanks a lot. I am working with GNU bash, version 4.2.46, I get `a.txt b.txt c.txt` when `echo $flist`. I tested `echo "${flist#*.txt}"` works. But what I'm trying to do is to set line break as seperator. – lufy Feb 21 '22 at 02:29
  • @lufy Don't ever `echo $flist`; only ever `echo "$flist"`. Because your variable still contains linebreaks, you would need to use `${flist#*$'\n'}` to delete content up to the first linebreak, because even though `echo` doesn't show you the difference when you don't quote its arguments, _a linebreak and a space are two different characters_. – Charles Duffy Feb 21 '22 at 02:48
  • @CharlesDuffy Yes, got it. `${flist#*$'\n'}` is exactly what I'm looking for. `echo "$flist"` would would see the difference. Thanks very much for correcting so many faults in my habits. – lufy Feb 21 '22 at 15:51
0

Generally speaking the shell is not a great tool to process data and modify variable. (It is a great tool to work with Unix though, keep up learning and using it.) Some dialect can address this in some ways but it is still a good thing to keep in mind that complex data processing is better done outside the shell.

Most data processing is expected to be made with external tools rather than with shell primitives.

Here there is several ways to prune the first entry of the file you just read, the most idiosyncratic would be:

omit_first_line()
{
   tail -n+2 "$1"
}

flist=$(omit_first_line tfile.txt)

See how the $(…) are used instead of backticks, use of backticks is discouraged because they are hard to pair and to quote.

There is plenty of other tools that would skip the first line in a file for you, such as sed or awk.

If you are working on files, you really should consider using find and xargs, they work nicely together as in

find . -type f -print0 | xargs -0 stat

or

find . -type f -print | { while read filename; do printf 'Filename %s\n' "${filename}"; done }

Generally speaking, it is easier to compose filters than processing lists. Instead of storing data in lists you would do it in a classical imperative language, store your data in files, each record on a separate line, and pipe your data between filters. If you are able to just print your data, you might not need to store in a file it at all.

Now if you have good reasons to process your list in the shell, you can use positional arguments:

gobble()
(
  set -- "$@"
  shift
  echo "$@"
)

flist='a.txt b.txt c.txt'
flist=$(gobble ${flist})
echo ${flist}
Michaël Le Barbier
  • 6,103
  • 5
  • 28
  • 57
  • 1
    The unquoted expansions in the last example (using `gobble`) opens a can of worms. What if the last file in `flist` is named `*`? Every single time `gobble $flist` runs, the `*` will be replaced with a new list of names; and with one of those names being `*`; it'll never finish. – Charles Duffy Feb 21 '22 at 02:53
  • Beyond that, why not use `-print0 | { while IFS= read -r -d '' filename; do ...` and thus have the same level of safety you have there as in the example preceding? – Charles Duffy Feb 21 '22 at 02:54
-1

Many thaks for relies. Finally, I resolved this by re-assigning flist with flist=`echo $flist`;.

lufy
  • 33
  • 5