0

I am trying to append the contents of files that match a pattern to another file while wrapping the contents of the file in an identifying line. I am attempting to do this in a single line as I want to add this to a prerotate hook in logrotate.

This is what I have been tinkering with

root@KTCH0S3HY:/opt/teamcity/logs# ls /opt/teamcity/logs/catalina.*.log | xargs -I {} echo -e "-----START {}------\n`tail
 "{}" 2>&1`\n------END {}------"  | tee -a test.log
-----START /opt/teamcity/logs/catalina.2021-04-08.log------
tail: cannot open '/opt/teamcity/logs/catalina.2021-04-08.log' for reading: No such file or directory
------END /opt/teamcity/logs/catalina.2021-04-08.log------
-----START /opt/teamcity/logs/catalina.2021-04-09.log------
tail: cannot open '/opt/teamcity/logs/catalina.2021-04-09.log' for reading: No such file or directory
------END /opt/teamcity/logs/catalina.2021-04-09.log------
-----START /opt/teamcity/logs/catalina.2021-04-10.log------
tail: cannot open '/opt/teamcity/logs/catalina.2021-04-10.log' for reading: No such file or directory
------END /opt/teamcity/logs/catalina.2021-04-10.log------
-----START /opt/teamcity/logs/catalina.2021-04-11.log------
tail: cannot open '/opt/teamcity/logs/catalina.2021-04-11.log' for reading: No such file or directory
------END /opt/teamcity/logs/catalina.2021-04-11.log------
-----START /opt/teamcity/logs/catalina.2021-04-12.log------
tail: cannot open '/opt/teamcity/logs/catalina.2021-04-12.log' for reading: No such file or directory
------END /opt/teamcity/logs/catalina.2021-04-12.log------

Here is the same command except the I am tailing the full path rather than building the path from the output of the ls command

root@KTCH0S3HY:/opt/teamcity/logs# ls /opt/teamcity/logs/catalina.*.log | xargs -I {} echo -e "-----START {}------\n`tail "/
opt/teamcity/logs/catalina.2021-04-12.log" 2>&1`\n------END {}------"  | tee -a test.log
-----START /opt/teamcity/logs/catalina.2021-04-08.log------
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 java.lang.Thread.run(Thread.java:748)
12-Apr-2021 19:03:52.962 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8111"]
12-Apr-2021 19:03:52.962 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8111"]
------END /opt/teamcity/logs/catalina.2021-04-08.log------
-----START /opt/teamcity/logs/catalina.2021-04-09.log------

So the file does actually exist but I am unable to loop over all files and tail them as shown in the first output.

Buster
  • 685
  • 1
  • 5
  • 28
  • Note that `ls | xargs` is itself bad practice -- it doesn't work for all possible filenames. See [ParsingLs](https://mywiki.wooledge.org/ParsingLs). – Charles Duffy Apr 12 '21 at 22:26
  • ...for `tail`, you can just pass it multiple filenames and let it do the work of saying where each starts and ends. `tail /opt/teamcity/logs/catalina.*.log` and there you are. – Charles Duffy Apr 12 '21 at 22:27
  • 1
    ...even if you don't want to do that, though -- use a `for` loop. `xargs`, like every other command, runs **after** the shell has expanded its arguments. That means that the code you run in backticks runs before `xargs` even starts, so you get `tail: cannot open '{}' for reading: No such file or directory` being spit out to stderr, captured by the command substitution, and then having the `{}` replaced by xargs with a list of individual filenames before `xargs echo` substitutes the filenames in and writes to stdout. – Charles Duffy Apr 12 '21 at 22:29
  • And _beyond that_, `echo -e` has a bunch of bugs that are pretty much unique to it. See [Why is `printf` better than `echo`?](https://unix.stackexchange.com/a/65819/3113) over at our sister site [unix.se]. – Charles Duffy Apr 12 '21 at 22:32
  • `for f in /opt/teamcity/logs/catalina.*.log; do echo "--- START $f ---"; tail -- "$f"; echo "--- END $f ---"; done` and there you are, much simpler, more reliable, more robust, etc. – Charles Duffy Apr 12 '21 at 22:34

0 Answers0