-2

I wrote a script which count number of lines in each file:

todays_day=$(date +%d) 
if ((todays_day==1)); then
  month="$(date --date='1 day ago' +%y%m)"
else
  month="$(date +%y%m)"
fi
for catalog in $(find ./ -type d -name "$month")
   do 
    sum=0
   find $catalog -type f -name "*.z" | while read FN 
   do   
    sum=$((sum+$(zcat $FN | awk 1 | wc -l)))
    echo "no of lines $sum;source $(find $catalog -type d );data $(date +"%d-%m-%Y %T")"
    done
done

Now it accumulate number of lines and shows in result

no of lines 3;source ./AXE/CNA5/LBN/2211;data 01-12-2022 17:35:49
no of lines 6;source ./AXE/CNA5/LBN/2211;data 01-12-2022 17:35:50
no of lines 9;source ./AXE/CNA5/LBN/2211;data 01-12-2022 17:35:50
no of lines 13;source ./AXE/CNA5/LBN/2211;data 01-12-2022 17:35:51
no of lines 3;source ./AXE/TELLIN/2211;data 01-12-2022 17:35:51
no of lines 7;source ./AXE/TELLIN/2211;data 01-12-2022 17:35:51

But it must show

no of lines 13;source ./AXE/CNA5/LBN/2211;data 01-12-2022 17:35:51 no of lines 7;source ./AXE/TELLIN/2211;data 01-12-2022 17:35:51

How can i fix it?

I read Shell command to sum integers, one per line? but it did not help me . I expecting someone show me how can i change my code

Kid Jeremy
  • 1
  • 1

1 Answers1

0

You have a few issues.

First, why bother to test this?

todays_day=$(date +%d) 
if ((todays_day==1)); then
  month="$(date --date='1 day ago' +%y%m)"
else
  month="$(date +%y%m)"
fi

I think all you need is

month="$(date --date='1 day ago' +%y%m)"

which accomplishes the same thing.

Second, your output is inside your loop, which isn't what you want - but since you are using a pipe, your variables will be scoped to the subshell of the loop.
Try this -

month="$(date --date='1 day ago' +%y%m)"
while read catalog 
do sum=0
   while read lines 
   do ((sum+=lines))
   done < <( zgrep -ch . "$catalog"/*.z )
   echo "no of lines $sum;source $catalog;data $(date +"%d-%m-%Y %T")"
done < <(find ./ -type d -name "$month" )

Note only one find. If your example output can be trusted for a pattern, we could simplify the outer loop to

for catalog in ./AXE/*/"$month"

or, if there could be deeper more then one level of subdirectories between,

shopt -s globstar
for catalog in ./AXE/**/"$month"

which eliminates the need for the find at the end entirely.

#!/bin/bash
month="$(date --date='1 day ago' +%y%m)"   # sums previous month on the 1st
for catalog in ./AXE/*/"$month"            # processes each $month dir
do sum=0                                   # reinit sum for each dir
   while read lines                        # lines is the count per file
   do ((sum+=lines))                       # sum the files in the dir
   done < <( zgrep -ch . "$catalog"/*.z )  # zgrep is your friend
   echo "no of lines $sum;source $catalog;data $(date +"%d-%m-%Y %T")"
done

edit

As I said above,

or, if there could be deeper more then one level of subdirectories between,

shopt -s globstar
for catalog in ./AXE/**/"$month"

Substituting that in,

#!/bin/bash
shopt -s globstar                          # allow ** for arbitrary depth
month="$(date --date='1 day ago' +%y%m)"   # sums previous month on the 1st
for catalog in ./AXE/**/"$month"/          # / at the end gets dirs only
do sum=0                                   # reinit sum for each dir
   while read lines                        # lines is the count per file
   do ((sum+=lines))                       # sum the files in the dir
   done < <( zgrep -ch . "${catalog}"*.z ) # / is part of $catalog
   echo "no of lines $sum;source $catalog;data $(date +"%d-%m-%Y %T")"
done

The outer loop above selects applicable directories; the inner loop counts records in each file and sums by directories, which is what I thought you wanted.

In your answer post with your edited code, you have changed the outer loop to directly represent each individual file, so the inner loop is only reading one line, which leaves nothing to sum, so there's no need for an inner loop, but no sums by directory.

Did you want totals per file, per directory, or something else?

updated

Apologies. I included the *.z glob inside the quotes. Corrected, above.

If there are no naughty characters such as spaces in any of the path names you could leave the quotes off entirely, but I tend not to make that assumption - it will surprise you.

Paul Hodges
  • 13,382
  • 1
  • 17
  • 36
  • If you use a pipe the value of `sum` will be scoped to the subshell it creates, and lost as soon as the loop ends, unavailable to be reported by the next line. – Paul Hodges Dec 02 '22 at 15:13
  • Thanks. I need total per directory. For ex I have directories WORK6\AXE\CNA5\LBN\2212 - with 2 files and 3 lines in each , WORK6\AXE\TELLIN\2212 - 2 files in 2 in each . Result must be no of lines WORK6\AXE\CNA5\LBN\2212 - 6 , WORK6\AXE\TELLIN\2212 - 4 . How can I change these `for catalog in ./AXE/**/$month/ ` if in directories these part can be change (add,new directories or add more child directories for ex AXE\TELLIN\..\...\2212) – Kid Jeremy Dec 05 '22 at 07:04
  • Should work as-is unless they are "hidden" directories with names that start with a dot. Is that last code version above not working? The first version I gave with `while read catalog` from the `done < <(find ./ -type d -name "$month" )` should work too, much the same. Are they not? – Paul Hodges Dec 05 '22 at 15:51
  • It writes `gzip: ./AXE/CNA5/LBN/2212/*.z: No such file or directory` in both cases, with `while read catalog` and with last code , my catalogs are not hidden. I use GitBash, may be it's cause – Kid Jeremy Dec 06 '22 at 14:07
  • Apologies, my fault - updated. Try now. – Paul Hodges Dec 06 '22 at 14:37
  • It shows `no of lines 27;source ./AXE/CNA5/LBN/2212/;data 08-12-2022 08:52:29 no of lines 26;source ./AXE/TELLIN/2212/;data 08-12-2022 08:52:31` thought there are 2 files in each catalog with 7 and 6 lines total – Kid Jeremy Dec 08 '22 at 05:57
  • First, on the CLI, run `shopt -s globstar` - then what does show when you run `ls -la ./AXE/**/2212/*.z` and `zgrep -c . ./AXE/**/2212/*.z` and `zgrep . ./AXE/**/2212/*.z`? – Paul Hodges Dec 08 '22 at 14:31