How to use bash to get unique dates from list of file names

Question

I have a large number of file names. I need to create a bash script that gets all of the unique dates from the file names.

Example:

input:

opencomposition_dxxx_20201123.csv.gz     
opencomposition_dxxv_20201123.csv.gz
opencomposition_dxxu_20201123.csv.gz     
opencomposition_sxxv_20201123.csv.gz
opencomposition_sxxe_20211223.csv.gz 
opencomposition_sxxe_20211224.csv.gz  
opencomposition_sxxe_20211227.csv.gz  
opencomposition_sxxesgp_20230106.csv.gz

output:

20201123 20211224 20211227 20230106

Code:

for asof_dt in `find -H ./ -maxdepth 1 -nowarn -type f -name *open*.gz
| sort -r | cut -f3 -d "_" | cut -f1 -d"." | uniq`; do
    echo $asof_dt
done

Error:

line 20: /bin/find: Argument list too long

Answer 1

Like this (GNU grep):

You need to add quotes on the glob: '*open*.gz', if not, the shell try to expand the wildcard *.

find -H ./ -maxdepth 1 -nowarn -type f -name '*open*.gz' |
    grep -oP '_\K\d{8}(?=\.csv)' |
    sort -u

Output

20201123
20211223
20211224
20211227
20230106

The regular expression matches as follows:

Node	Explanation
_	_
\K	resets the start of the match (what is Kept) as a shorter alternative to using a look-behind assertion: perlmonks look arounds and Support of K in regex
\d{8}	digits (0-9) (8 times)
(?=	look ahead to see if there is:
\.	.
csv	'csv'
)	end of look-ahead

Answer 2

Using tr:

find -H ./ -maxdepth 1 -nowarn -type f -name '*open*.gz' | tr -d 'a-z_.' | sort -u

Answer 3

If filenames don't contain newline characters, a quick-and-dirty method, similar to your attempt, might be

printf '%s\n' open*.gz | cut -d_ -f3 | cut -d. -f1 | sort -u

Note that printf is a bash builtin command and argument list too long is not applied to bash builtins.