How to use the ‘find’ command (from Cygwin) to print the list of values (file size, file name), no matter what chars occur in a directory/file name?

Question

Given a path to some directory (i.e. path/to/directory), I want to obtain the output in the following format:

100|path/to/directory/a/file1.txt
200|path/to/directory/b/file2.txt
321|path/to/directory/c/d/file3.txt
...

Here 100, 200, 321 etc. are the sizes (in bytes) of the corresponding files (the symbol | is used as a separator.)

I tried to use the following command:

find path/to/directory -maxdepth 3 -type f -printf "%s|%p\n" > path/to/output.txt

But then I had a lot of error messages of the following form:

find: `path/to/directory/my directory 1': No such file or directory

I noticed that this happened when the name of some directory contained at least one space. So my question is: what command can I use so that it will produce the desired output, no matter what chars occur in file names and directory names?

The question is not a duplicate of this question because there is an additional issue of problematic names. This answer mentions the issue of subdirectory or file names containing white space, then says that it can be solved by -print0 | xargs -0, then says that it's easier to avoid using the xargs construction altogether. I have found one relevant answer here, but was unable to modify it so that it would fit my needs.

EDIT

I have found the issue that causes the problem. The thing is, I run these commands from Cygwin or MSYS2, which leads to this problem because the long filenames contain characters encoded by more than one byte. The problem is also described here and here. Therefore, Cygwin and MSYS2 are unable to deal with some files whose paths do not exceed the maximum length (255 characters) allowed by the OS (Windows). For example, I have created a file whose full path (in Windows) is the following:

E:/test1/αβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγ.txt

Then I run the following command:

find E:/test1 -maxdepth 3 -type f -printf "%s|%p\n" > E:/test2/output.txt

But I got the following error:

find: ‘E:/test1/αβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγαβγα\316’: No such file or directory

As far as I understand, it cannot be fixed.

Answer 1

I think you can do it like this

find path/to/directory -type f -exec stat -c "%s|%n" {} \;

• -exec stat -c "%s|%n" {} \; executes the stat command for each file found and %s represents the file size in bytes, and %n represents the file name, the | character is used as a separator between the size and the path!

you can also try this:

find path/to/directory -maxdepth 3 -type f -exec sh -c 'size=$(stat -c "%s" "$1"); echo "$size|$1"' sh {} \; > path/to/output.txt

Answer 2

Using nftw function via TXR Lisp:

$ txr sizepath.tl /usr/share/man
2408|/usr/share/man/vi/man1/help2man.1.gz
2246|/usr/share/man/ru/man8/groupmems.8.gz
2451|/usr/share/man/ru/man8/groupmod.8.gz
[...]

The nftw POSIX C library function has a FTW_CHDIR flag specified in the standard which causes the function to chdir into every directory it traverses.

I believe that when FTW_CHDIR is used, nftw is not thwarted by long paths, since internally, it's using short relative paths to do its descent.

In TXR Lisp, this flag value appears as the variable ftw-chdir. The sizepath.tl program is:

(ftw *args*
     (lambda (path type stat . rest)
       (when (eql type ftw-f)
         (put-line `@{stat.size}|@path`)))
     ftw-chdir)

Regardless of whether you use ftw-chdir, the path argument to the callback function specifies the full path as if chdir calls were not taking place; so that is to say, if the starting points are relative paths, then path will be a relative path from the original directory. If we had to open the file, we would have to use (base-name path), but we don't: ftw gives us a stat argument which is the stat structure; we just pull the size field from that.

Although nftw is a POSIX function, this works in the Windows version of TXR, which carries an extensive POSIX run-time library.

At the Windows 10 cmd.exe prompt:

C:\Users\kaz>copy con sizepath.tl
(ftw *args*
     (lambda (path type stat . rest)
       (when (eql type ftw-f)
         (put-line `@{stat.size}|@path`)))
     ftw-chdir)
^Z
        1 file(s) copied.

C:\Users\kaz>txr sizepath C:\Users\kaz\AppData\Local\Microsoft
51959|C:\Users\kaz\AppData\Local\Microsoft/CLR_v4.0/ngen.log
2339|C:\Users\kaz\AppData\Local\Microsoft/CLR_v4.0/UsageLogs/AsusJWTDecoder.exe.log
1327|C:\Users\kaz\AppData\Local\Microsoft/CLR_v4.0/UsageLogs/AsusLiveUpdateToast.exe.log
[...]

Native paths with drive letter names and all.