Progress bar while Unix find command is executing

Question

I have a simple Unix shell script which is executing a time-consuming find command. Till it executes, my script appears non-responsive. However it's actually running in reality. How can I add a progress bar or print dots till the find command is executing?

I want to print some dots or hash till the function doSomething is executing.

function doSomething()
{
  # Time-consuming 'for' loop below
  for var in `find XXXXXX`
  do
      # Some more processing
  done;
}

# Call the function doSomething. Need the progress DOTS till doSomething is executing
doSomething

# Remaining script processing

I have tried to google, but none of the solutions fits what I am trying to achieve. Any suggestions?

The concept of `find` is that it searches through an unknown amount of files. Progress implies that you know how many searches it will take. What you are after is more an activity indicator. — kvantour, Oct 18 '19 at 09:11
Also it is very unclear what you really want. You could just write in your do-loop `echo -n .` and you will get a single dot every time a file is processed. — kvantour, Oct 18 '19 at 09:13
Even more related: https://serverfault.com/questions/341143/ — kvantour, Oct 18 '19 at 09:17
Hmm... certainly for find it is difficult to know how it is progressing, since you don't know how many files it will find. But you could do some sort of pulse bar or "spinny". Probably you could do this as a tool on the output pipe that emits to stderr: `find what-ever | spinny --expect_lines 50 | xargs archive`. If there is a `--expect` argument then it counts the amount of piped input.to generate a progress bar, otherwise it just does a pulsing `. .. ...` or zylon bar or similar. In either case it pipes the content through to the next pipe stage. — Gem Taylor, Oct 18 '19 at 10:03
As already commented out, given unknown size of work, impossible to create a pipeline with % competed. As an alternative, consider using 'pv', which will show progress bar. — dash-o, Oct 18 '19 at 10:05
Can you provide some information about the 'find' command ? Is 'some more processing' time consuming ? you might want to pipe the `find` into the processing, potentially, using xargs -P to process files in parallel, if relevant. — dash-o, Oct 18 '19 at 10:07

Jon · Answer 1 · 2019-10-18T10:27:57.387

0

As suggested in the comments, how about a simple spinner? It is easy enough to knock something up in awk, which copies input to standard output and prints a spinner to standard error:

$ cat spinner.awk
{ printf "%s\r", substr("|/-\\", NR % 4 + 1, 1) > "/dev/stderr" }

1

which you can then pipe find through.

  for var in `find XXXXXX | awk -f spinner.awk`
  do
      #some more processing
  done;

edited Oct 18 '19 at 10:27

answered Oct 18 '19 at 10:10

Jon

3,573
2
17
24

score 0 · Answer 2 · answered Oct 18 '19 at 10:37

You can run a spinner in the background

#!/usr/bin/env bash

spinner() {
  local s="|/-\\"
  local -i i=0
  while :; do
    printf "%s\\r" "${s:$((i++%4)):1}" >&2
    sleep .05
  done
}

function doSomething()
{
  # time consuming for loop below
  # See: https://github.com/koalaman/shellcheck/wiki/SC2044
  for var in $(find XXXXXX)
  do
    : #some more processing
  done
}

# Start the spinner in the background
spinner &

# Get the spinner PID
spinner_pid=$!
#call the function doSomething. Need the progress DOTS till doSomething is executing
doSomething

# Terminate the background running spinner
kill $spinner_pid

# remaining script processing

score 0 · Answer 3 · answered Nov 27 '19 at 07:25

It's not clear from the OP how many files are selected by the find command, and if the do-something has any significant impact on the overall processing time.

Assuming one of the common cases, where (1) find will scan large number of folders (2) relatively small number of matches and (3) little processing for each file, it is possible to create progress indication by logging directory that are scanned.

If the original find was using find STARTING-POINT ... FIND-EXPRESSION, the modified command is find STARTING-POINT '(' LOGGING-EXPRESSION ')' -o '( FIND-EXPRESSION ')'.

Following script will produce a progress report whenever find start scanning a new folder. Note that FIND-EXPRESSION should include explicit '-print', if the original find was relying on the implicit -print.

find STARTING-POINT   \
     '(' -type d -maxdepth 3 -fprintf /dev/stderr "Processing: %p" ')'   \
     -o \
     '(' FIND-EXPRESSION ')'

# Single Line
find STARTING-POINT '(' -type d -maxdepth N -fprintf /dev/stderr "Processing\n: %p" ')' -oo '(' FIND-EXPRESSION ')'

The above will log each scanned folder (up to N level deep) to stderr.

Example:

# Find all '*.py' files on the system, showing '*' for each folder
find / '(' -type d -fprintf /dev/stderr "*" ')' -o '(' -name '*.py' -print ')'

# Find system '*.so' files, showing each folder scanned
find /lib /usr/lib '(' -type d  -fprintf /dev/stderr "Scanning: %p" ')' -o '(' -name '*.so' -print ')'

Additional customization to the logging can be performed - Using the '-regex' or '-path' in the logging filter

Progress bar while Unix find command is executing

3 Answers3

Linked