0

I am banging my head against a wall and I'm certain I'm just being dense.

I am attempting to run a for loop with a lftp command against files in a watched folder basically a backup and move process. But due to the way LFTP and the loop works if I process files on the destination before everything is done I get a failure.

So what I want to do is read all files with *.log and then if there are say 20 files only process 10, then on its next run process 10 files again eventually it will be at a point of 1 file at a time. I can loop through everything fine. just can not seem to figure out how to only read the first 10 files as I say.

for FILE in *.log; do
lftp -p 2252 -u $FTPUser,$FTPPass $Location <<EOF
set ftp:ssl-allow no
set xfer:use-temp-file on
set xfer:temp-file-name *.tmp
set log:file/xfer /log/LFTP_$FILE.log;
mput $LogPath/$FILE
quit
EOF
   if [ $? == "0" ]; then
        rm $LogPath/$FILE
    else
        echo "Error"
    fi
done;
Digi
  • 9
  • 7

2 Answers2

1

You only need one lftp connection per batch.

Also, don't use upper case for private variables, and quote your variables; see also Why is testing “$?” to see if a command succeeded or not, an anti-pattern?, and take care to send diagnostic messages to standard error, not standard output.

files=(*.log)
for ((i=0; i<${#files[@]}; i+=10)); do
    slice=("${files[0]:$i:10}")
    slice=("${slice[@]/#/$LogPath/}")
    if { cat <<________EOF
        set ftp:ssl-allow no
        set xfer:use-temp-file on
        set xfer:temp-file-name *.tmp
        set log:file/xfer /log/LFTP_$FILE.log;
________EOF
        printf "mput %s\n" "${slice[@]}" 
        echo quit; } |
      lftp -p 2252 -u "$FTPUser,$FTPPass" "$Location"
    then
        rm "${slice[@]}"
    else
        echo "$0: Error for ${slice[@]}" >&2
    fi
done

Bash doesn't let you combine slicing and prefix substitution on an array in one go, so we copy the current slice to slice so that we can add $LogPath/ in front of each entry.

tripleee
  • 175,061
  • 34
  • 275
  • 318
0

Using an array:

cd "$LogPath"
files=( *.log )

if lftp -p 2252 -u "$FTPUser,$FTPPass" "$Location" <<EOF
    set ftp:ssl-allow no
    set xfer:use-temp-file on
    set xfer:temp-file-name *.tmp
    set log:file/xfer /log/LFTP_$FILE.log;
    mput ${files[@]:0:10}
    quit
EOF
then
    rm ${files[@]:0:10}
fi

Explanations

${files[@]:0:10}

is a parameter expansion that retrieve only the first 10 elements

Gilles Quénot
  • 173,512
  • 41
  • 224
  • 223