Truncate file in linux with wget download active

Question

i have background download with wget, i want, when the file is greater than 20mb, truncate the first 10mb of file, i have created this script:

if [ $filesize -ge $maxSize ]; then
            echo "Truncate.."
            kill -STOP $pidDwn
            fallocate -c -o 0 -l 10M $fileName
            kill -CONT $pidDwn
fi

this snippet recognize file size and truncate it for 10MB from beginning of file.

I stop wget progess, use fallocate for delete first 10MB, and after i resume the wget process for continue the download. The problem is strange, if the file size is 20mb and i use fallocate WITHOUT resume the process wget, the file remains 10mb, but if i resume the wget process the file return to 20mb instantly and continue to increase with download. If i use this command after resume the pid sed -i 1d $fileName the file remains 10mb but not increase anymore with the download, seems like download is interrupted, but wget process still alive if i use ps aux for see all process active

any idea for fix it?

While it's better to limit the scope of the questions on SO, I can say that the current question is an XY problem because of your comments on my answer. I can't even start making a guess on what you're really trying to achieve. — Fravadona, Aug 10 '22 at 15:26
i have solved by creating new file after 20mb, after some time i delete the oldfile, on the php page i use new file when i reach end of current file, the solution work, i want thank you for yuo time and patience, i mark you correct answer because it work in generale case, but in my case i need other. :-) — Salvosnake, Aug 11 '22 at 13:20

score 0 · Answer 1 · answered Aug 09 '22 at 08:02

truncate the first 10mb of file

If server from which you are downloading said file support Partial Content gimmick then you might request Range (part) of file. First check if server support that feature do

wget --spider --server-response <url_to_resource>

if what was printed contain

Accept-Ranges: bytes

then it does and you might request file starting from, say 100th byte using Range header that is

wget --header "Range: bytes=100-" <url_to_resource>

You should then get response with code 206 and download will start.

Truncate file in linux with wget download active

1 Answers1