0

Inside C:\cygwin64\home\Administrator I have 3 files

script.sh
links_da_scaricare.txt
cookies.txt

To download from dplay I use successfully if I use this command, videos are downloaded

yt-dlp --referer "https:www.discoveryplus.com/" --add-header "Cookie:gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA@AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect" --batch-file links_da_scaricare.txt

ok, I try to download using Cygwin and I use this command

bash -x script.sh

But I get this error

$ bash -x script.sh
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas could not be read
+ IFS=
+ read -r url

script.sh have this code

#!/bin/bash

while IFS= read -r url; do
folder_name=$(echo "$url" | sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1')

    echo "folder_name: $folder_name"
    mkdir -p "/cygdrive/c/cygwin64/home/Administrator/$folder_name"
    echo "The script is running and creating folder: $folder_name" > ~/script.log
        yt-dlp --referer "https://www.discoveryplus.com/" --cookies "/cygdrive/c/cygwin64/home/Administrator/cookies.txt" -a "$url" -o "/cygdrive/c/cygwin64/home/Administrator/$folder_name/%(title)s.%(ext)s"
    done < /cygdrive/c/cygwin64/home/Administrator/links_da_scaricare.txt

links_da_scaricare.txt have these 3 lines - I use notepad to copy paste these links

https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

cookies.txt is this - is the same that I use in powershell command so it should download because with powershell I can do it but with bash i have problems, I don't know why

gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA@AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect

In other words script.sh should be do this

--> read urls from links_da_scaricare.txt and extract only these strings

citta-mortale
il-boss-delle-cerimonie

--> make folders with that names

--> download

https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas

in this folder

citta-mortale

--> download

https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

--> in this folder

il-boss-delle-cerimonie

Script creates the first folder but doesn't download anything inside it

user3520363
  • 360
  • 1
  • 15
  • 1
    The sed expression is indeed buggy, just as the error message tells you. You need to have a sigil character after `s` before the thing to be replaced, then repeat that sigil between the regex to be replaced and the content to replace it with, and again at the end of the replacement content. – Charles Duffy Jan 12 '23 at 18:40
  • 2
    See [Using different delimiters in sed commands and range addresses](https://stackoverflow.com/questions/5864146/using-different-delimiters-in-sed-commands-and-range-addresses) -- granted, the problem your code has most immediately is that it isn't using any delimiter _at all_ (so you end up with a delimiter of `h`, even though that's clearly intended to be the first character of the search string). Maybe you meant to use a `~` as the delimiter but left it out in the first position? – Charles Duffy Jan 12 '23 at 18:43

1 Answers1

0

You are looking to download files into a directory named according to the 6th field delimited by "/" in the url specification (between slash #5 and #6).

That is accomplished as

folder_name=$( echo "${url}" | cut -f6 -d/ )

Regarding sed, you must specify a delimiter character which does not conflict with any possible characters that might be found in the text string that you are trying to edit.

For URLs that always have "/", you cannot use "/" as that delimiter. You must choose something else. I tend to use either the "+" or the "|" depending on circumstances.

So to strip the prefix from the URL string, you would use

echo "${url}" | sed 's+https://www.discoveryplus.com/it/video/++'

to obtain

citta-mortale/stagione-1-episodio-1-strada-per-linferno
citta-mortale/stagione-1-episodio-2-non-in-kansas
il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

You would make a second, similar, usage of sed to strip the tail end after the "/".

OR ... you can again use cut in this manner

id_string=$( echo "${url}" | cut -f7 -d/ )

to obtain

stagione-1-episodio-1-strada-per-linferno
stagione-1-episodio-2-non-in-kansas
stagione-5-il-quarantesimo-compleanno-di-nello
Eric Marceau
  • 1,601
  • 1
  • 8
  • 11
  • `#!/bin/bash while IFS= read -r url; do folder_name=$( echo "${url}" | cut -f6 -d/ ) echo "${url}" | sed 's+https://www.discoveryplus.com/it/video/++' mkdir -p "/cygdrive/c/cygwin64/home/Administrator/$folder_name" echo "The script is running and creating folder: $folder_name" > ~/script.log yt-dlp --referer "https://www.discoveryplus.com/" --cookies "/cygdrive/c/cygwin64/home/Administrator/cookies.txt" -a "$url" -o "/cygdrive/c/cygwin64/home/Administrator/$folder_name/%(title)s.%(ext)s" done < /cygdrive/c/cygwin64/home/Administrator/links_da_scaricare.txt` – user3520363 Jan 13 '23 at 14:10
  • I edit script.sh in that way but cygwin terminal returns me this error `Administrator@pip ~ $ bash -x script.sh + IFS= + read -r url ++ echo citta-mortale/stagione-1-episodio-1-strada-per-linferno ++ cut -f6 -d/ + folder_name= + echo citta-mortale/stagione-1-episodio-1-strada-per-linferno + sed s+https://www.discoveryplus.com/it/video/++ citta-mortale/stagione-1-episodio-1-strada-per-linferno + mkdir -p /cygdrive/c/cygwin64/home/Administrator/ + echo 'The script is running and creating folder: ' + yt-dlp --referer https://www.discoveryplus.com/ --cookies` – user3520363 Jan 13 '23 at 14:11
  • `/cygdrive/c/cygwin64/home/Administrator/cookies.txt -a citta-mortale/stagione-1-episodio-1-strada-per-linferno -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s' ERROR: batch file citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read` – user3520363 Jan 13 '23 at 14:11
  • I don't understand why always it give me "ERROR: batch file citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read" – user3520363 Jan 13 '23 at 14:12
  • Step away from the code. If you start by writing the code logic - *in pseudo-code*, - fine-grained for each task that you are trying to accomplish, - in the correct sequence and - in the correct context, then having that worded so that it does exactly what you want it to do ... will, *almost explicitly*, tell you **WHAT** you need to code for each of those, not the **HOW**. The **HOW** is the nitty gritty of coding. The solution should then be obvious. What you have provided above in your question does not reflect that level of thought on the task breakdown. – Eric Marceau Jan 14 '23 at 18:28