0

I have URLs in JSON format which contain a lot of .mp4 file URLs, is there a way to use Python to scan the URLs and download all .mp4 files using a loop and urlib -request?

For downloading 1 file with exact URL, the below code works, but my issue is that url.json contains 100 videos.

import urllib
urllib.url_retrive("http://example.com/helo.mp4","/var/opt")

Is there a module that will scan the page and get all .mp4 URLs and loop and download each one?

ggorlen
  • 44,755
  • 7
  • 76
  • 106
Jecki
  • 802
  • 3
  • 16
  • 32
  • `import json`, and go from there – Vlad Dec 17 '15 at 09:16
  • I got this part , but my main concern here is i've 100 videos on that json body url , how parse it and not to pass 100 urllib.url_retrive – Jecki Dec 17 '15 at 09:19
  • If you have 100 files to download, you'll eventually have to `url_retrieve()` 100 times, right? – Vlad Dec 17 '15 at 09:29
  • @Vlad true looping it and parsing the problem – Jecki Dec 17 '15 at 09:37
  • It's not really clear what the format of the links is. Is the JSON file a bunch of links to pages with HTML that have the mp4 links on them or are the mp4 links in the JSON itself? – ggorlen Jan 28 '21 at 15:49

1 Answers1

3

Break down this problem first.

First, it's downloading JSON from a URL and parsing it. See https://stackoverflow.com/a/13921930/230340

Then, it's downloading files. There are some great solutions right here: How do I download a file over HTTP using Python?

The whole process may look like this:

import urllib, urllib2
import json
import uuid

# download and parse JSON
response = urllib2.urlopen('https://someprovider/of.json')
data = json.load(response)

# data.links is the array of download links, rename it to fit your JSON.
for link in data.links:
  urllib.url_retrive(link, "/var/opt")
Community
  • 1
  • 1
Zippo
  • 15,850
  • 10
  • 60
  • 58
  • I got this part , but my main concern here is i've 100 videos on that json body url , how parse it and not to pass 100 urllib.url_retrive – Jecki Dec 17 '15 at 09:19
  • @Jecki First, you load the download the JSON and parse it. Only then you begin looping over the files and downloading them. – Zippo Dec 17 '15 at 09:25
  • thats help me alot ill take it from here and build the rest of the code. – Jecki Dec 17 '15 at 09:35
  • @Jecki `urllib.url_retrieve` is your download function. If you want to download a 100 files, you must call it a 100 times. (Sorry, I didn't get you the first time) – Zippo Dec 17 '15 at 09:37