Download a remote image and save it to a Django model

Question

I am writing a Django app which will fetch all images of particular URL and save them in the database.

But I am not getting on how to use ImageField in Django.

Settings.py

MEDIA_ROOT = os.path.join(PWD, "../downloads/")

# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash.
# Examples: "http://example.com/media/", "htp://media.example.com/"
MEDIA_URL = '/downloads/'

models.py

class images_data(models.Model):
        image_id =models.IntegerField()
        source_id = models.IntegerField()
        image=models.ImageField(upload_to='images',null=True, blank=True)
        text_ind=models.NullBooleanField()
        prob=models.FloatField()

download_img.py

def spider(site):
        PWD = os.path.dirname(os.path.realpath(__file__ ))
        #site="http://en.wikipedia.org/wiki/Pune"
        hdr= {'User-Agent': 'Mozilla/5.0'}
        outfolder=os.path.join(PWD, "../downloads")
        #outfolder="/home/mayank/Desktop/dreamport/downloads"
        print "MAYANK:"+outfolder
        req = urllib2.Request(site,headers=hdr)
        page = urllib2.urlopen(req)
        soup =bs(page)
        tag_image=soup.findAll("img")
        count=1;
        for image in tag_image:
                print "Image: %(src)s" % image
                filename = image["src"].split("/")[-1]
                outpath = os.path.join(outfolder, filename)
                urlretrieve('http:'+image["src"], outpath)
                im = img(image_id=count,source_id=1,image=outpath,text_ind=None,prob=0)
                im.save()
                count=count+1

I am calling download_imgs.py inside one view like

        if form.is_valid():
                url = form.cleaned_data['url']
                spider(url)

https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ImageField — Paulo Bu, Apr 23 '13 at 16:11
Do you want to save the images as blobs in the db or the image paths — mossplix, Apr 23 '13 at 16:38
@mossplix - Either way... But if i save images as path then I also want images on my server — Mayank Jain, Apr 23 '13 at 16:48
@PauloBu - I read that document many times.But not getting on how to proceed — Mayank Jain, Apr 23 '13 at 16:49

score 71 · Accepted Answer · edited Oct 27 '20 at 18:12

71

Django Documentation is always good place to start

class ModelWithImage(models.Model):
    image = models.ImageField(
        upload_to='images',
    )

UPDATED

So this script works.

Loop over images to download
Download image
Save to temp file
Apply to model
Save model

.

import requests
import tempfile

from django.core import files

# List of images to download
image_urls = [
    'http://i.thegrindstone.com/wp-content/uploads/2013/01/how-to-get-awesome-back.jpg',
]

for image_url in image_urls:
    # Stream the image from the url
    response = requests.get(image_url, stream=True)

    # Was the request OK?
    if response.status_code != requests.codes.ok:
        # Nope, error handling, skip file etc etc etc
        continue
    
    # Get the filename from the url, used for saving later
    file_name = image_url.split('/')[-1]
    
    # Create a temporary file
    lf = tempfile.NamedTemporaryFile()

    # Read the streamed image in sections
    for block in response.iter_content(1024 * 8):
        
        # If no more file then stop
        if not block:
            break

        # Write image block to temporary file
        lf.write(block)

    # Create the model you want to save the image to
    image = Image()

    # Save the temporary image to the model#
    # This saves the model so be sure that it is valid
    image.image.save(file_name, files.File(lf))

Some reference links:

requests - "HTTP for Humans", I prefer this to urllib2
tempfile - Save temporay file and not to disk
Django filefield save

edited Oct 27 '20 at 18:12

Lord Elrond

13,430
7
40
80

answered Apr 23 '13 at 16:49

rockingskier

9,066
3
40
49

I did this Even changed MEDIA_ROOT path accordingly.. Now what? – Mayank Jain Apr 23 '13 at 16:51
OK, so is the problem getting the image into the DB or accessing it after? – rockingskier Apr 23 '13 at 16:52
Not able to get image to DB with image on my server – Mayank Jain Apr 23 '13 at 16:52
Show us more code. The `view` where you try to fetch the images, the `model` definition and maybe `settings.py` – Paulo Bu Apr 23 '13 at 17:04
Will do, I'm almost there for you but got to dash, will post later if no-one else gets there – rockingskier Apr 23 '13 at 17:15
@rockingskier np please guide me when you are free.Thanks – Mayank Jain Apr 23 '13 at 17:17
@Mayank Jain. There you go. – rockingskier Apr 24 '13 at 08:55
@rockingskier - This isn't working either..Am able to save it to DB but not able to download images to appropriate path – Mayank Jain Apr 24 '13 at 15:42
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/28831/discussion-between-rockingskier-and-mayank-jain) – rockingskier Apr 24 '13 at 16:31
My mistake solution is working Perfectly...Actually Heroku's servers are read only...and as soon as images are downloaded to server they are deleted..This was the problem – Mayank Jain Apr 24 '13 at 18:15
2

I know this isn't an acceptable comment, but this helped me tons! Thank you, @rockingskier – teewuane Jul 15 '14 at 00:13
I don't understand this statement: `image = Image()` – avi Dec 25 '15 at 15:32
That is creating an instance of a Model so you can save the downloaded image against it. Ie `user = User(); user.avatar.save(...)` – rockingskier Dec 26 '15 at 21:22
use res.headers['content-disposition'] for [getting filename](https://stackoverflow.com/questions/31804799/how-to-get-pdf-filename-with-python-requests) – Pavel1114 Dec 20 '17 at 16:33
A couple tangent questions if anyone can answer: why is it called "streaming" and why is a temporary file needed? – Jay Jung Sep 19 '19 at 07:49
Thank you Brother :) – Muhammad Usman Oct 14 '20 at 19:07

score 37 · Answer 2 · edited Oct 26 '17 at 12:24

If you want to save downloaded images without saving them to disk first (without using NamedTemporaryFile etc) then there's an easy way to do that.

This will be slightly quicker than downloading the file and writing it to disk as it is all done in memory. Note that this example is written for Python 3 - the process is similar in Python 2 but slightly different.

from django.core import files
from io import BytesIO
import requests

url = "https://example.com/image.jpg"
resp = requests.get(url)
if resp.status_code != requests.codes.ok:
    #  Error handling here

fp = BytesIO()
fp.write(resp.content)
file_name = url.split("/")[-1]  # There's probably a better way of doing this but this is just a quick example
your_model.image_field.save(file_name, files.File(fp))

Where your_model is an instance of the model you'd like to save to and .image_field is the name of the ImageField.

See the documentation for io for more info.

score 5 · Answer 3 · answered Jul 16 '21 at 19:42

5

# this is my solution
from django.core import files
from django.core.files.base import ContentFile

import requests
from .models import MyModel

def download_img():
    r = requests.get("remote_file_url", allow_redirects=True)
    filename = "remote_file_url".split("/")[-1]

    my_model = MyModel(
        file=files.File(ContentFile(r.content), filename)
    )
    my_model.save()

    return

answered Jul 16 '21 at 19:42

Gus Tavo

51
1
1

While this code may solve the question, [including an explanation](//meta.stackexchange.com/q/114762) of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please [edit] your answer to add explanations and give an indication of what limitations and assumptions apply. – Yunnosch Aug 01 '21 at 20:52

score 2 · Answer 4 · answered Apr 23 '13 at 18:47

As an example of what I think you're asking:

In forms.py:

imgfile = forms.ImageField(label = 'Choose your image', help_text = 'The image should be cool.')

In models.py:

imgfile =   models.ImageField(upload_to='images/%m/%d')

So there will be a POST request from the user (when the user completes the form). That request will contain basically a dictionary of data. The dictionary holds the submitted files. To focus the request on the file from the field (in our case, an ImageField), you would use:

request.FILES['imgfield']

You would use that when you construct the model object (instantiating your model class):

newPic = ImageModel(imgfile = request.FILES['imgfile'])

To save that the simple way, you'd just use the save() method bestowed upon your object (because Django is that awesome):

if form.is_valid():
    newPic = Pic(imgfile = request.FILES['imgfile'])
    newPic.save()

Your image will be stored, by default, to the directory you indicate for MEDIA_ROOT in settings.py.

Accessing the image in the template:

<img src="{{ MEDIA_URL }}{{ image.imgfile.name }}"></img>

The urls can be tricky, but here's a basic example of a simple url pattern to call the stored images:

urlpatterns += patterns('',
        url(r'^media/(?P<path>.*)$', 'django.views.static.serve', {
            'document_root': settings.MEDIA_ROOT,
        }),
   )

I hope it helps.

I don't want to involve forms in middle because I am not asking user to input the image.Can you please tell me bug in code which I pasted in question ? — Mayank Jain, Apr 23 '13 at 19:06
You claimed you don't know how to use ImageField, so I guided you on that. You didn't really specify what you're trying to do exactly, and your code is vague. If you're trying to save the image in a special way, you'll have to modify save(). — Mr_Spock, Apr 23 '13 at 19:33
sorry for vauge code...what i want is this..i am asking user to enter one url...i will extract all images of this url and then i have to save these extracted urls along with some other information in db... — Mayank Jain, Apr 23 '13 at 19:39

score 1 · Answer 5 · edited Dec 09 '17 at 04:52

Similar to @boltsfrombluesky's answer above you can do this in Python 3 without any external dependencies like so:

from os.path import basename
import urllib.request
from urllib.parse import urlparse
import tempfile

from django.core.files.base import File

def handle_upload_url_file(url, obj):
    img_temp = tempfile.NamedTemporaryFile(delete=True)
    req = urllib.request.Request(
        url, data=None,
        headers={
            'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'
        }
    )
    with urllib.request.urlopen(req) as response:
        img_temp.write(response.read())
    img_temp.flush()
    filename = basename(urlparse(url).path)
    result = obj.image.save(filename, File(img_temp))
    img_temp.close()
    return result

score 0 · Answer 6 · answered Apr 23 '13 at 19:48

Try doing it this way instead of assigning path to the image...

    import urllib2
    from django.core.files.temp import NamedTemporaryFile
    def handle_upload_url_file(url):
        img_temp = NamedTemporaryFile()
        opener = urllib2.build_opener()
        opener.addheaders = [('User-agent', 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120427 Firefox/15.0a1')]
        img_temp.write(opener.open(url).read())
        img_temp.flush()
        return img_temp

use the above function like this..

    new_image = images_data()
    #rest of the data in new_image and then do this.
    new_image.image.save(slug_filename,File(handle_upload_url_file(url)))
    #here slug_filename is just filename that you want to save the file with.

score 0 · Answer 7 · answered May 05 '21 at 16:22

In case you are saving image by overriding models' save method to modify the name of file and struggling with random invalid filename(like me) in django. You can follow up below code (Copied from Accepted answer):

lf = tempfile.NamedTemporaryFile()


for block in response.iter_content(1024*8):

        if not block:
            break

        lf.write(block)
    lf.name = name.  # Set your custom file name here
    dc = ImageFile(file=files.File(lf))

    dc.file.save()

I have configured my storage with django-storages, in order to directly upload media content to s3. For some reasons I wasn't able to replace file name. After some R&D it worked.

Note: I have used FileField in the model, hence few line of code is not needed

score -2 · Answer 8 · answered Dec 16 '13 at 11:25

-2

def qrcodesave(request): 
    import urllib2;   
    url ="http://chart.apis.google.com/chart?cht=qr&chs=300x300&chl=s&chld=H|0"; 
    opener = urllib2.urlopen(url);  
    mimetype = "application/octet-stream"
    response = HttpResponse(opener.read(), mimetype=mimetype)
    response["Content-Disposition"]= "attachment; filename=aktel.png"
    return response

answered Dec 16 '13 at 11:25

Saurabh Chandra Patel

12,712
6
88
78

What you've done there is describe how to force the user's browser to download the image. OP wants to save the image to an `ImageField` in his database. – Michael Bates Apr 27 '17 at 06:55

Download a remote image and save it to a Django model

8 Answers8

Linked