How do I read an image using Rawpy image processing library, from a URL?

Question

How do I read an image using the rawpy library, from a url?

I have tried

filepath = www.google.com/image.jpeg

im = rawpy.imread(filepath)

but it doesn't work.

I looked deeper into rawpy's code and it says it takes "path or file".

I have also tried other methods like using tempfile, an in memory object, BytesIO, but none of it works.

I would appreciate some help on this please.

Edit One:

In regards to the urllib.request library, these were the errors I got:

i)

import rawpy
import io
import urllib

filepath = 'https://filesamples.com/samples/image/jpeg/sample_640%C3%97426.jpeg'

data = urllib.request.urlopen(filepath).read()
stream = io.BytesIO(data)
im = rawpy.imread(stream)

with the resultant error

LibRawFileUnsupportedError                Traceback (most recent call last)
<ipython-input-88-7965dccc6908> in <module>
     35 stream = io.BytesIO(data)
---> 36 im = rawpy.imread(stream)
     37 
     38 """

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/rawpy/__init__.py in imread(pathOrFile)
     16     d = RawPy()
     17     if hasattr(pathOrFile, 'read'):
---> 18         d.open_buffer(pathOrFile)
     19     else:
     20         d.open_file(pathOrFile)

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_buffer()

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawFileUnsupportedError: b'Unsupported file format or not RAW file'

ii)

import rawpy
import io
import urllib
import tempfile
import shutil

filepath = 'https://filesamples.com/samples/image/jpeg/sample_640%C3%97426.jpeg'

with urllib.request.urlopen(filepath) as response:
    with tempfile.NamedTemporaryFile(delete = False) as tmp_file:
        shutil.copyfileobj(response,tmp_file)
        im = rawpy.imread(tmp_file)

with the resultant error

LibRawIOError                             Traceback (most recent call last)
<ipython-input-89-ae0405bcc04a> in <module>
     42         shutil.copyfileobj(response,tmp_file)
---> 43         im = rawpy.imread(tmp_file)
     44 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/rawpy/__init__.py in imread(pathOrFile)
     16     d = RawPy()
     17     if hasattr(pathOrFile, 'read'):
---> 18         d.open_buffer(pathOrFile)
     19     else:
     20         d.open_file(pathOrFile)

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_buffer()

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawIOError: b'Input/output error'

iii)

import rawpy
import io
import urllib
import tempfile
import shutil

filepath = 'https://filesamples.com/samples/image/jpeg/sample_640%C3%97426.jpeg'

with urllib.request.urlopen(filepath) as response:
    data = response.read()
    im = rawpy.imread(data)

with the resultant error

AttributeError                            Traceback (most recent call last)
<ipython-input-90-f10bfdef4333> in <module>
     55     data = response.read() 
---> 56     im = rawpy.imread(data)
     57 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/rawpy/__init__.py in imread(pathOrFile)
     18         d.open_buffer(pathOrFile)
     19     else:
---> 20         d.open_file(pathOrFile)
     21     return d

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_file()

AttributeError: 'bytes' object has no attribute 'encode'

Edit Two: Other stuff I've tried that didn't use the urllib library were

i) in memory file

import io
import rawpy 

filepath = 'https://filesamples.com/samples/image/jpeg/sample_640%C3%97426.jpeg'

response = requests.get(filepath)
in_mem_file = io.BytesIO(response.content)
im = rawpy.imread(in_mem_file)

with the resultant error

LibRawFileUnsupportedError                Traceback (most recent call last)
<ipython-input-91-1ef650bdc042> in <module>
     48 response = requests.get(filepath)
     49 in_mem_file = io.BytesIO(response.content)
---> 50 im = rawpy.imread(in_mem_file)
     51 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/rawpy/__init__.py in imread(pathOrFile)
     16     d = RawPy()
     17     if hasattr(pathOrFile, 'read'):
---> 18         d.open_buffer(pathOrFile)
     19     else:
     20         d.open_file(pathOrFile)

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_buffer()

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawFileUnsupportedError: b'Unsupported file format or not RAW file'

ii) smart open

from smart_open import open
import rawpy 

filepath = 'https://filesamples.com/samples/image/jpeg/sample_640%C3%97426.jpeg'

def get_file(filepath):
    with open(filepath, 'rb') as s3_source:
        return s3_source
        
s3_source = get_file(filepath)
im = rawpy.imread(s3_source)

with the resultant error

LibRawIOError                             Traceback (most recent call last)
<ipython-input-92-03b94616ed66> in <module>
     55         return s3_source
     56 s3_source = get_file(filepath)
---> 57 im = rawpy.imread(s3_source)

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/rawpy/__init__.py in imread(pathOrFile)
     16     d = RawPy()
     17     if hasattr(pathOrFile, 'read'):
---> 18         d.open_buffer(pathOrFile)
     19     else:
     20         d.open_file(pathOrFile)

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_buffer()

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawIOError: b'Input/output error'

Edit Three (Solution):

Ok I managed to resolve it by using raw image format data (.dng).

Here is my code

import rawpy
import BytesIO
import imageio

filepath = 'www.google.com/sample_file.dng'
response = requests.get(filepath)
raw = rawpy.imread(BytesIO(response.content))
rgb = raw.postprocess()
imageio.imsave('test_rawpy.jpeg', rgb)

You might consider adding the `python` tag. And putting back the `import` statements. And surrounding strings with quotes. And using sensible URLs. And adding error messages... — Mark Setchell, Jan 27 '22 at 09:42
*And* researching how to fetch a file from the web server into a `bytes` object... you might have heard of `urllib.request.urlopen`? then you'd have solved your own question. you'll need to dig into that `rawpy` thing and find out how it can decode data from a `bytes` object so you don't have to needlessly store the web server's response to a file, only to load it again in the next line. — Christoph Rackwitz, Jan 27 '22 at 11:32
I've tried the urllib.request, and have added the errors I got from it to the question. How do I find how rawpy can decode data from a bytes object? I've searched google and can't find any article/stack overflow article on it. — CoderMath, Jan 27 '22 at 12:00

score 0 · Accepted Answer · answered Jan 27 '22 at 12:20

0

JPEG is not a Raw Image Format. You need to send some raw data as input.

So,

If you just want to process some JPEGs, try Pillow.
If you want to process raw images, change your input data.

answered Jan 27 '22 at 12:20

Trey Cai

1,195
1
12
18

It worked, thanks. I used a raw image format (.dng) and manage to read and save it. I've added the solution to the question. – CoderMath Jan 27 '22 at 13:16

How do I read an image using Rawpy image processing library, from a URL?

1 Answers1