2

I need to take a Pillow image and either convert it to a byte array or pack it somehow to send it over a ZeroMQ socket. Since the other socket connection isn't in python (it's c#), I can't use pickle, and I'm not sure JSON would work since it's just the image (dimensions sent separately). I'm working with an image created from a processed numpy array out of opencv, so I don't have an image file. Sending the numpy array over didn't work, either.

Since I can't read the image bytes from a file, I've tried Image.getdata() and Image.tobytes() but I'm not sure either were in the right form to send over the socket. What I really need is a bytestream that I can reform into an array after crossing the socket.

UPDATE: So now I'm specifically looking for anything easily undone in C#. I've looked into struct.pack but I'm not sure there's an equivalent to unpack it. Turning the image into an encoded string would work as long as I could decode it.

UPDATE 2: I'm thinking that I could use JSON to send a multidimensional array, then grab the JSON on the other side and turn it into a bitmap to display. I'm using clrzmq for the C# side, though, and I'm not sure how that handles JSON objects.

rllogan
  • 21
  • 4
  • Sending a byte array is very inefficient. Think about the dimensions of your image and realise a 1080p image is 6MB. Are you actually hoping to send continuous frames like video, because if you do 30fps, you suddenly have exceeded the bandwidth of a solid Gigabit Ethernet. Have you considered sending a JPEG? – Mark Setchell Jul 23 '19 at 16:45
  • You can have a look here at a way to store an image (Numpy array) into Redis, and you could do something similar with ZeroMQ... https://stackoverflow.com/a/55313342/2836621 – Mark Setchell Jul 23 '19 at 16:51
  • @Mark I have not--I was hoping to avoid saving images and reloading them as the sockets are inter-process on the same EEPROM machine, and I don't want to wear out the memory. If there's a way to create a jpeg image without saving it to file, that would work. If I saved the image, I would just reload it on the C# side and ditch the socket entirely in this part of the application. – rllogan Jul 23 '19 at 16:51
  • Sure, just create. `BytesIO` and write into that... https://stackoverflow.com/a/52281257/2836621 See the 3 lines after the comment starting `# Encode...` – Mark Setchell Jul 23 '19 at 16:58
  • Looking into the redis reply on that other question, I see you use struct.pack with a key--would I have to use the direct equivalent in c# to unpack the array? As long as I can get a byte array to put into a memorystream, I can get a bitmap to display. – rllogan Jul 23 '19 at 17:04
  • There isn't an easy way to send a jpeg across ZeroMQ, so I'm looking into the C# equivalent of struct.unpack. Thanks for your help, and let me know if you have further insight – rllogan Jul 23 '19 at 17:16
  • Since efficiency is not the main concern here, how about encoding it to base64, then decoding back on receiving end? – altunyurt Jul 27 '19 at 11:39

1 Answers1

0

In case anyone was wondering, here's what I ended up doing:

On the python side: 

_, buffer = cv2.imencode('.jpg', cFrame ) 
jpg_enc = base64.b64encode(buffer).decode('utf-8')

self.frameSender.send_string(jpg_enc)
self.frameSender.send_string(str(height))
self.frameSender.send_string(str(width))

On the C# side:

byte[] bytebuffer = Convert.FromBase64String(frameListener.ReceiveFrame().ReadString());
int height = int.Parse(frameListener.ReceiveFrame().ReadString());
int width = int.Parse(frameListener.ReceiveFrame().ReadString());
using (MemoryStream ms = new MemoryStream(bytebuffer)) //image data
{
     Bitmap img = new Bitmap(Image.FromStream(ms), new Size(height, width));
     pictureboxVideo.BeginInvoke((MethodInvoker)delegate{pictureboxbVideo.Image = img;});
}
rllogan
  • 21
  • 4