25

This question has been asked in a similar way here but the answer was way over my head (I'm super new to python and web development) so I'm hoping there's a simpler way or it could be explained differently.

I'm trying to generate an image using matplotlib and serve it without first writing a file to the server. My code is probably kind of silly, but it goes like this:

import cgi
import matplotlib.pyplot as pyplot
import cStringIO #I think I will need this but not sure how to use

...a bunch of matplotlib stuff happens....
pyplot.savefig('test.png')

print "Content-type: text/html\n"
print """<html><body>
...a bunch of text and html here...
<img src="test.png"></img>
...more text and html...
</body></html>
"""

I think that instead of doing pyplot.savefig('test.png'), I am supposed to create a cstringIO object and then do something like this: 

mybuffer=cStringIO.StringIO()
pyplot.savefig(mybuffer, format="png")

But I am pretty lost from there. All the examples I've seen (e.g. http://lost-theory.org/python/dynamicimg.html) involve doing something like 

print "Content-type: image/png\n"

and I don't get how to integrate that with the HTML I'm already outputting.

Community
  • 1
  • 1
Ben S.
  • 3,415
  • 7
  • 22
  • 43
  • When you output the blob from `imageout.py` , then in the HTML can you set image source to the py script? In php you would do something along those lines. – ninMonkey Feb 12 '13 at 03:07
  • 1
    Check out http://stackoverflow.com/questions/1207190/embedding-base64-images – Alex L Feb 12 '13 at 05:41

6 Answers6

21

You should

  • first write to a cStringIO object
  • then write the HTTP header
  • then write the content of the cStringIO to stdout

Thus, if an error in savefig occured, you could still return something else, even another header. Some errors won't be recognized earlier, e.g., some problems with texts, too large image dimensions etc.

You need to tell savefig where to write the output. You can do:

format = "png"
sio = cStringIO.StringIO()
pyplot.savefig(sio, format=format)
print "Content-Type: image/%s\n" % format
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) # Needed this on windows, IIS
sys.stdout.write(sio.getvalue())

If you want to embed the image into HTML: 

print "Content-Type: text/html\n"
print """<html><body>
...a bunch of text and html here...
<img src="data:image/png;base64,%s"/>
...more text and html...
</body></html>""" % sio.getvalue().encode("base64").strip()
Thorsten Kranz
  • 12,492
  • 2
  • 39
  • 56
13

The above answers are a little outdated -- here's what works for me on Python3+ to get the raw bytes of the figure data.

import matplotlib.pyplot as plt
from io import BytesIO
fig = plt.figure()
plt.plot(range(10))
figdata = BytesIO()
fig.savefig(figdata, format='png')

As mentioned in other answers you now need to set a 'Content-Type' header to 'image/png' and write out the bytes.

Depending on what you are using as your webserver the code may vary. I use Tornado as my webserver and the code to do that is:

self.set_header('Content-Type', 'image/png')
self.write(figdata.getvalue())
Mike N
  • 6,395
  • 4
  • 24
  • 21
  • I am having trouble getting the correct content length for the buffer. When I download an image via ajax, I always get an `ERR_CONTENT_LENGTH_MISMATCH` error, having tried `sys.getsizeof(figdata)` and `figdata.__sizeof__()`. I ended up not specifying content length at all. – TomTom101 Jan 08 '17 at 21:47
  • 1
    @TomTom101 - What library are you using for your HTTP server? You probably want `len(figdata.getvalue())` (you were getting the size of the BytesIO object, not the actual data stream) – Mike N Jan 09 '17 at 08:18
  • Using nginx, Gunicorn and Falcon. I'll try your suggestions – TomTom101 Jan 09 '17 at 08:24
9

what works for me with python3 is:

buf = io.BytesIO()
plt.savefig(buf, format='png')
image_base64 = base64.b64encode(buf.getvalue()).decode('utf-8').replace('\n', '')
buf.close()
trulio
  • 93
  • 1
  • 3
5

My first question is: Does the image change often? Do you want to keep the older ones? If it's a real-time thing, then your quest for optimisation is justified. Otherwise, the benefits from generating the image on the fly aren't that significant.

The code as it stands would require 2 requests:

  1. to get the html source you already have and
  2. to get the actual image

Probably the simplest way (keeping the web requests to a minimum) is @Alex L's comment, which would allow you to do it in a single request, by building a HTML with the image embedded in it.

Your code would be something like:

# Build your matplotlib image in a iostring here
# ......
#

# Initialise the base64 string
#
imgStr = "data:image/png;base64,"

imgStr += base64.b64encode(mybuffer)

print "Content-type: text/html\n"
print """<html><body>
# ...a bunch of text and html here...
    <img src="%s"></img>
#...more text and html...
    </body></html>
""" % imgStr

This code will probably not work out of the box, but shows the idea.

Note that this is a bad idea in general if your image doesn't really change too often or generating it takes a long time, because it will be generated every time.

Another way would be to generate the original html. Loading it will trigger a request for the "test.png". You can serve that separately, either via the buffer streaming solution you already mention, or from a static file.

Personally, I'd stick with a decoupled solution: generate the image by another process (making sure that there's always an image available) and use a very light thing to generate and serve the HTML.

HTH,

Laur Ivan
  • 4,117
  • 3
  • 38
  • 62
2

Unless I badly miscomprehend your question, all you need to do is cd to the location of the image and run: python -m SimpleHTTPServer 8000 &

Then open your browser, and type http://localhost:8000/ in the URL bar.

Alptigin Jalayr
  • 719
  • 4
  • 12
1

I know I'm a bit late to the party here, but I had this same problem and ended up with the small script below.

This python 3.6+ code:

  • Starts a web server and tells you where to view it
  • Scans itself for class methods beginning with 'plot_' and provides the browser with a list of plots
  • For a clicked plot, prompts for required parameters (if any), including an automatic refresh period (in seconds)
  • Executes the plot and refreshes

As you can tell by the code, it is deliberately minimal for temporary diagnostics and monitoring (of machine learning progress in my case).

You may need to install any dependencies (plac + any other libs needed for plotting e.g. I use pandas, matplotlib)

You can run the file via double click (no parameters) or command line (with/without parameters)

Code:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import io
from http.server import HTTPServer,BaseHTTPRequestHandler
import urllib
import inspect


class PlotRequestHandler(BaseHTTPRequestHandler):

    def do_GET(self):
        args = urllib.parse.parse_qs(self.path[2:])
        args = {i:args[i][0] for i in args}
        html = ''

        if 'mode' not in args:
            plots = ''
            for member in dir(self):
                if member[:5] == 'plot_':
                    plots += f'<a href="http://{self.server.server_name}:{self.server.server_port}/?mode=paramcheck&graph={member}">{member[5:].replace("_"," ").title()}</a><br/>\n'
            html = f'''<html><body><h1>Available Plots</h1>{plots}</body></html>'''

        elif args['mode'] == 'paramcheck':
            plotargs = inspect.getargspec(getattr(self,args['graph'])).args
            if len(plotargs) == 1 and plotargs[0].lower()=='self':
                args['mode'] = 'plotpage'
            else:
                for arg in plotargs:
                    if arg.lower() != 'self':
                        html += f"<input name='{arg}' placeholder='{arg}' value='' /><br />\n"
                html = f"<html><body><h1>Parameters:</h1><form method='GET'>{html}<input name='refresh_every' value='60' />(Refresh in sec)<br /><input type='hidden' name='mode' value='plotpage'/><input type='hidden' name='graph' value='{args['graph']}'/><input type='submit' value='Plot!'/></form></body></html>"

        if 'mode' in args and args['mode'] == 'plotpage':
            html = f'''<html><head><meta http-equiv="refresh" content="{args['refresh_every']};URL=\'http://{self.server.server_name}:{self.server.server_port}{self.path}\'" /></head>
                       <body><img src="http://{self.server.server_name}:{self.server.server_port}{self.path.replace('plotpage','plot')}" /></body>'''

        elif 'mode' in args and args['mode'] == 'plot':
            try:
                plt = getattr(self,args['graph'])(*tuple((args[arg] for arg in inspect.getargspec(getattr(self,args['graph'])).args if arg in args)))
                self.send_response(200)
                self.send_header('Content-type', 'image/png')
                self.end_headers()
                plt.savefig(self.wfile, format='png')
            except Exception as e:
                html = f"<html><body><h1>Error:</h1>{e}</body></html>"

        if html != '':
            self.send_response(200)
            self.send_header('Content-type', 'text/html')
            self.end_headers()
            self.wfile.write(bytes(html,'utf-8'))

    def plot_convergence(self, file_path, sheet_name=None):
        if sheet_name == None:
            data = pd.read_csv(file_path)
        else:
            data = pd.read_excel(file_path, sheet_name)

        fig, ax1 = plt.subplots()

        ax1.set_xlabel('Iteration')
        ax1.set_ylabel('LOSS', color='tab:red')
        ax1.set_ylim([0,1000])
        ax1.plot(data.iteration, data.loss, color='tab:red')

        ax2 = ax1.twinx()

        ax2.set_ylabel('Precision, Recall, f Score')
        ax2.set_ylim([0,1])
        ax2.plot(data.iteration, data.precision, color='tab:blue')
        ax2.plot(data.iteration, data.recall, color='tab:green')
        ax2.plot(data.iteration, data['f-score'], color='tab:orange')

        fig.tight_layout()
        plt.legend(loc=6)
        return plt


def main(server_port:"Port to serve on."=9999,server_address:"Local server name."=''):
    httpd = HTTPServer((server_address, server_port), PlotRequestHandler)
    print(f'Serving on http://{httpd.server_name}:{httpd.server_port} ...')
    httpd.serve_forever()


if __name__ == '__main__':
    import plac; plac.call(main)
QA Collective
  • 2,222
  • 21
  • 34