3

My code produces a zip file from Google App Engine for loading in a browser ie 10 KB < size < 1000 KB and I wonder if I can use memcache here or if the file is too large or already cached. I already use both memcache and set the cache control when generating the actual file first. 

class KMZHandler(webapp.RequestHandler):
    def add_file(self, zip_file, url, file_name):
        """Fetch url, and add content as file_name to the zip file."""
        result = urlfetch.fetch(url)
        if not result.content:
            return
        zip_file.writestr(file_name, result.content)

    def get(self):
        """Attempt to create a zip file."""
        # you could set 'count' like this:
        # count = int(self.request.get('count', 1000))

        zipstream = StringIO.StringIO()
        zip_file = zipfile.ZipFile(zipstream, "w")

        # repeat this for every URL that should be added to the zipfile
        url = 'http://www.koolbusiness.com/list.kml'
        self.add_file(zip_file, url, "list.kml")

        # we have finished with the zip so package it up and write the directory
        zip_file.close()

        # set the headers...

        self.response.headers["Cache-Control"] = "public,max-age=%s" % 86400
        self.response.headers['Content-Type'] ='application/zip'
        self.response.headers['Content-Disposition'] = 'attachment;filename="list.kmz"'

        # create and return the output stream
        zipstream.seek(0)
        self.response.out.write(zipstream.read())
        zipstream.close()

Here is the part that uses memcache and creates the actual file:

class KMLHandler(webapp.RequestHandler):
 def get(self):   
    self.response.headers["Cache-Control"] = "public,max-age=%s" % 86400
    start=datetime.datetime.now()-timedelta(days=20)
    count = int(self.request.get('count')) if not self.request.get('count')=='' else 1000        
    from google.appengine.api import memcache
    memcache_key = "ads"
    data = memcache.get(memcache_key)
    if data is None:
      a= Ad.all().filter("modified >", start).filter("url IN", ['www.koolbusiness.com']).filter("published =", True).order("-modified").fetch(count)
      memcache.set("ads", a)  
    else:
      a = data
    dispatch='templates/kml.html'
    template_values = {'a': a , 'request':self.request,}
    path = os.path.join(os.path.dirname(__file__), dispatch)
    output = template.render(path, template_values)
    self.response.headers['Content-Type'] = 'application/vnd.google-earth.kml+xml'
    self.response.headers['Content-Length'] = len(output)
    self.response.out.write(output)

Thanks for the answers

Niklas Rosencrantz
  • 25,640
  • 75
  • 229
  • 424
  • Should we know how large your file actually is? How should be give a recommendation if you don't provide the most obvious information about the size of the files? –  Apr 17 '11 at 08:41
  • I linked to the zip file whose size may vary. It is a KMZ file for loading in a browser and I thought maybe you knew the max value for files in these cases. – Niklas Rosencrantz Apr 17 '11 at 11:31

1 Answers1

2

If the file is under 1mb, and it does not change for each request, then memcaching the KMZ would likely reduce resource usage.

The KMZ file is not already in memcache, though it could be in a front-end cache. You are memcaching the results from the query when you generate the KML (see Nick's blog for a description of how to memcache entities), but the cache does not take different values of count or start into account. Tf that is not important, you might also consider directly memcaching the KML (or KMZ) file; if it is important you ned to fix the caching strategy.

Robert Kluin
  • 8,282
  • 25
  • 21