0

I have a matrix of data (96cols by >5000rows), where each value is between -1 and +1. I've written a script to generate a heatmap in SVG format, but the SVG file size with my real datasets becomes massive. Inkscape struggles, with rendering and screen updating really slow. Compressing it to svgz doesn't help either.

I went with the approach of drawing a rect for each value, with each rect having 5 attributes: x, y, width, height and fill. As you can imagine, with 96x5000 values, that's a lot of rects. I reduced the file size from ~34MB to ~29MB by turning each attribute into an integer. The other idea I had was to remove the fill attribute and replace it with one of 10-20 class, one for each colour group. I haven't done that last bit yet though, as I think there must be an even better way of doing it.

As an example:-

    <rect fill="#000000" height="14" width="10" x="50" y="81" />

In HTML, I'd use a <table>, with <tr's> and <th's> with a short class name or id referencing the colour group. If the value is 0, or the middle value, no class name would be needed and the <th> could inherit a preset neutral colour.

With SVG though, apparently there's no formal table specification, so how should I go about this? I'm currently considering the following options:-

  1. The display property does have a (poorly-documented) inline-table option. Apparently though, it could be "an accidental remnant from CSS". I guess this would mean having a <g> for each row, with an id referencing a style with display:table-row-group.
  2. Use <text> elements as rows, with nested <tspan>s, as suggested here.
  3. Use a layout manager written in ECMASCript.

I've also briefly checked out XSLT, which looks powerful, but complicated, and also seems like it would only work in a web browser. I'd ideally like to keep everything in a single file that can be saved, viewed and edited in a vector graphics editor, and am not sure if XSLT allows for this.

What would be the most efficient way of doing this? Filesize isn't so important as the speed at which it renders. Any help much appreciated.

UPDATE:-

  • 2 wouldn't work as text's and nested tspan's have no fill or background-color attribute. The example uses rect's to provide the background colour, as text's and tspan's don't provide any method for creating a fill.

I also wondered whether line length might have something to do with the slow rendering.. I'm using Python's (c)ElementTree module to generate the SVG, and by default it writes all XML without a single new-line character. Might it help parsers / renderers to separate each of the image's rows into a new line in the XML?

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
Alex Leach
  • 1,199
  • 2
  • 12
  • 26
  • see this question http://stackoverflow.com/q/2369492/1301710 for generating a heatmap with matplotlib. you can save the figure to svg with matplotlib. don't know if that is what you are searching for. – bmu May 22 '12 at 20:56

1 Answers1

2

I suspect you may be better off using canvas for this. Create a canvas and then draw rectangles on the canvas. The rectangles will only exist as part of the canvas bitmap.

Robert Longson
  • 118,664
  • 26
  • 252
  • 242
  • I would like to start using the canvas, but I wouldn't know how to then save as an SVG. I imagine there's libraries for that, but atm I'm not even generating HTML, so canvas seems inappropriate. Thanks for the suggestion though – Alex Leach May 07 '12 at 11:03
  • 1
    You've not given any reason for using SVG. If all you want to do is display something data driven that looks and acts like a bitmap with coloured squares then canvas is what you want. Anything else is a square peg in a round hold unless you have other undisclosed requirements. – Robert Longson May 07 '12 at 15:05
  • The figures have to be suitable for offline viewing, editing and printing, primarily. At some point I'll embed this in a web app, but at the moment, I've developed this as a stand-alone module for python's svgplotlib. I've already done the work, I just need to know how to make it fast! – Alex Leach May 07 '12 at 17:25
  • You can view canvas offline, and print it. How easy editing is depends on what changes you envisage editing would make. – Robert Longson May 07 '12 at 18:04