9

I'm building a website for work, and one of the more important features is a rich content grid displaying data. By default, it only displays 20 items per page, but we have ~200 items in the database that can be filtered, sorted, and searched.

Our sales and marketing team has also requests a "list all" feature so they can display all of the data in one place and scroll through rather than page through the data.

Flexigrid data

This entire system is built using ASP.Net MVC on the server side, jQuery and Flexigrid on the client side, and uses JSON to exchange data between the two via AJAX.

I've gotten the actual data transfer part pretty solid. A page of 20 results takes 800ms for the entire request (POST a request to the server via Flexigrid and get the response). It's more the client-side processing that takes a while.

I could offload some of the client-side processing to the server. But this would make the server-side operation take longer and make the size of the document returned that much larger. Not a problem in situations with a high-speed Internet connection ... but that's not necessarily the case.

The other option I have is to download as much data as possible and shift the majority of the data processing to the client. This cuts the request time down to basically nil (only fetching changed elements rather than the entire data set). It will work pretty well on machines with fast CPUs and a lot of RAM, but that's not necessarily the case, either.

Since at least one person flagged this as "not a real question," let me clarify ...

  • What can I possibly do to alleviate the client-side processing time issues without moving so much processing to the server that I end up with a data transfer time issue?
  • What are some best practices when it comes to balancing client-side processing with server-side processing?
  • Is it better to err on the side of the server or the client?
  • What are some tools I can use to better optimize these exchanges so that things don't continue to go awry?
EAMann
  • 4,128
  • 2
  • 29
  • 48

3 Answers3

2

What are you processing on the client side that is taking so long? Processing a JSON object (even a very large one) should not be too intensive.

A lot of DOM look ups when writing your data client side could slow things down. Reducing DOM lookups can greatly help performance. I believe good practice for balancing server & client side processing is to error on the server. Since the server is under your control you can always choose to upgrade your server. Keeping the majority of processing on the server will also make things easier for mobile devices and older computers.

You should utilize AJAX & client side capabilities in a way that enhances the user experience. Load and process data as it is requested by the users. By loading only what they request you can decrease the load on your server & client.

If you are also requesting the same sort of data over and over you can look in to both server & client side caching. By utilizing caching you can reduce request times and/or bandwidth.

Kevin M
  • 1,524
  • 17
  • 38
  • The server sends a JSON object to the client. The JSON object is then loaded into Flexigrid. Flexigrid itself reads the object into a table, then does some expensive `document.createElement()` operations to format the table for display. It's converting the object into an HTML table that's so intensive. – EAMann Nov 02 '11 at 15:37
1

As it turns out, the problem is more with the JavaScript engines on the client side than the data I'm working with. I've spent much of the day benchmarking the process and timing various operations.

Everything runs quickly in Chrome. Everything runs pretty fast (thought not as fast as Chrome) in Firefox. The real performance laggard is Internet Explorer.

When I load the entire data set - all 200 rows - Flexigrid attempts to do some post-processing on every cell in the table. As you can see in the screenshot, each row has 29 cells ... so we're looking through a bunch of formatting operations a total of 5800 times.

I was able to pull some of the more expensive operations (i.e. creating jQuery objects) out of the lower-level cell loop so they're only run once per row, but ultimately I'm still running into IE-related performance issues.

To give you some real-world benchmarks, I set the code to spit out the total time before it hits certain events:

  • populate fires when the browser first sends off the XHR request
  • addData fires after the request has returned and before the JSON object is parsed
  • addCellProp fires after the initial parsing of data and iterates through each cell in the table
  • done fires when everything is finsihed

Processing 20 rows of data (default):

------------------------------------------------------
| browser | populate | addData | addCellProp | done  |
------------------------------------------------------
| Chrome  | 0        | 84      | 123         | 286   |
| IE9     | 0        | 151     | 309         | 799   |
| IE8     | 0        | 226     | 481         | 1105  |

Processing the full data set (179 rows on this machine):

------------------------------------------------------
| browser | populate | addData | addCellProp | done  |
------------------------------------------------------
| Chrome  | 0        | 318     | 669         | 1963  |
| IE9     | 0        | 157     | 1813        | 9428  |
| IE8     | 0        | 229     | 2188        | 13335 |

The most expensive operation is between addCellProp and done. I've gone through and made the code as efficient as possible, but there's only so much you can do when you're running through that many iterations of a data set, particularly when manipulating the DOM.

I've modified Flexigrid (despite many recommendations not to) to touch the DOM as little as possible and that's actually sped things up quite a bit. When I started this research, IE9 would take between 20 and 30 seconds to hit the done event.

The unfortunate truth here is that not all platforms are created equal, and IE doesn't seem to be the best engine for working with data within the display in this fashion.

A better approach might be to create and manipulate the HTML table on the server side and send the entire thing (markup and all) to the browser when requested for IE users rather than depending on IE to create the markup from a raw JSON object.

EAMann
  • 4,128
  • 2
  • 29
  • 48
  • The cross browser benchmarking for each event was really informative. It really helps lends clarity to the root of the problem. Formatting server side for IE users sounds like a solid idea. – Kevin M Nov 03 '11 at 04:42
  • I went through and rewrote things so that the entire table is sent in the AJAX request. In the original system, only the data was sent and JS was used to process it and populate a table. On the one hand, I'm now sending 10x the info in the XHR response. On the other hand, the page loads reliably 10x as fast in IE. – EAMann Nov 03 '11 at 23:19
0

What can I possibly do to alleviate the client-side processing time issues without moving so much processing to the server that I end up with a data transfer time issue?

Is the data coming out of a database? If so restrict the data there. This is what the db is good at. I use flexigrid and keep all the paging sorting and filtering there. The db only returns the required data sorted and filtered as requested. All the server has to do is return it and all the client has to do is render it.

What are some best practices when it comes to balancing client-side processing with server-side processing?

Keep client side light as possible

Is it better to err on the side of the server or the client?

Yes server has much more power

What are some tools I can use to better optimize these exchanges so that things don't continue to go awry?

IE developer tools use the network tab to see what's coming over the wire

Leigh Ciechanowski
  • 1,297
  • 17
  • 33
  • That's not the problem. Paging, sorting, and filtering *are* all done on the server. But when the data set is 200+ rows of 30+ columns, the client-side rendering slows down *significantly* on IE. It has a lot to do with poorly-coded event handlers in Flexigrid, not so much the data itself. – EAMann Dec 22 '11 at 14:57
  • If the data is restricted server side then the client will never receive such a large dataset. e.g. the client will only receive a page of data. When the user clicks next page the flexigrid calls back to the server and gets the next page. Unless you are wanting to show 200+ rows on one page??? – Leigh Ciechanowski Dec 22 '11 at 15:04
  • Pagination is set up by default to limit to 5, 10, 15, or 20 results on one page. But as I called out in the original question, the sales/marketing team specifically requested a "view all" option to display all of the data on a single page. It was the performance when doing a "view all" that I was attempting to debug. – EAMann Dec 22 '11 at 17:33