2

In my application, I want to return a collection of objects from a WCF Service (hosted as a Windows Service) to populate a DataGrid in a WPF application. The amount of objects in the collection ranges from one to several hundred depending on the method called.

I'm curious as what is the 'best' way to deal with returning large collections from a service.

These are the options I've seen suggested are:

  • Increasing the max message size and returning all the objects in one go. This seems like a bad idea because there could possibly come a time when I need to return more than 2GB of data.
  • Paginating the records and calling the method repeatedly until all objects have been retrieved. I've seen this one suggested for ASP.NET projects but I don't know how well it would work for desktop apps.
  • Using a stream. To be honest, I don't understand how this works since it appears to be meant for transferring large single objects rather than many smaller ones.
  • Doing something with the yield keyword, but this went over my head and I couldn't follow it. :-/

What would be the best way approach this task, and why?

Jake
  • 1,701
  • 3
  • 23
  • 44
  • it is extremely unlikely users will ever use 2 gb worth of data in a single datagrid. There are several UI options (e.g. search, pagination, infinite scrolling) available to deal with presenting large datasets to users. I would try tackling the problem from a UI point of view, by finding a better way of presenting the data to the user. The scenario itself (big dataset on the backend, user at the frontend) is pretty common. – Dirk Trilsbeek Feb 15 '16 at 14:20

1 Answers1

5

Increasing the max message size and returning all the objects in one go. This seems like a bad idea because there could possibly come a time when I need to return more than 2GB of data.

Definitely not the good choice, unless your're sure that your data will never exceed the new limit you set. Otherwise you will just push the problem back and have it again in a few months. 2Gb is already a lot by the way (think how long your user will wait)

Paginating the records and calling the method repeatedly until all objects have been retrieved. I've seen this one suggested for ASP.NET projects but I don't know how well it would work for desktop apps.

The most common and obvious approach, you can use pagination and only query for a defined number of elements on each page. I can't understand your question about "desktop apps" though? The relevant concept here is client/server. Your client (desktop app) need to query your server for the content of a page (if you use pagination) to display. If your client was a web page, the concept would still be relevant.

Using a stream. To be honest, I don't understand how this works since it appears to be meant for transferring large single objects rather than many smaller ones.

I guess you read things like "manage you own stream". In a few words you could consider any stream as a bit flow, and just interpret it as you wish in your client side. I would certainly not recommend that, unless you have really specific transfer issue (and having a high number of object to transfer is certainly not specific enough). Having a few very big object to transfer may be specific enough, but even here I would challenge the implementation before going this way.

Doing something with the yield keyword, but this went over my head and I couldn't follow it. :-/

Sorry I don't follow you here, but yield is only syntaxic sugar so I don't think it's relevant to solve your problem. Still have a look to understad the concept: What is the yield keyword used for in C#?

Community
  • 1
  • 1
Ouarzy
  • 3,015
  • 1
  • 16
  • 19
  • I'll expand on the second and fourth point. Regarding the desktop apps, you can't really paginate a DataGrid control, so my understanding was that you just repeatedly call the method until there's nothing new to return. For point 4, one of the methods it talked about appeared to be using the yield keyword to return objects one at a time from the service. I didn't really read too much into it. – Jake Feb 15 '16 at 14:12
  • "you can't really paginate a DataGrid control"? You can always find a way to do it, but I agree it's less native than with web views. See http://stackoverflow.com/questions/784726/how-can-i-paginate-a-wpf-datagrid – Ouarzy Feb 15 '16 at 16:03
  • And I don't see how returning object one at a time is a good solution for you, So I insist you don't need yield to solve your problem. – Ouarzy Feb 15 '16 at 16:04
  • @Quarzy. I'm pulling 250 at a time with the current method I'm testing (similar to your link). Need to do a bit more experimenting, but so far it's looking promising. Sorry for the delay on the answer, busy night last night. – Jake Feb 16 '16 at 07:58