2

I have a web service that returns HTML for a table (for optimization purposes) in a string. One thing I am cautious about is the limit of data I am allowed to put in a string variable in C#, as well as any other possible limitations or warnings?

I read on online forums that string is quite accommodating and you can shove up to 1GB of data in it (which would certainly suffice for my purposes). Any input?

Leniel Maccaferri
  • 100,159
  • 46
  • 371
  • 480
sarsnake
  • 26,667
  • 58
  • 180
  • 286
  • 1
    Is there any reason you're building the whole thing as a single string rather than streaming chunks of it (e.g. rows) as you generate them? – jball Jan 04 '11 at 00:41
  • 1
    If you're returning that much HTML, you should definitely consider returning the raw data as JSON instead and then build the table with a client-side templating engine like jQuery Templates, jTemplates, etc. – Dave Ward Jan 04 '11 at 00:53
  • @Dave Ward, good call. Is there a particular engine you recommend for C#? – Anonymous Type Jan 04 '11 at 01:05
  • It depends more on what framework you're using on the client-side than what you're using on the server-side. jQuery Templates works great if you're using jQuery on the client-side (and of course, jQuery works great with ASP.NET/C#). – Dave Ward Jan 04 '11 at 01:11
  • Dave I am returning 600 rows of data, and I tried building a table on the client by building the table html inside a string and then feeding the string to a div. which is fast. The part that's slow is the looping though the raw JSON data, 600 rows of it...it takes 15 seconds. Is there a plugin that will allow me to optimize the looping part? – sarsnake Jan 04 '11 at 01:49
  • To paraphrase, does Jquery Templates provide good performance for large amounts of data? I found this http://encosia.com/2010/11/10/composition-with-jquery-templates-why-and-how/ on your blog. Would this be applicable here in terms of performance? Or is it intended to be used only with table that are few rows long? thanks – sarsnake Jan 04 '11 at 01:56
  • @gnomixa sounds like time for a new question mate. – Anonymous Type Jan 04 '11 at 04:38
  • @AnonymousType: for sure - see http://stackoverflow.com/questions/4590270/loading-large-data-in-jquery/4590337#4590337 – sarsnake Jan 04 '11 at 05:16

3 Answers3

3

A string -- or any other object -- can be up to 2GB in size in the Microsoft CLR. (This is only an implementation detail, not a specified standard. Other implementations might have different restrictions.)

Since strings are UTF-16 each character requires two bytes, which means that a string can contain a maximum of 1 Giga-characters.

LukeH
  • 263,068
  • 57
  • 365
  • 409
  • Isn't this subject to memory fragmentation? – jball Jan 04 '11 at 00:40
  • 1
    why should fragmentation be an issue? the data can/and is allocated to non-contingious blocks of memory. Fragmentation might reduce the overall performance aspect, but how would it reduce the availability of memory? – Anonymous Type Jan 04 '11 at 00:48
  • 1
    @Anonymous Type: I'm pretty sure that a string is laid out as a contiguous "string" of characters in memory (not sure whether this is a requirement of the CLI spec or just an implementation detail). If you don't have 2GB of contiguous memory available then you're not going to be able to allocate a 2GB string. – LukeH Jan 04 '11 at 01:20
  • 1
    @jball: Yes, this could be an issue (assuming that a string needs to be allocated as a single contiguous chunk, which I think is the case). Stating the obvious, but the 2GB max object size restriction will only be hit if/when nothing else is limiting the size of your object. – LukeH Jan 04 '11 at 01:25
2

Take a look here:

What is the maximum possible length of a .NET string?

Community
  • 1
  • 1
Leniel Maccaferri
  • 100,159
  • 46
  • 371
  • 480
2

Depending on the size of the payload you are sending over http (assuming your remote service is using this binding) you may wish to consider chunking the whole string down into smaller payloads on the server before sending across the wire, then reassembling it on the client.

One thing you should be aware of if you should investigate is the timeout value being used by the webservice, and whether it is appropriate for the time taken to return your data.

As other answers have said 1gb in theory for limit of string, but in practise this is user mode memory bound. So if your application has access to 2Gb of user mode address space, then you can keep using that till it runs out. Note this is not physical RAM, but total "per application" user mode memory available.

Anonymous Type
  • 3,051
  • 2
  • 27
  • 45