C# check conversion from List to single string using String.Join, is possible or not?

Question

I have one List<string> which length is undefined, and for some purpose I'm converting entire List<string> to string, so I want's to check before conversion that it is possible or not(is it gonna throw out of memory exception?) so I can process that much data and continue in another batch.

Sample

int drc = ImportConfiguration.Data.Count;
List<string> queries = new List<string>() { };

//iterate over data row to generate query and execute it
for (int drn = 0; drn < drc; drn++)//drn stands to Data Row Number
{
    queries.Add(Generate(ImportConfiguration.Data[drn], drn));

//SO HERE I WANT"S TO CHECK FOR SIZE
//IF IT"S NOT POSSIBLE IN  NEXT ITERATION THAN I'LL EXECUTE IT RIGHT NOW
//AND EMPTIED LIST AGAIN FOR NEXT BATCH

    if (drn == drc - 1 || drn % 5000 == 0)
    {
        SqlHelper.ExecuteNonQuery(connection, System.Data.CommandType.Text, String.Join(Environment.NewLine, queries));
        queries = new List<string>() { };
    }
}

yes you can , take a look at this http://stackoverflow.com/questions/559415/concat-all-strings-inside-a-liststring-using-linq — Thorarins, Apr 22 '16 at 07:38
How many items do you have in your list? Because if the list can hold them without an exception, the string should be able to do so too (unless the list takes up around 2 GB, because then the string will so to and the app cannot store both large objects in memory unless you run in 64-bit mode). — Visual Vincent, Apr 22 '16 at 07:38
@PatrickHofman: No executing it using traditional sqlclient, so i want's to make posssible batch dynamically. — Nilesh Moradiya, Apr 22 '16 at 07:39
And what is that supposed to mean? One byte? Did you mean 1GB? — Visual Vincent, Apr 22 '16 at 07:41
@VisualVincent: No number of item in List from which i need to convert it. — Nilesh Moradiya, Apr 22 '16 at 07:43
What does 1B mean then? One billion? Perhaps this can help: http://stackoverflow.com/questions/1128315/find-size-of-object-instance-in-bytes-in-c-sharp — Visual Vincent, Apr 22 '16 at 07:46

score 1 · Answer 1 · edited May 23 '17 at 11:59

You could try:

List<string> theList;

try {
    String allString = String.Join(",", theList.ToArray());
} catch (OutOfMemoryException e) {
    // ... handle OutOfMemoryException exception (e)
}

EDIT

Based on your comment.

You could give an estimation in the following way.

Get available memory: Take a look at this post

Get sum size of your list strings theList.Sum(s => s.Length);

List<string> theList = new List<string>{ "AAA", "BBB" };
// number of characters
var allSize = theList.Sum(s => s.Length);

// available memory
Process proc = Process.GetCurrentProcess();
var availableMemory = proc.PrivateMemorySize64;;

if (availableMemory > allSize) {
   // you can try
   try {
      String allString = String.Join(",", theList.ToArray());
   } catch (OutOfMemoryException e) {
      // ... handle OutOfMemoryException exception (e)
   }
} else {
   // it is not going to work...
}

As string is immutable, i can't afford conversion every time. — Nilesh Moradiya, Apr 22 '16 at 07:46

Eren Ersönmez · Answer 2 · 2016-04-22T07:56:53.583

1

I can't say it is not possible but I think a better way would be to do the join and catch any exceptions:

try
{
     var joined = string.Join(",", list);
}
catch(OutOfMemoryException)
{
     // join failed, take action (log, notify user, etc.)
}

Note: if the exception is happening, then you need to consider a different approach than using a list and joining.

edited Apr 22 '16 at 07:56

answered Apr 22 '16 at 07:41

Eren Ersönmez

38,383
7
71
92

*I want to check before conversion that it is possible or not* > This is all afterwards... I think it is better to solve the actual issue instead of just failing. – Patrick Hofman Apr 22 '16 at 07:45
As string is immutable, i can't afford conversion every time – Nilesh Moradiya Apr 22 '16 at 07:46
Acrually i'm batching operations, and i need this check because other 3rd party API is requiring string as input. – Nilesh Moradiya Apr 22 '16 at 07:47
1

@PatrickHofman catching the exception would be a step towards telling that string join is not the right approach here, if it is happening so often that it is not an "exception". In that case there is something wrong with the design of the whole thing. We can only speculate what that is. – Eren Ersönmez Apr 22 '16 at 07:54

score 1 · Answer 3 · answered Apr 22 '16 at 07:42

1

Since you are trying to send a large amount of text to a SQL Server instance, you could use SQL Server's streaming support to write the string to the stream as you go, minimizing the amount of memory needed to construct the data to send.

answered Apr 22 '16 at 07:42

Patrick Hofman

153,850
22
249
325

C# check conversion from List to single string using String.Join, is possible or not?

3 Answers3