0

I have a utility class that translates objects into DataTables and vice versa. We're having problems loading large amounts of data when converting from a DataTable to a List of Objects. We use a custom attribute to determine and relate column information. Here's the pseudocode:

For each row in the table
    For each property in an object
        For each attribute on that property
            If the attribute is our column information attribute
                Grab the data from the table and insert the value into the objects property
            End
        End
    End
End

For DataTable results that have hundreds of rows, though, this process is taking minutes... and that's simply unacceptable in a web app.

So, my question is: Is there some easy way to translate a DataTable and a .NET (custom) data object back and forth that doesn't require a lot of reflection (which is probably where all the overhead is in this case)?

Edit: Turns out it was another issue within the data object itself. Still, I did optimize the loader a bit with the reflection calls, so thank you all.

qJake
  • 16,821
  • 17
  • 83
  • 135
  • Maybe you should post the actual code -- your problem could be that you are doing the reflection calls for every single row, instead of caching some of it. See the answers to this question for some ideas: http://stackoverflow.com/questions/1204748/cache-reflection-results-class-properties . – patmortech Jun 27 '11 at 18:03
  • And check out this post about ValueInjector as a mapper between datatables and objects: http://stackoverflow.com/questions/5462671/valueinjecter-and-datatable – patmortech Jun 27 '11 at 18:13
  • I'm legally unable to post the code, but I was not caching my reflection calls. I'm trying that now, hopefully it'll speed it up enough to where we'll get tolerable page load times. – qJake Jun 27 '11 at 18:22

1 Answers1

0

Does it have to be dynamic? If not, you can use LINQ to DataSets:

from tr in table.AsQueryable()
select new MyObject
{
   A = tr.Field<string>("A"),
   .
   .
}

Otherwise, it has to be reflection dependent. Make sure you aren't getting the type and property info within the loop, as that can take some time. Same thing with the attributes: consider building an index of the properties that have the attribute before the loop, then use this index of property names to get the properties that you need. This way, the only reflection is the setting of values into the object.

Alternatively, you can try to offload work by using the async feature in ASP.NET, or consider using the Parallel extensions to use multi-core processing for the conversion.

Brian Mains
  • 50,520
  • 35
  • 148
  • 257
  • Yeah, unfortunately it has to be dynamic... this is a utility feature, and it has been completely abstracted. – qJake Jun 27 '11 at 18:00
  • OK, then move the reflection on the object out of the loops, and do it only once. That's a good start. – Brian Mains Jun 27 '11 at 18:50
  • Yep, I just did that and it sped it up by a fairly decent amount. Thanks for the suggestion. – qJake Jun 27 '11 at 18:58