3

For reasons that don't make a lot of sense (Read: Not my decision) I need to keep a large number of rows, about ~90,000, in a DataTable and I do not have the option of using a database.

I need to be able to search the DataTable efficiently to find rows that match some basic criteria. For example, I might be looking at a row that has the value 2 in two specific columns.

What is the best way to do this?

Edit: Please take a look at https://chat.stackoverflow.com/transcript/message/62648#62648 for more details; after I work on this I will try and summarize the extra details from the chat here as well as provide my solution.

Community
  • 1
  • 1
  • 2
    What other limitations do you have? .NET 2.0 for instance? Are you allowed to use LINQ? – jcolebrand Oct 29 '10 at 19:38
  • Are you asking how to persist the data (Access, Excel, Flat file) or how to query the data efficiently? Are you going to build this DataTable every time your application runs or do you need to persist it somewhere? – Dismissile Oct 29 '10 at 19:45
  • May be you have memory problem, but do what you wanna to do and if there is a problem talk about it. – Saeed Amiri Oct 29 '10 at 19:52
  • for those following along on the home game (static SO only) you might want to review the transcript where this was discussed in chat (might make things easier to chew on? http://chat.stackoverflow.com/transcript/message/62648#62648) so it boils down to no LINQ and no fun stuff. Basic .NET 4.0 data processing on a DataTable – jcolebrand Oct 29 '10 at 19:54
  • @Dismissile, I am building the DataTable each time I run the program form a flat file. –  Oct 29 '10 at 20:33

2 Answers2

6

You could easily use DataTable.Select()

Justin Niessner
  • 242,243
  • 40
  • 408
  • 536
  • Given the requirements, this is the only way to do this. – NotMe Oct 29 '10 at 20:56
  • Correct, but not the entire solution when dealing with more than one selection criteria per row. –  Nov 03 '10 at 01:12
  • @Angelina - The Select method supports AND syntax for multiple filter criteria. You can read about the full set of supported syntax at http://msdn.microsoft.com/en-us/library/system.data.datacolumn.expression.aspx – Justin Niessner Nov 03 '10 at 02:10
1

The solution I ended up using for this painfully awkward and inconvenient situation was to use DataTable.Select(), populate a new DataTable and then use the same operation to select the rows I needed from the refined DataTable.

I think that this solution is clumsy, but then again the constraints on the problem were somewhat unrealistic seeing as I was on a tight schedule as well.