Storing large amount of data as XML for faster and efficient querying

Question

I want a temporary data storage where I can store student information grouped by number of students in a class. As using an RDMS is not an option so for now I have decided in favor of storing it as an XML. In a structure similar to this.

<ListOfStudents>
    <ClassSet noOfStudents="1">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>
    <ClassSet noOfStudents="2">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>
    <ClassSet noOfStudents="3">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>
    <ClassSet noOfStudents="4">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>
    <ClassSet noOfStudents="5">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>
…
…

<ClassSet noOfStudents="50065">
        <Student first="asf" last="asf">
        <Student first="asf" last="asf">
        ...
    </ClassSet>

</ListOfStudents>

The basic purpose is to be able to retrieve all students of a class with specified no of students. For e.g. querying all students of class with noOfStudents equal to 93.

First will I be able to retrieve/filter XML nodes like this i.e. by specifying an attribute value, will the whole XML will be parsed and searched everytime?
How efficient solution is it? Keep in mind the number of <ClassSet> nodes can reach up to 1000 or more and each having hundreds of students.
if not very efficient, what other options I can have while the limitation of not being able to use a RDMS or any embedded database in place?

-1 Your question boils down to "I want to store my data in an Database, but not an RDBMS". Since "efficient storage of generic data" means "database". So instead of writing off the entire solution space to your problem, perhaps you should describe why you don't want to use an RDBMS. — Aron, Oct 28 '16 at 06:46

score 1 · Answer 1 · edited May 23 '17 at 12:33

If you use an XDocument it will be parsed once into objects. However, the performance would be better if you wrote classes to hold this data and then read/wrote that data by serializing it.

This is because to find a property value your code will go directly to a property value ...Select(x => x.Age >= 18) whereas with the XDocument your code will have to look up the value from a list ...Select(x => x.Element("Age").Value >= 18)

Take a look here for an example Select XElement where child element has a value

Why can't you use a DB? If there is a lot of data then in-memory is not going to be a good solution. There are embedded DBs you could use (SQLite, RavenDB) so that you don't need to install a server, these will keep data on disk and just cache most recently used indexes.

Storing large amount of data as XML for faster and efficient querying

1 Answers1