In need of a data source that is less cumbersome than XML

Question

I'm currently working on a project that will be taking in a large amount of data over a set period of time. I currently do not have access to any database engine on the network my software will be deployed on. As such, I'm working with an XML dataset, and employing an MVC type framework. I chose XML originally because it can mimic to a decent degree the relational design of a database. I store the data in a singleton and my win-forms access the data via objects in my singleton and the win-forms do not have direct knowledge of the underlying data structure.

I'm using .NET 3.5, as the PC's my software will be deployed on are XP computers and not internet connected, and therefore, do not have access to download .NET 4.0. I'm accessing the XML using LINQ to XML objects and storing the element/attribute values into objects and collections that can be accessed through properties and methods in my data access layer.

My main concern is that as the XML file gets larger, the overhead could become overly cumbersome. Other than setting up a database server, would there be a better data storage system to use that would be better than XML? I need my data to follow a relational format for what I'm attempting to accomplish. I've already looked into using delimited/csv formats and those do not satisfy my project requirements.

EDIT: This datasource cannot be embedded into the application, and there will be multiple applications accessing the datasource. There are 2 applications that will be used, a "host" application that will be managing the data and a "client" application which will be used for data collection. The "client" application will have multiple instances over multiple workstations on the LAN.

I would embed a no-sql instance, like [Raven](http://ravendb.net/) — theor, Nov 28 '12 at 15:27
well you have access databases, ini files, home-rolled text files. multiple xml files instead of 1, etc. — Sam I am says Reinstate Monica, Nov 28 '12 at 15:27
have you considered json, or something like MongoDB, or SQLite — Antarr Byrd, Nov 28 '12 at 15:28
+1 for an "embedded SQL" (not necessarily NoSQL - especially if relations are important to you). BTW, you can even embed SQL express http://channel9.msdn.com/posts/SQL11UPD03-REC-08 — Lorenzo Dematté, Nov 28 '12 at 15:33
(and this is valid also for older engines: http://msdn.microsoft.com/en-us/library/bb264562%28v=sql.90%29.aspx) — Lorenzo Dematté, Nov 28 '12 at 15:35

score 3 · Accepted Answer · answered Nov 28 '12 at 15:43

3

SQL Server Express LocalDB is an embedded version of SQL server, but requires only a file on the machine running it. This might be a good option, will be smaller, faster, and easier to work with than XML, will have identical syntax and similar performance to a real SQL server and you can use all of the built in .Net SQL libraries and features without installing a third party data store.

You can also have multiple apps connect to the same DB file, and still get ACID features of SQL server.

http://www.microsoft.com/sqlserver/en/us/editions/2012-editions/express.aspx

http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx

answered Nov 28 '12 at 15:43

user15741

1,392
16
27

I would have the data file on a single machine, but there would be multiple machines in my LAN connecting to it. If I understand you correctly, I would have to install Sql Express on the "client" machines, as well as the "host" machine that has the file, is that correct? – Loren Shaw Nov 28 '12 at 16:06
1

LocalDB would be installed with your client application. If you have multiple simultaneous clients accessing relational data in a central data store you might just want to use a real DB like SQL Express, which should require no client apart from built-in .Net libraries. If you've got several machines accessing and updating an XML file on a network location you must be having a lot of contention issues. – user15741 Nov 28 '12 at 19:30
I'm working around the contention issues. I'll have to talk to the project requester and see about getting a database server setup, as it is the best solution. The current machines we have will not adequately run Sql Server. I'm not even sure they will run Sql Express without causing performance issues with some of the other software that is on these machines. – Loren Shaw Nov 29 '12 at 16:26

score 2 · Answer 2 · answered Nov 28 '12 at 15:28

2

If file size is your primary concern, you could use compression on the files to keep them smaller on disk. .NET 3.5 has some compression algorithms built in. XML is very repetitive text, which compresses very well. If you are free to break your data up into many files, that many also help keep the sizes down.

answered Nov 28 '12 at 15:28

Dave Swersky

34,502
9
78
118

1

Unfortunately, my main concern isn't the file size, though that could become an issue also. My biggest concern is memory overhead. – Loren Shaw Nov 28 '12 at 15:31

score 1 · Answer 3 · edited May 23 '17 at 12:15

1

JSON is lighter-weight than XML. As for whether you can use it in a relational fashion , that would depend on exactly what you are trying to do. Some relational uses of JSON have been covered here:

Using JSon like a Relational SQL Database (Javascript)

edited May 23 '17 at 12:15

Community

1
1

answered Nov 28 '12 at 15:29

RonaldBarzell

3,822
1
16
23

In need of a data source that is less cumbersome than XML

3 Answers3