1

I know basic SQL, and SQL is all I know when it comes to storing and retrieving data. I want to create 1 .exe and it should contain all ~100,000 key-value pairs (i have the data in .txt files) and maybe an extra attribute for description (this I would add myself - like a note to myself).

I also would like to write it in a new language I don't know yet; like python or C# (I have made desktop apps written in Java & VB.net all with SQL databases). So language will not be an issue and I would appreciate suggestions.

These key-value pairs might not need to be updated and I'm willing to re-compile/repackage the code to make 1 change in the data. The key is 6 letters long and 2 numbers at the end like hxnaaa01. Each of these letters represent or describe something about itself so I would also need to search for a specific letter on a specific position to get exactly what I need.

I know that regex would work well with what I need but all I mentioned is all I know. I don't know enough and I don't know what keywords to google.

I have read about XML and CSV. I don't really know what they are and I'm not sure how all of this would fit in 1 executable.

To summarize, I need:

  • 1 executable (Windows Desktop App)
  • Search function ~100k KVP+1more attribute (using regex?)
  • no database
  • with GUI
  • ability to add a "note" to each KVP
  • should be fast and lightweight
xyzit
  • 89
  • 1
  • 8
  • Have you considered JSON? It allows you to store data as objects, containing properties and values (key/values), you can then decide how you want to organize it, add extra properties (description for ex). Or even SQLite? Physical files don't often lend themselves well to fast or lightweight though. – Isolated May 11 '20 at 01:32
  • @Isolated I just read from this other [question](https://stackoverflow.com/questions/45583052/can-you-bundle-sqlite-in-your-exe-file) that there is no possibility of having a persistent db when using just 1 file (for SQLite at least). There has to be another file other than the .exe. Is this correct? Are there any other way? Would JSON do it in 1 .exe file? – xyzit May 11 '20 at 01:47
  • I would trade it being a few MBs large as long as it is lightning fast. – xyzit May 11 '20 at 01:50
  • JSON isn't a database, it's a format, so providing your programming language can support it, then there shouldn't be an issue. As you posted a link referencing C#, I'll imagine you use it. I'd recommend having a look here: https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to you can't avoid there being additional files though, if you can't have any files at all other than the .exe then in-memory is your only option, or remote. – Isolated May 11 '20 at 01:51
  • @Isolated Can't a JSON or a text file be packaged in an exe? Are you saying that for data to be persistent, like _starring_ (add to favorites) a specific KVP, would mean more than 1 file? – xyzit May 11 '20 at 02:01
  • I'll post my answer for your review, I think it'll cover most things. – Isolated May 11 '20 at 02:06

1 Answers1

0

1 executable (Windows Desktop App), no database

Data persistence will require either additional files, or a database. It's pretty much unavoidable, you can store data in memory, but it's only persisted for as long as it resides there.

You have another requirement: "fast and lightweight".

To achieve this requirement, you'll need to really think about your solution, what technology you use and how you can improve it in future.

Although searching through data is pretty trivial, an efficient solution is not. It requires upfront research into algorithms, data structures and general practices. (which is a rabbit hole itself).

In the case of JSON [1], you'll need to create an additional file to contain all your key/value pairs, you can use C# to create the extra file (on first launch, for example).

JSON promises to be lightweight, I tend to agree, some may not. When dealing with the filesystem, I think it can be agreed is often far from lightweight solution.

JSON is very readable though:

{
  "key": "value",
  "comment": "oh this is cool"
}

There's a lot of factors that play into something being fast and lightweight, so there's a need for some research on your part.

Honestly, depending on your experience, I wouldn't focus so much on the fast, I'd focus more on it working, then refactor that into something that's fast if it's too slow. [2]

And again, depending on your experience, I'd stick to opening the file, using a for/loop to find my key and do something with the data found, plus reward myself for having something that works.

TL;DR: you need either a file, or database for truly persistent storage, JSON or a remotely hosted MySQL would work. Try not to focus too much on fast before you have something that works.

Isolated
  • 2,167
  • 14
  • 14
  • Sure, hardcode all the data in an array or dictionary, and use functions to search over it, if you already know the data. Hardcoding 100k key/values will be very tedious though, but it would work. Is having the one file turn into two really that bad though? You can still deliver it as one executable, and you or buddies can remove the extra file if and when needed. – Isolated May 11 '20 at 02:38