How do I deal with dots in MongoDB key names?

Question

I'm developing this piece of software in Node and MongoDB in which I essentially want to store versions of packages with the following structure:

{
    "versions":
    {
        "1.2.3": { stuff }
    }
}

(similar to how npm does things in couch)

The issue is that when I updated MongoDB I discovered that it doesn't allow dots in key names (due to dot notation existing), causing my code to fail. After researching this, all I could find is that you need to transform the dots to some other character before storing in the db, then transform them back again when accessing. Is there really no better way to deal with this?

If there isn't, how can I do this transformation without copying the data over to another key and deleting the original?

Are you saying you already have data like that in Mongo? That should not have been possible even before the update. What version were you using? — Thilo, Jul 19 '12 at 01:25
@Thilo I don't actually remember, but it may have been the driver that was buggy and allowed it. — jli, Jul 19 '12 at 01:26
@c0deNinja It allows me to look up by version name without iterating through the whole array of potentially very many versions. — jli, Jul 19 '12 at 01:26
@Thilo yep, fortunately this hasn't gone into production yet — jli, Jul 19 '12 at 01:27
Then I think you do have to copy the data over to another key and delete the original (hoping that the current version of the database even lets you work with that invalid key). — Thilo, Jul 19 '12 at 01:29
@Thilo I may just scrap mongo completely and use another database, since this is a huge pain. — jli, Jul 19 '12 at 01:29
@jli, oic... but you can still look up the version without iterating through the array! — c0deNinja, Jul 19 '12 at 01:30
@c0deNinja This is being exposed as an API, I don't want to make API consumers have to do the transformation themselves. — jli, Jul 19 '12 at 01:31
@jli you can query for a specific key in the array and get the specific version out of the array... without iterating and without the costumer doing any transformation — c0deNinja, Jul 19 '12 at 01:33
@c0deNinja I'm exposing the entire object as shown in the question at one endpoint, and `stuff` at another endpoint. — jli, Jul 19 '12 at 01:39
@jli: What driver + version did you use to insert the dotted keys? Would be helpful to [file that as a bug](https://jira.mongodb.org/). — Stennie, Jul 19 '12 at 01:53
so does the entire object contain several different versions? or does each object represent one version? — c0deNinja, Jul 19 '12 at 01:54
@Stennie the native node driver, but it's been patched since then. — jli, Jul 19 '12 at 01:55
php driver fails updating such fields too, unfortunately I had to spend much time before discovering that dots were the problem. I will just replace the dot with underscore. — Melsi, Apr 22 '13 at 10:38
Possible duplicate of [How do you store keys in Mongodb with a '.' in them](http://stackoverflow.com/questions/2717065/how-do-you-store-keys-in-mongodb-with-a-in-them) — Dana the Sane, Oct 15 '15 at 15:43

score 4 · Answer 1 · edited Mar 05 '15 at 16:16

4

Can you use a collection of versions with stuff?

Like:

{
    "versions":
    [
              {
                   "version_num": "1.2.3",
                   "stuff": { stuff }
              },
              {
                   "version_num": "1.2.4",
                   "stuff": { stuff }
              }
    ]
}

edited Mar 05 '15 at 16:16

Adam Yost

3,616
23
36

answered Jul 19 '12 at 01:29

keaplogik

2,369
2
25
26

Well besides this being an invalid object, I'm trying to avoid having to iterate over an array here. – jli Jul 19 '12 at 01:30
Yes it is invalid, just wondering if an array would work. Okey – keaplogik Jul 19 '12 at 01:36

score 2 · Accepted Answer · answered Jul 19 '12 at 01:33

2

Dot restrictions are currently driver enforced, and not all drivers have prevented dots in field names since the beginning. You can write raw protocol code to do all sorts of crazy stuff in Mongo, including using really weird characters in collection names.

You will be much better off if you clean that up (probably replace dots with - or some other valid character), but it's going to be difficult to do it well with any kind of intelligent filtering. You will most likely need to iterate through the entire collection, munge the values in your app, and then overwrite the entire "versions" field in your doc. In place overwrites this should be reasonably speedy since they won't resize the doc, and likely won't change any indexes.

answered Jul 19 '12 at 01:33

MrKurt

5,080
1
23
22

Yeah that's what I figured.. one issue here is that users of this application can add any string as a version, so I don't have a standard to work with. Would the speed decrease from performing these operations on every single read outweigh the performance benefit of mongo compared to something like couch? I may just switch databases since the project is in very early development right now. – jli Jul 19 '12 at 01:38
Actually this seems like it'll be relatively fast upon testing, thanks. – jli Jul 19 '12 at 01:54
Do you actually need query within the versions of the doc? You could actually store a json string in "versions" and just parse it out by hand at load/view time. Doing string processing on reads probably won't slow you down much, read speed in Mongo is much faster than Couch. – MrKurt Jul 19 '12 at 01:55
I'm exposing basically everything as an API so unfortunately I have to parse it all right when it's loaded. I've already written the replacement routine so I guess I'll just go with that. – jli Jul 19 '12 at 01:59

How do I deal with dots in MongoDB key names?

2 Answers2