I need to store business data from mysql into bolt. The data is a map[string]string looks like this:
{"id": "<uuid>", "shop_id":"12345678", "date": "20181019"... }
Since the amount of data will be huge and increasing, apart from splitting the data into standalone files (such as 201810.db), I would like to make the final file as small as possible. So I plan to encode the data myself, by using "lookup tables", i.e. since all keys will be same for all data items, I map id=1, shop_id=2,... so that the key will only consume 1~2 bytes. And for values, I also do the same encoding, so that columns which are highly redundant (i.e. select distinct only return a few results) will consume less space in the bolt file.
Now my question is: how does bolt stores keys and values? If I use the method described above, will bolt store more objects "per page" so that eventually space efficiency can be improved?
Or, since it utilize "pages" so that even storing one byte of data will consume a full page? If this is the case, do I have to manually group a bunch of objects until their combined size is larger than a bolt page in order to make it "fully filled"? Of course, this will hurt random access, but for my application, this can be overcome at the expense of increased coding complexity.
