5

I have an otherwise perfectly relational data schema in place for my Postgres 8.4 DB, but I need the ability to associate arbitrary key/value pairs with several of my tables, with the assigned keys varying by row. Key/value pairs are user-generated, so I have no way of predicting them ahead of time or wrangling orderly schema changes.

I have the following requirements:

  • Key/value pairs will be read often, written occasionally. Reads must be reasonably fast.
  • No (present) need to query off of the keys or values. (But it might come in handy some day.)

I see the following possible solutions:

  1. The Entity-Attribute-Value pattern/antipattern. Annoying, but the annoyance would be generally offset by my ORM.
  2. Storing key/value pairs as serialized JSON data on a text column. A simple solution, and again the ORM comes in handy, but I can kiss my future self's need for queries good-bye.
  3. Storing key/value pairs in some other NoSQL db--probably a key/value or document store. ORM is no help here. I'll have to manage the separate queries (and looming data integrity issues?) myself.

I'm concerned about query performance, as I hope to have a lot of these some day. I'm also concerned about programmer performance, as I have to build, maintain, and use the darned thing. Is there an obvious best approach here? Or something I've missed?

Community
  • 1
  • 1
David Eyk
  • 12,171
  • 11
  • 63
  • 103

3 Answers3

7

That's precisely what the hstore datatype is for in PostgreSQL.
http://www.postgresql.org/docs/current/static/hstore.html

It's really fast (you can index it) and quite easy to handle. The only drawback is that you can only store character data, but you'd have that problem with the other solutions as well.

Indexes support "exists" operator, so you can query quite quickly for rows where a certain key is present, or for rows where a specific attribute has a specific value.

And with 9.0 it got even better because some size restrictions were lifted.

  • Very interesting. Character data is no problem. – David Eyk Jan 20 '11 at 13:36
  • I'll go ahead and accept this, as it seems quite reasonable. I now have a related hstore question over at http://stackoverflow.com/questions/4776563/how-do-i-install-the-hstore-module-on-postgresql-9-0-macports-install – David Eyk Jan 23 '11 at 21:19
1

hstore is generally good solution for that, but personally I prefer to use plain key:value tables. One table with definitions, other table with values and relation to bind values to definition, and relation to bind values to particular record in other table.

Why I'm against hstore? Because it's like a registry pattern. Often mentioned as example of anti pattern. You can put anything there, it's hard to easy validate if it's still needed, when loading a whole row (in ORM especially), the whole hstore is loaded which can have much junk and very little sense. Not mentioning that there is need to convert hstore data type into your language type and convert back again when saved. So you get some overhead of type conversion.

So actually I'm trying to convert all hstores in company I'm working for into simple key:value tables. It's not that hard task though, because structures kept here in hstore are huge (or at least big), and reading/writing an object crates huge overhead of function calls. Thus making a simple task like that "select * from base_product where id = 1;" is making a server sweat and hits performance badly. Want to point that performance issue is not because db, but because python has to convert several times results received from postgres. While key:value is not requiring such conversion.

Drachenfels
  • 3,037
  • 2
  • 32
  • 47
0

As you do not control data then do not try to overcomplicate this.

create table sometable_attributes (
  sometable_id int not null references sometable(sometable_id),
  attribute_key varchar(50) not null check (length(attribute_key>0)),
  attribute_value varchar(5000) not null,
  primary_key(sometable_id, attribute_key)
);

This is like EAV, but without attribute_keys table, which has no added value if you do not control what will be there.

For speed you should periodically do "cluster sometable_attributes using sometable_attributes_idx", so all attributes for one row will be physically close.

Tometzky
  • 22,573
  • 5
  • 59
  • 73