Parameterized IN clause using multiple columns

Question

I have a query along these lines, where I am trying to filter the result set by comparing tuples (like SQL multiple columns in IN clause):

select *
from mytable
where (key, value) in (values
 ('key1', 'value1'),
 ('key2', 'value2'),
 ...
);

This is valid syntax and works fine on my PostgreSQL 9.3 database.

I want to invoke this query through Spring JDBC where the in value pairs come from a List<Map<String, String>>.

It would be nice to do something like this:

List<Map<String, String>> valuesMap = ...;
String sql = "select * from mytable where (key, value) in (values :valuesMap)";
SqlParameterSource params = new MapSqlParameterSource("valuesMap", valuesMap);
jdbcTemplate.query(sql, params, rowMapper);

When I try this, I get:

org.postgresql.util.PSQLException: No hstore extension installed.
    at org.postgresql.jdbc2.AbstractJdbc2Statement.setMap(AbstractJdbc2Statement.java:1707) ~[postgresql-9.3-1101-jdbc41.jar:na]
    at org.postgresql.jdbc2.AbstractJdbc2Statement.setObject(AbstractJdbc2Statement.java:1910) ~[postgresql-9.3-1101-jdbc41.jar:na]
    at org.postgresql.jdbc3g.AbstractJdbc3gStatement.setObject(AbstractJdbc3gStatement.java:36) ~[postgresql-9.3-1101-jdbc41.jar:na]
    at org.postgresql.jdbc4.AbstractJdbc4Statement.setObject(AbstractJdbc4Statement.java:47) ~[postgresql-9.3-1101-jdbc41.jar:na]
    at org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:427) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]
    at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:235) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]
    at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:150) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]
    at org.springframework.jdbc.core.PreparedStatementCreatorFactory$PreparedStatementCreatorImpl.setValues(PreparedStatementCreatorFactory.java:287) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]
    at org.springframework.jdbc.core.PreparedStatementCreatorFactory$PreparedStatementCreatorImpl.createPreparedStatement(PreparedStatementCreatorFactory.java:244) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:623) ~[spring-jdbc-4.2.3.RELEASE.jar:4.2.3.RELEASE]

I've looked at the the hstore extension it mentions. It doesn't seem relevant to my problem.

Is there a way to accomplish this without dynamically building the SQL and parameter list?

score 15 · Accepted Answer · edited Feb 05 '16 at 19:32

15

All you have to do is to pass a list of arrays, where each array contains a key and value, like this:

HashMap<String , String > map = new HashMap<>();
map.put("key0", "value0");
map.put("key1", "value1");
Set<String> keys = map.keySet();
List<String[]> valuesMap = new ArrayList<>();
for(String key:keys){
    String[] entry = {key,map.get(key)};
    valuesMap.add(entry);
}
String sql = "select * from mytable where (key, value) in (values :valuesMap)";
SqlParameterSource params = new MapSqlParameterSource("valuesMap", valuesMap);
jdbcTemplate.query(sql, params, rowMapper);

This is mentioned in the Spring documentation: http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/jdbc.html#jdbc-in-clause

edited Feb 05 '16 at 19:32

Alex Wittig

2,800
1
33
42

answered Feb 05 '16 at 18:59

achabahe

2,445
1
12
21

That's exactly what I was looking for, thanks. In hindsight, Object[] makes more sense than a Map anyway. – Alex Wittig Feb 08 '16 at 21:54
Just to be clear, in the code sample the final list will contain two arrays: [key1, value1] [key0, value0] – Ena May 23 '18 at 13:37
If like me you were here to do the same thing in jpa, see this: https://stackoverflow.com/questions/37190664/jpa-criteria-in-clause-with-many-columns – Ena May 23 '18 at 14:12

Lukas Eder · Answer 2 · 2016-02-05T18:18:57.250

Unfortunately, there isn't any easy way to bind a nested collection bind variable to PostgreSQL. You could generate the following SQL string instead

SELECT *
FROM mytable
WHERE (key, value) IN (
  (?, ?),
  (?, ?),
  ...
);

That's a bit of work to keep the SQL string and the variable bindings in sync. You could, however, encode the map as JSON as such:

SELECT *
FROM mytable
WHERE (key, value) IN (
  SELECT 
    t->>'key', 
    t->>'value'
  FROM json_array_elements(?) AS t(v)
)

E.g.

SELECT *
FROM mytable
WHERE (key, value) IN (
  SELECT 
    t->>'key', 
    t->>'value'
  FROM json_array_elements(
    '[{"key":"key1","value":"value1"}, 
      {"key":"key2","value":"value2"}]'
  ) AS t(v)
)

In that case, you would only ever need a single VARCHAR bind variable

score 1 · Answer 3 · answered Feb 05 '16 at 20:10

1

If you can't get your solution to work, you could also just concatenate the key and value. Perhaps JDBC has less problems with this more basic syntax:

select *
from mytable
where (key||value) in (
 ('key1value1'),
 ('key2value2'),
 ...
);

For this to work, you'd need to first convert your Java Map to a List with the key and values concatenated as well.

answered Feb 05 '16 at 20:10

Bartez

172
8

It's probably not a bad idea to use a separator between key and value, which doesn't appear in the data. E.g. `key || '###' || value` (until it *does* appear in the data, of course) – Lukas Eder Feb 06 '16 at 14:57

score -1 · Answer 4 · edited May 23 '17 at 12:26

It might not be an issue with the query, it might be that you don't have any hstore created/installed.

I suggest the following steps to debug your problem:

Try a very simple query, without any parameters.
If you get the same issue, check how to create extensions: http://www.postgresql.org/docs/9.1/static/sql-createextension.html
Otherwise, if the query executed correctly, try to use a simple parameter (with =?)
Finally, try named queries. Something like:

ParsedSql parsedSql = NamedParameterUtils.parseSqlStatement(namedSql);
List<Integer> parameters = new ArrayList<Integer>();
for (A a : paramBeans)
    parameters.add(a.getId());
MapSqlParameterSource parameterSource = new MapSqlParameterSource();
parameterSource.addValue("placeholder1, parameters);
// create SQL with ?'s
String sql = NamedParameterUtils.substituteNamedParameters(parsedSql, parameterSource);
return sql;

Also check this discussion, I find it useful: How to execute IN() SQL queries with Spring's JDBCTemplate effectivly?

Parameterized IN clause using multiple columns

4 Answers4

Linked