querying in-memory collection

Question

I have a collection, a List of Map, the map is Map<String, String>. I need to query this collection with comparison, logical, like, not in operators. Something like SQL.

I will be populating the list from a database stored procedure, so I am not sure about the size. But, I guess the size should not be more than 10,000 records.

While posting this I am having a look at Apache functors, I don't know if they will help here.

Other way that I am thinking of is using the in-memory database Derby to achieve this.

Please let me know of any Java library or any other way of doing this.

The maps in the list will be like below:

Map<String, String> m1 = new Map<String, String>();
m1.put("name","Mark");
m1.put("age","21");
m1.put("city","some city");

Map<String, String> m1 = new Map<String, String>();
m1.put("name","David");
m1.put("age","25");
m1.put("city","other city");

I need to query the list to get Map which has:

name=Mark
name=Mark and age > 30
city not in "other city"

Define "better". Also, what is the size of the collection, and how frequently do you need to query it? — NPE, May 29 '12 at 18:43
I am still not fully clear on what you want to do. Perhaps a few examples might be in order? — thkala, May 29 '12 at 18:45
I don't understand the design. If the collections are created and queried (and destroyed?) with every request, then why not do all your filtering at the point where you are reading the data and creating the collections? Why store all the data and then query? — theglauber, May 29 '12 at 19:10
@theglauber The stored procedures that I get data from, I don't have control over them. The only thing I can use the stored procedures is to get data, rest all the data massaging has to be done at java side. — Abdullah Shaikh, May 29 '12 at 19:20
It looks like you're creating your collections from what the stored procedures return, i.e. the SPs are returning something else (ResultSet?) that you then use to create the collections. If you're querying those collections only once, you should be able to inspect what the SPs are returning, and if it doesn't satisfy your query, discard it and move to the next record. But if you are making more than one query per data set, then it makes sense to store everything and query the collections. You may be able to simplify your task through designing collections that support your queries well. — theglauber, May 29 '12 at 19:27

score 3 · Answer 1 · answered May 06 '14 at 14:10

Take a look at CQEngine - Collection Query Engine.

I'm biased as the author, but this will outperform and scale better than approaches which rely on iterating and filtering.

It provides an implementation of java.util.Set (IndexedCollection) which allows objects to be retrieved based on (arbitrarily complex) queries on their attributes.

score 2 · Answer 2 · edited May 23 '17 at 12:33

2

You can use Google Collections/Gauva. It provides collection filtering with predicates as :

Iterables.filter(Iterable, Predicate)

See this answer for an example of filtering using predicates.

edited May 23 '17 at 12:33

Community

1
1

answered May 29 '12 at 19:36

Suraj Chandran

24,433
12
63
94

1

or [Functional Java](http://functionaljava.org/) or [Jedi](http://jedi.codehaus.org/) or someone else already mentioned LambdaJ. – Kevin Welker May 29 '12 at 19:42

score 0 · Answer 3 · answered May 29 '12 at 18:52

0

Quaere can query collections. You should have a look before going for an in-memory db based solution.

answered May 29 '12 at 18:52

Hakan Serce

11,198
3
29
48

score 0 · Accepted Answer · answered Jul 07 '12 at 07:51

I found JFilter which helps you with filtering in-memory data (java objects ). Below is the description from JFilter website.

JFilter is a simple and high performance open source library to filter (query), map (select) and reduce (aggregate) objects in a Java collection. Query is given in json format like Mongodb queries.

I have written a post on using JFilter with Maps, which I am using as a generic way of storing data and querying/filtering using JFilter

Here is the link to the post Using JFilter to query/filter in-memory data

querying in-memory collection

4 Answers4

Linked