Postgres full text search over json field

Question

I can not understand why index work but I got empty result set.

https://www.db-fiddle.com/f/n9SyXK6GY3va2CZ41jNGQ5/2

I have table:

create table content
(
    id bigserial not null constraint content_pk primary key,
    created_at timestamp with time zone not null,
    form json not null
);

Field form store data in format:

{
  "title_text": "Test test",
  "content": {
    "blocks": [
      {
        "key": "6j131",
        "text": "Lorem ipsum dolor sit amet,"
      },
      {
        "key": "6nml9",
        "text": "In tincidunt tincidunt porttitor."
      }
    ],
  }
}

I tried create index to search by value from title_text and from concatination of all nodes content->blocks[]->text.

My queries:

(function by sample of https://www.facebook.com/afiskon thak you)

CREATE OR REPLACE FUNCTION make_tsvector(title TEXT, content json)
  RETURNS tsvector AS
'
BEGIN
    RETURN (setweight(to_tsvector(''simple'', title), ''A'')
    || setweight(to_tsvector(''simple'', STRING_AGG(content ->> ''text'', '' '')), ''B''));
END
'
    LANGUAGE 'plpgsql' IMMUTABLE;

(create index query)

DROP INDEX IF EXISTS idx_content__form__title_text_and_block_text;
CREATE INDEX IF NOT EXISTS idx_content__form__title_text_and_block_text
  ON content
    USING GIST (make_tsvector(
                            content.form ->> 'title_text',
                            content.form -> 'content' -> 'blocks'
                    ));

(and check of my query with EXPLAIN)

EXPLAIN
  SELECT c.id, c.form ->> 'title_text'
  FROM content c,
     json_array_elements(c.form -> 'content' -> 'blocks') block
  WHERE make_tsvector(
                  c.form ->> 'title_text',
                  c.form -> 'content' -> 'blocks'
          ) @@ to_tsquery('ipsum')
  GROUP BY c.id;

and I see index works (!)

HashAggregate  (cost=15.12..15.15 rows=2 width=40)
Group Key: c.id
->  Nested Loop  (cost=4.41..14.62 rows=200 width=64)
    ->  Bitmap Heap Scan on content c  (cost=4.41..10.62 rows=2 width=64)
          Recheck Cond: (make_tsvector((form ->> 'title_text'::text), ((form -> 'content'::text) -> 'blocks'::text)) @@ to_tsquery('ipsum'::text))
          ->  Bitmap Index Scan on idx_content__form__title_text_and_block_text  (cost=0.00..4.40 rows=2 width=0)
                Index Cond: (make_tsvector((form ->> 'title_text'::text), ((form -> 'content'::text) -> 'blocks'::text)) @@ to_tsquery('ipsum'::text))
    ->  Function Scan on json_array_elements block  (cost=0.01..1.01 rows=100 width=0)

but if I use this query I will get empty result.

Is it problem of STRING_AGG call in index build function?

Hey, man! Have you managed to solve this issue? I'm currently struggling with something very similar. Could you please share your solution? — TrejGun, Jun 25 '21 at 02:36
@TrejGun hi! I can't remember project where I use it. But if I understand right, I should create new immutable function MY_STRING_AGG that wraps std func STRING_AGG. If I find project I drop the solution. — GFB, Jun 26 '21 at 00:59

score 0 · Answer 1 · answered Sep 11 '19 at 17:06

Take a closer look at this snippet of your code here.

make_tsvector(
  c.form ->> 'title_text',
  c.form -> 'content' -> 'blocks'
)

You're not selecting what you think.

c.form -> 'content' -> 'blocks'

Returns a JSON array, not the individual elements. On the other hand in your function, you have this (escaped quotes removed for clarity):

content ->> 'text'

The JSON you are passing in isn't an object; it's an array of objects. Therefore the lookup fails because the path query is wrong.

The reason the planner reports that your index is being used is because both the index and your query are pointing to the same invalid path. Since they match, the index is used. Doesn't mean the index holds useful info though.

Find a way to iterate through the array either in the function or in the query calling the function and it should start working.

Oh, thanks. I can understand how write func with json[] right, so I make immutable version of string_agg and use it for create index. — GFB, Sep 11 '19 at 19:38

score 0 · Answer 2 · answered Jul 01 '21 at 07:17

For @GFB who has already forgotten this nightmare and for those who are still seeking an answer for "how to search in JSON array" especially in Draft.js output

CREATE TABLE IF NOT EXISTS content
(
    id bigserial not null constraint content_pk primary key,
    created_at timestamp with time zone not null,
    form json not null
);

INSERT INTO content (created_at, form) 
VALUES 
('2021-06-25', '{"blocks": [{"key": "6j131","text": "Lorem ipsum dolor sit amet,"},{"key": "6nml9","text": "In tincidunt tincidunt porttitor."}]}'),
('2021-06-25', '{"blocks": [{"key": "6j131","text": "hello world"},{"key": "6nml9","text": "hello Dolly"}]}')
;


SELECT c.*
FROM content c
LEFT JOIN LATERAL json_array_elements(c.form->'blocks') blocks ON TRUE
WHERE blocks->>'text' ILIKE '%hello%'
GROUP BY id;


SELECT c.*
FROM content c, json_array_elements(c.form->'blocks') blocks
WHERE blocks->>'text' ILIKE '%hello%'
GROUP BY id;

you can try this solution here http://sqlfiddle.com/#!17/6ca7c/21

PS more about CROSS JOIN LATERAL you can read at this thread

Postgres full text search over json field

2 Answers2