21

I am working in Postgres 9.4 with the following table:

     Column      │         Type         │ Modifiers
─────────────────┼──────────────────────┼──────────────────────
 id              │ integer              │ not null default
 practice_id     │ character varying(6) │ not null
 date            │ date                 │ not null
 pct_id          │ character varying(3) │
 total_list_size │ double precision     │ not null
 star_pu         │ jsonb                │

I have the following query:

SELECT date,
       AVG(total_list_size) AS total_list_size,
       json_object_agg(key, val) AS star_pu
FROM (SELECT date,
             SUM(total_list_size) AS total_list_size,
             key, SUM(value::numeric) val FROM frontend_practicelist p,
             jsonb_each_text(star_pu)
       GROUP BY date, key ) p
GROUP BY date
ORDER BY date;

It gives me results with a JSON object attached to star_pu:

date            │ 2013-04-01
total_list_size │ 56025123.000000000000
star_pu         │ { "antidepressants_cost" : 180102416.8036909901975399, "antiepileptic_drugs_cost" : 296228344.171576079922216... }

Instead I would like to flatten the JSON result to a series of namespaced keys, so the result looks like this:

date                             │ 2013-04-01
total_list_size                  │ 56025123.000000000000
star_pu.antidepressants_cost     │ 180102416.8036909901975399
star_pu.antiepileptic_drugs_cost │ 296228344.171576079922216 
...

Is this possible?

klin
  • 112,967
  • 15
  • 204
  • 232
Richard
  • 62,943
  • 126
  • 334
  • 542
  • *Is this possible?* Not without explicitly defining your output columns. Even [crosstab](http://www.postgresql.org/docs/current/static/tablefunc.html) requires that. – pozs Feb 03 '16 at 13:47

1 Answers1

63

This particular case

The function below dynamically creates a view based on a table:

create or replace function create_totals_view(table_name text)
returns void language plpgsql as $$
declare
    s text;
begin
    execute format ($fmt$
        select string_agg(format('star_pu->>''%s'' "%s"', key, key), ',')
        from (
            select distinct key
            from %s, json_each(star_pu)
            order by 1
            ) s;
        $fmt$, '%s', '%s', table_name)
    into s;
    execute format('
        drop view if exists %s_view;
        create view %s_view as 
        select date, total_list_size, %s from %s', 
        table_name, table_name, s, table_name);
end $$;

First, create a table from your query.

create table totals as

    SELECT date,
           AVG(total_list_size) AS total_list_size,
           json_object_agg(key, val) AS star_pu
    FROM (SELECT date,
                 SUM(total_list_size) AS total_list_size,
                 key, SUM(value::numeric) val FROM frontend_practicelist p,
                 jsonb_each_text(star_pu)
           GROUP BY date, key ) p
    GROUP BY date
    ORDER BY date;

Next, use the function, which will create a view named after the table with _view postfix:

select create_totals_view('totals');

Finally, query the view:

select * from totals_view;

Generalized solution (for jsonb)

create or replace function create_jsonb_flat_view
    (table_name text, regular_columns text, json_column text)
    returns text language plpgsql as $$
declare
    cols text;
begin
    execute format ($ex$
        select string_agg(format('%2$s->>%%1$L "%%1$s"', key), ', ')
        from (
            select distinct key
            from %1$s, jsonb_each(%2$s)
            order by 1
            ) s;
        $ex$, table_name, json_column)
    into cols;
    execute format($ex$
        drop view if exists %1$s_view;
        create view %1$s_view as 
        select %2$s, %3$s from %1$s
        $ex$, table_name, regular_columns, cols);
    return cols;
end $$;

Usage:

create table example (id int, name text, params jsonb);
insert into example values
(1, 'Anna', '{"height": 175, "weight": 55}'),
(2, 'Bob', '{"age": 22, "height": 188}'),
(3, 'Cindy', '{"age": 25, "weight": 48, "pretty": true}');

select create_jsonb_flat_view('example', 'id, name', 'params');

select * from example_view;

 id | name  | age | height | pretty | weight 
----+-------+-----+--------+--------+--------
  1 | Anna  |     | 175    |        | 55
  2 | Bob   | 22  | 188    |        | 
  3 | Cindy | 25  |        | true   | 48
(3 rows)
klin
  • 112,967
  • 15
  • 204
  • 232
  • 9
    This is by far the single most useful and profound answer that I have ever seen on stack overflow. The general capability of flattening jsonb fields into views like this should be part of postgresql based on the technique outlined here as it is solves a variety of problems that occur with jsonb fields. – David Watson Mar 02 '16 at 18:18
  • This is excellent! Is there a straightforward way to make this work with multiple jsonb columns? E.g. `select create_jsonb_flat_view('example', 'id, name', 'params1, params2');`? – adilapapaya Aug 07 '17 at 19:01
  • 2
    @adilapapaya - a simple way: `select create_jsonb_flat_view('example', 'id, name', 'params1 || params2');` – klin Aug 07 '17 at 19:23
  • No wonder you just spotted that duplicate, I'd probably remember this too. – mu is too short Mar 14 '18 at 01:54
  • Is there a way to extend this such that it flattens arrays of objects? – Tyler DeWitt Jan 20 '19 at 04:49
  • @TylerDeWitt - I think so but the issue is ambiguous. Ask a new question with json structure description, sample data and expected output. – klin Jan 20 '19 at 20:03
  • @klin - haha, good point! I've tried to expand on it here: https://stackoverflow.com/questions/54285373/postgres-flatten-jsonb-cell-into-row-in-view. Thanks! – Tyler DeWitt Jan 21 '19 at 07:42
  • This answer creates columns for each JSONB key, but the OP asked for (and I am looking for) a single column with namespaced keys (i.e. '.'-separated key paths). – jrc Oct 05 '21 at 11:49
  • @jrc - I think you are misinterpreting the OP's intention. Nevertheless, ask a new question with a table structure, sample data and expected result. I suppose your expectations are simpler than the issue described here. – klin Oct 05 '21 at 12:12
  • @klin, you are right!! Looks like I mixed up the OP's rows vs. columns. And I _think_ I figured out what I need with `jsonb_each()`… Thanks for the fantastic answer on this one, BTW – jrc Oct 05 '21 at 12:57
  • JSON values are coerced to text, how could we automatically cast to the correct json value type (int, float, timestamp, true/false, string)? BTW, one of the best answers I've seen here. You rock @klin! – Cássio Oct 27 '21 at 02:30
  • @CássioJandirPagnoncelli - JSON has only two data types other than string, i.e. `number` and `boolean`. Theoretically, it is possible to expand the function to automatically generate `numeric` and `boolean` columns but the function code would be much more complex because of potential corner cases. – klin Oct 27 '21 at 17:05
  • @klin you're right! I'm working on a numeric/string adaptation and found `CAST(json_value AS ANYELEMENT)` would do the job —we can run math and logical operations in the query. I'm struggling to find where I can plug this in the query. – Cássio Oct 27 '21 at 20:05
  • @klin There's a cool project `ahoy_js` where this query would serve perfectly for the analytics community. They even built a side project `blazer` to query json because querying json from pg is complicated whilst this `VIEW` is probably all that's needed :) – Cássio Oct 27 '21 at 20:06
  • Actually `ANYELEMENT` won't do since multiple `ANYELEMENT` are not allowed: `SELECT CAST('1' AS ANYELEMENT) + CAST('2' AS ANYELEMENT) + 3;` yields `ERROR: operator is not unique: unknown + unknown`. – Cássio Oct 27 '21 at 20:12