0

I am using SQL functions to perform multi-inserts, but because they can't accept recordsets as arguments I have to convert them to an array first. It works fine for array of primitives because they can simply be cast with CAST (${value} as primitive_type[]) and be done with it.
However multi-insert queries require composite type arrays and it doesn't look like CAST() works with them, since it expects one-column input.
All queries are shown on this fiddle: https://dbfiddle.uk/w_Qbq-lw

Tables and Types

CREATE TABLE accounts (
  id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  created_at timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP,
  login text NOT NULL,
  password text NOT NULL,
  email text
);

CREATE TYPE account_init AS (
  login text,
  password text,
  email text
);

Functions

CREATE FUNCTION get_accounts(
  pagination_limit bigint DEFAULT 25,
  pagination_offset bigint DEFAULT 0,
  account_ids bigint[] DEFAULT NULL
)
RETURNS TABLE (
  id bigint,
  created_at timestamptz,
  login text,
  password text,
  email text
)
LANGUAGE SQL
AS $BODY$
  WITH input_accounts AS (
    SELECT
      id,
      created_at,
      login,
      password,
      email
    FROM
      accounts
    WHERE
      account_ids IS NULL OR id = ANY (account_ids)
    ORDER BY
      id
    LIMIT pagination_limit
    OFFSET pagination_offset
  )
  SELECT
    id,
    created_at,
    login,
    password,
    email
  FROM
    input_accounts
  ORDER BY
    id
$BODY$;

CREATE FUNCTION create_accounts(
  account_inits account_init[]
)
RETURNS TABLE (
  id bigint,
  created_at timestamptz,
  login text,
  password text,
  email text
)
LANGUAGE SQL
AS $BODY$
  WITH new_accounts AS (
    INSERT INTO accounts ( 
      login, 
      password, 
      email 
    )
    SELECT 
      login, 
      password, 
      email
    FROM 
      unnest(account_inits)
    RETURNING
      id
  )
  SELECT
    id,
    created_at,
    login,
    password,
    email
  FROM
    get_accounts(
      NULL,
      NULL,
      ARRAY(
        SELECT
          id
        FROM
          new_accounts
      )
    )
  ORDER BY
    id
$BODY$;

Init data

const account_inits = [
  {
    login:"EC4A42323F", 
    password: "3DF1542F23A29B73281EEC5EBB55FFE18C253A7E800E7A541B"
  },
  {
    login:"1D771C1E52", 
    password: "2817029563CC722FBC3D53F9F29F0000898F9843518D882E4A", 
    email: "a@b"
  },
  {
    login:"FB66381D3A", 
    password: "C8F865AC1D54CFFA56DEBDEEB671C8EF110991BBB3B9EE57D2", 
    email: null
  }
]

Usage

--- insert data
WITH input_inits AS (
  SELECT
    login,
    password,
    email
  FROM
    json_to_recordset(${account_inits:json}) AS input_init(
      login text,
      password text,
      email text
    )
),
input_data AS (
  SELECT
    array_agg(
      CAST (
        (
          login,
          password,
          email
        ) AS account_init
      )
    ) AS account_inits
  FROM
    input_inits
)
SELECT
  new_accounts.id,
  new_accounts.created_at,
  new_accounts.login,
  new_accounts.password,
  new_accounts.email
FROM
  input_data
  CROSS JOIN
  create_accounts(input_data.account_inits) AS new_accounts
ORDER BY
  new_accounts.id ASC
;

Currently I interpolate it as :json and then convert it to recordset in the CTE, which then gets converted to a composite type array in the second CTE to pass as an argument to the function. This seems to be awfully a lot of work for passing the array of objects to the function arguments. I've tried to work without :json conversion but either encountered array[] related or malformed object literal syntax errors.

Biller Builder
  • 303
  • 3
  • 10
  • Your question hasn't attracted any response, perhaps because it is too broad, i.e. lacks focus on the actual problem you have. – vitaly-t Oct 22 '22 at 07:51
  • Actually it's pretty specific, that's why there is no response. The problem is transforming an array of objects, which is the result of `pg-promise` interpolation (or whatever `pg-promise` transforms an array of objects without casting it to json), into a postgresql composite type array. The `interpolate as json -> turn into a set of records -> aggregate into array` routine works, but It's pretty verbose and won't scale well when the objects' keys themselves can be arrays of objects. – Biller Builder Oct 23 '22 at 05:11
  • Part where `pg-promise` is concerned - in generating [multi-row inserts](https://stackoverflow.com/questions/37300997/multi-row-insert-with-pg-promise), but I see that you are not even using those, so it is likely on the outside of where I can help (as the author of `pg-promise`). – vitaly-t Oct 24 '22 at 14:30
  • Yeah I am not using `pg-promise` multi-inserts because they require way too much fiddling and back-and-forth between the application code and the SQL. Not to mention `INSERT INTO ... ( ... ) SELECT ... FROM ... RETURNING ...` syntax is way more clear on what is passed for insertion versus string concatenation/raw interpolation. And can't be used within stored procedures. Nonetheless, what is a postgresql type of the interpolated array of objects when it isn't interpolated as `:json`? – Biller Builder Oct 25 '22 at 03:43
  • Too much fiddling? They are the most efficient way of inserting in-memory data. As for the last thing, tuples is the only other type for interpolated data, other than json. – vitaly-t Oct 25 '22 at 10:47
  • By "fiddling" I meant writing and supporting their related code is too much pain. Create a `Table` class. Create a query, describe all details for its columns, pass that query as an argument to another query, interpolate it as `:raw` there. And whatever speed/memory advantages taper off once there is a need to perform multi-inserts into several tables within the same transaction. By "tuples" do you mean rows? As in is it possible to run `SELECT ... FROM (${ obj_array }) AS table_name`? – Biller Builder Oct 25 '22 at 12:47
  • You cannot have dynamic table or column names, among many other restrictions within native Postgres, and client-side formatting via `pg-promise` removes all those restrictions, and adds many high-level functions. So, it is a compromise of native SQL versus universality. – vitaly-t Oct 25 '22 at 12:54
  • So how is the interpolated tuple supposed to be consumed then? Assuming its underlying composite type is known and declared beforehand. – Biller Builder Oct 25 '22 at 13:32
  • Composite types are presented as tuples in Postgres, they can be converted into each other directly, via type casting. – vitaly-t Oct 25 '22 at 14:01
  • I've read about [`FROM`](https://www.postgresql.org/docs/current/sql-select.html#SQL-FROM) clause more and turns out it can use [`VALUES`](https://www.postgresql.org/docs/13/sql-values.html) as a sub-select expression. And `VALUES` clause can coerse into types too. It does however require same-length tuples as an input. Does `pg-promise` ensure `NULL`s for missing keys in an interpolated array of objects? Now that I think about it, the interpolated tuple has to have the same order as the composite type, something which `pg-promise` wouldn't know out of the box for a given query. – Biller Builder Oct 25 '22 at 14:15

1 Answers1

0

Turns there isn't. The "proper" alternative would need to know the expected keys of the objects in the input array (and for all nested objects/arrays of objects) in order to construct the proper array of tuples.
This basically requires some sort of runtime referential schema which, even if implemented perfectly, will still require to write boilerplate code for each new query file. At best it will not be better than the functions in helpers module or the json -> record set -> composite type array casting within queries.

Biller Builder
  • 303
  • 3
  • 10