Copy from csv into table with id serial column auto-incrementing PSQL

Question

Looking for a parse solution that copies from CSV into my database with the end result of serial ID's generated. Either on the backend, or upon copying. As is, my ID's are null after the copy.

echo CREATE TABLE IF NOT EXISTS qs_facts_template (id SERIAL NOT NULL, symbol varchar(8), timestamp date, close real null, open real null, high real null, low real null, volume real null, CONSTRAINT qs_facts_template_key PRIMARY KEY (id)) WITH (OIDS=TRUE) TABLESPACE pg_default; ALTER TABLE qs_facts_template OWNER to postgres; | psql -U postgres -h %host% readyloop

echo CREATE TABLE IF NOT EXISTS qs_facts AS select * from qs_facts_template;| psql -U postgres -h %host% readyloop

echo \copy qs_facts (symbol, timestamp, close, open, high, low, volume) from PROGRAM 'cat C:\Users\user\Documents\quantshare\quotes.csv' DELIMITER ';' CSV HEADER| psql -U postgres -h %host% %dbName%

this post: https://unix.stackexchange.com/questions/277080/copy-csv-data-while-simultaneously-filling-serial-column did not solve my issue. I used the command mentioned by Guido and got the copy to work, but the id's are all null.

I was thinking maybe I could set the values AFTER the copy.

score 1 · Answer 1 · answered Sep 03 '19 at 02:36

You could write a BEFORE trigger:

CREATE FUNCTION id_trigger() RETURNS trigger
   LANGUAGE plpgsql AS
$$BEGIN
   NEW.id = nextval(pg_get_serial_sequence(TG_ARGV[0], 'id'));
   RETURN NEW;
END;$$;

CREATE TRIGGER id_trigger BEFORE INSERT ON qs_facts_template
   FOR EACH ROW EXECUTE PROCEDURE id_trigger('qs_facts_template');

score 0 · Answer 2 · answered Sep 02 '19 at 08:46

Problem is copy doesn't copy everything other than columns, add to the fact that you can't alter a table to serial.

So I exported the script in pg4admin from the original template table and was able to derive this which worked

    echo DROP materialized view if exists mv_qs_facts cascade| psql -U postgres -h %host% %dbName%
    echo DROP materialized view if exists mv_qs_symbols cascade| psql -U postgres -h %host% %dbName%

    echo DROP TABLE if exists qs_facts cascade| psql -U postgres -h %host% %dbName%

    echo drop table if exists qs_facts_template CASCADE;| psql -U postgres -h %host% readyloop
Rem create qs_fact table    
    echo CREATE TABLE IF NOT EXISTS qs_facts_template (id SERIAL, symbol varchar(8), timestamp date, close real null, open real null, high real null, low real null, volume real null, CONSTRAINT qs_facts_template_key PRIMARY KEY (id)) WITH (OIDS=TRUE) TABLESPACE pg_default; ALTER TABLE qs_facts_template OWNER to postgres; | psql -U postgres -h %host% readyloop

    echo CREATE TABLE IF NOT EXISTS qs_facts AS select * from qs_facts_template; | psql -U postgres -h %host% readyloop

    echo ALTER TABLE public.qs_facts ADD CONSTRAINT qs_facts_key PRIMARY KEY (id); | psql -U postgres -h %host% readyloop

    REM I looked at psqladmin and exported the sequence from qs_facts_template
    REM http://www.postgresqltutorial.com/postgresql-serial/
    echo CREATE SEQUENCE public.qs_facts_id_seq INCREMENT 1 START 1 MINVALUE 1 MAXVALUE 9223372036854775807 CACHE 1; ALTER SEQUENCE public.qs_facts_id_seq OWNER TO postgres;| psql -U postgres -h %host% readyloop

    echo ALTER TABLE public.qs_facts ALTER COLUMN id SET DEFAULT nextval('qs_facts_id_seq'); | psql -U postgres -h %host% readyloop

Copy from csv into table with id serial column auto-incrementing PSQL

2 Answers2

Linked