plpgsql Select statement in For loop to create multiple CSV files

Question

I would like to repeat the following query 8760 times, replacing ‘2’ with 1 to 8760 for every hour in the year. The idea is to create a separate CSV file for each hour for further processing.

COPY 
(SELECT *
FROM 
  public.completedsolarirad2012
WHERE 
  completedsolarirad2012."UniquetmstmpID" = 2)
TO 'C:\temp\2012hour2.csv' WITH DELIMITER ','
CSV HEADER

I have put together the following function (testing with only a few hours):

CREATE OR REPLACE FUNCTION everyhour()
  RETURNS void AS
$BODY$BEGIN
FOR i IN 0..5 LOOP
  EXECUTE $x$ 
  COPY (
    SELECT *
    FROM 
    public.completedsolarirad2012
    WHERE 
    completedsolarirad2012."UniquetmstmpID" = i
    )
   TO $concat$ 'C:\temp.' || i::text 
         || '.out.csv' WITH DELIMITER ',' CSV HEADER $concat$
    $x$;
END LOOP;
END;
$BODY$
  LANGUAGE plpgsql VOLATILE
  COST 100;
ALTER FUNCTION everyhour()
  OWNER TO postgres;

I seem to be having two separate problems:

Firstly, I’m getting:

Error: column "i" does not exist.

Secondly, when I test the concatenation statement only by replacing “i” with e.g. “2”, I get:

Error: relative path not allowed for COPY to file

I am the postgres superuser, so I do not understand why I am having this problem.

Note: Removing the $concat$ double-quoting around the concatenation statement gives the following error:

ERROR:  syntax error at or near "||"
LINE 9:    TO 'C:\temp.' || i::text

I would be very grateful for any help.

Answer 1

Assuming your server OS is Windows.

CREATE OR REPLACE FUNCTION everyhour()
  RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
   EXECUTE '
   COPY (
      SELECT *
      FROM   public.completedsolarirad2012 c
      WHERE  c."UniquetmstmpID" = ' || i || $x$)
   TO 'C:/temp.'$x$ || i || $x$'.out.csv' WITH DELIMITER ',' CSV HEADER$x$;
END LOOP;
END
$func$  LANGUAGE plpgsql;

• You had one layer of dollar-quoting too many.
• You also accidentally concatenated the letter "i" instead of its value.
• Use forward-slashes, even with windows. Or you may have to double up the backslashes, depending on your settings. More in this related answer:
  PostgreSQL: export resulting data from SQL query to Excel/CSV

Simpler with format()

Since Postgres 9.1 you can use format() to simplify complex concatenations:

CREATE OR REPLACE FUNCTION everyhour()
  RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
   EXECUTE format($x$COPY (
   SELECT *
   FROM   public.completedsolarirad2012 c
   WHERE  c."UniquetmstmpID" = %1$s)
   TO 'C:/temp.%1$s.out.csv' WITH DELIMITER ',' CSV HEADER$x$, i);
END LOOP;
END
$func$  LANGUAGE plpgsql;