Postgres - ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d

Question

I'm trying to import a massive txt file into the postgres. When I typed the following command:

\COPY denton_2018_rawdata FROM 'C:\Users\testu\Downloads\denton_county\2018-website-all-property\2018-08-28_005183_APPRAISAL_INFO.txt' delimiter E'\x01'

I got the following error:

ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d CONTEXT: COPY denton_2018_rawdata, line 22769: "000000027205R 02018000000000000 ..."

So I tried the following command (adding ENCODING 'WINDOWS-1252') :

\COPY denton_2018_rawdata FROM 'C:\Users\testu\Downloads\denton_county\2018-website-all-property\2018-08-28_005183_APPRAISAL_INFO.txt' delimiter E'\x01' ENCODING 'WINDOWS-1252';

But still got the same error. Could anyone help please?

score 1 · Accepted Answer · answered Nov 05 '18 at 21:15

PostgreSQL is very strict about UTF8 encoding. It is due possible SQL injection attacks based on invalid UTF8 characters. First you have to know, what is source encoding. Second you should to eliminate all broken chars before import to Postgres.

There are some application that can do this work - like iconv

Postgres - ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d

1 Answers1