pg_dump Fails When UTF-8 Characters In Output File Name

Question

I am trying to create a backup of a database using pg_dump. The database name has non ASCII characters, for example 本物ではないデータベース名.

I am making the following call:

pg_dump.exe --file="C:\Users\AUser\AppData\Local\Temp\本物ではないデータベース名_20150309185239.pgbak" pg_backupc4c978f4aebc4153adcabab6e097c347

However it is failing with the following error:

pg_dump: [archiver] could not open output file "C:\Users\AUser\AppData\Local\Temp\????????????_20150309185239.pgbak": Invalid argument

What I don't understand is that other commands work with non ASCII characters. For example mdkdir 本物ではないデータベース名 works fine.

Is this a bug in pg_dump or am I doing something wrong?

UPDATE

As per @Mike Sherrill 'Cat Recall' in the comments, I tested redirection and it worked in both Powershell and the standard command line.

However, I then tried to automate this with C# and when I try to run the commands above using a C# Process it fails:

        ProcessStartInfo psi = new ProcessStartInfo(command, args);
        psi.CreateNoWindow = true;
        psi.UseShellExecute = false;
        psi.RedirectStandardOutput = psi.RedirectStandardError = true;
        using (Process process = new Process())
        {
            process.StartInfo = psi;
            string s = "";
            process.ErrorDataReceived += delegate (object sender, DataReceivedEventArgs e)
            {
                s += e.Data;
            };
            string s1 = "";
            process.OutputDataReceived += delegate (object sender, DataReceivedEventArgs e)
            {
                s1 += e.Data;
            };
            process.Start();
            process.BeginErrorReadLine();
            process.BeginOutputReadLine();
            TimeSpan maxTime = TimeSpan.FromMinutes(10);
            if (!process.WaitForExit((int)maxTime.TotalMilliseconds))
            {
                throw new Exception("Command " + command + " " + args + " Timed out after " + maxTime.TotalSeconds);
            }
            if (process.ExitCode != 0)
            {
                throw new Exception("Command " + command + " " + args + " failed with " + s + " " + s1);
            }
        }

Again, I can make a C# Process and call mkdir with Japanese letters and everything works.

Works for me in PostgreSQL 9.4 under 64-bit Ubuntu 14.04 LTS. Usually, this is a symptom of a code page issue or font issue. See http://stackoverflow.com/q/388490/562459 — Mike Sherrill 'Cat Recall', Mar 09 '15 at 20:35
@MikeSherrill'CatRecall' I am running PostgreSQL 9.3 on Windows 8.1. Ill update to PostgreSQL 9.4 and see what happens. — shortspider, Mar 09 '15 at 21:01
What terminal are you using to type that command? if it's cmd then the position of the quotation marks seems incorrect. — Anya Shenanigans, Mar 09 '15 at 21:14
@shortspider: FWIW, I think it's unlikely to be a PostgreSQL or pg_dump issue. — Mike Sherrill 'Cat Recall', Mar 09 '15 at 21:37
windows does **not** use UTF-8 for wide-character encoding, it uses UTF-16. I don't know if the pg_dump utility supports the use of UTF-16 characters for the filename — Anya Shenanigans, Mar 09 '15 at 22:41
@Petesh: The OP could probably test by using redirection instead of the `--file` argument: `pg_dump database_name > C:\Users\AUser\AppData\Local\Temp\本物ではないデータベース名_20150309185239.pgbak`. — Mike Sherrill 'Cat Recall', Mar 10 '15 at 00:56
@MikeSherrill'CatRecall' redirection does work in Powershell and cmd. However when I try to run `pg_dump` with the file flag like I originally tried from a C# `Process` it also fails. — shortspider, Mar 10 '15 at 15:38
The file option `--file` is parsed and processed by pg_dump code, which might not be written to cope with UTF-16. ([Source code](http://doxygen.postgresql.org/pg__dump_8c_source.html)) Redirection is done by the shell. If you want this to work, you'll need to either rewrite part of pg_dump(), or use command-line redirection. — Mike Sherrill 'Cat Recall', Mar 11 '15 at 07:25
@MikeSherrill'CatRecall' I agree with this 100%. I manged to get the C# working by doing the redirect. If you want to post your comment in an answer I'll mark is as correct? — shortspider, Mar 12 '15 at 13:25

score 1 · Accepted Answer · answered Mar 17 '15 at 18:09

Per @MikeSherrill'CatRecall':

"The file option --file is parsed and processed by pg_dump code, which might not be written to cope with UTF-16. (Source code) Redirection is done by the shell. If you want this to work, you'll need to either rewrite part of pg_dump(), or use command-line redirection."

pg_dump Fails When UTF-8 Characters In Output File Name

1 Answers1