0

I am trying to migrate a MySQL db to Postgres. From what I've read online, it seems like pgloader is the best tool for this.

Background:

I installed pgloader with apt-get, but ran into an error when running the migration:

$ pgloader mysql://root:<password>@localhost/state_migrations postgresql:///state_migrations

ERROR mysql: Failed to connect to mysql at "localhost" (port 3306) as user "root": Condition QMYND:MYSQL-UNSUPPORTED-AUTHENTICATION was signalled.

I solved that issue by changing the security permissions for MySQL as suggested by these questions:

That appears to have worked, but then I ran into this error when running the migration command:

pgloader ERROR mysql: 76 fell through ECASE expression.

I found this issue on Github and it seems a couple of people had success after installing pgloader from source instead of apt-get.

Current issue:

This guide helped me figure out how to install from source, but the make pgloader command is bringing me to what I think is a lisp prompt and I'm not understanding what I need to do...

make pgloader
mkdir -p build
curl -o build/quicklisp.lisp http://beta.quicklisp.org/quicklisp.lisp
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 57144  100 57144    0     0   389k      0 --:--:-- --:--:-- --:--:--  387k
sbcl --noinform --no-sysinit --no-userinit --load build/quicklisp.lisp                        \
             --load src/getenv.lisp                                               \
             --eval '(quicklisp-quickstart:install :path "build/quicklisp" :proxy (getenv "http_proxy"))' \
             --eval '(quit)'

  ==== quicklisp quickstart 2015-01-28 loaded ====

    To continue with installation, evaluate: (quicklisp-quickstart:install)

    For installation options, evaluate: (quicklisp-quickstart:help)

WARNING: Making quicklisp part of the install pathname directory

debugger invoked on a SB-BSD-SOCKETS:OPERATION-TIMEOUT-ERROR in thread
#<THREAD "main thread" RUNNING {1004BD8173}>:
  Socket error in "connect": ETIMEDOUT (Connection timed out)

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Ignore runtime option --eval "(quicklisp-quickstart:install :path \"build/quicklisp\" :proxy (getenv \"http_proxy\"))".
  1: [ABORT   ] Skip rest of --eval and --load options.
  2:            Skip to toplevel READ/EVAL/PRINT loop.
  3: [EXIT    ] Exit SBCL (calling #'EXIT, killing the process).

(SB-BSD-SOCKETS:SOCKET-ERROR "connect" 110)
   error finding frame source: Bogus form-number: the source file has probably
                               changed too much to cope with.
   source: NIL
0]

The output stops for a bit after saying it is Making quicklisp part of the install pathname directory, but then it times out eventually. I have tried entering (quicklisp-quickstart:install), but it keeps timing out.

0] (quicklisp-quickstart:install)


debugger invoked on a SB-BSD-SOCKETS:OPERATION-TIMEOUT-ERROR in thread
#<THREAD "main thread" RUNNING {1004BD8173}>:
  Socket error in "connect": ETIMEDOUT (Connection timed out)

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT   ] Reduce debugger level (to debug level 1).
  1: [CONTINUE] Ignore runtime option --eval "(quicklisp-quickstart:install :path \"build/quicklisp\" :proxy (getenv \"http_proxy\"))".
  2:            Skip rest of --eval and --load options.
  3:            Skip to toplevel READ/EVAL/PRINT loop.
  4: [EXIT    ] Exit SBCL (calling #'EXIT, killing the process).

(SB-BSD-SOCKETS:SOCKET-ERROR "connect" 110)
   error finding frame source: Bogus form-number: the source file has probably
                               changed too much to cope with.
   source: NIL
0[2] 
  • Ubuntu 22.04.2 LTS
  • psql (PostgreSQL) 14.7 (Ubuntu 14.7-0ubuntu0.22.04.1) (installed with sudo apt install postgresql postgresql-contrib
  • apt-get installed: pgloader "3.6.3~devel" compiled with SBCL 2.1.11.debian
  • When building pgloader from source, I was installing 3.6.9 (latest release)

The instructions for building from source aren't very descriptive from pgloader, but they're:

Install the build dependencies first, then use the Makefile:

$ apt-get install sbcl unzip libsqlite3-dev make curl gawk freetds-dev libzip-dev
$ cd /path/to/pgloader

$ make save
$ ./build/bin/pgloader --help

Because I've never installed from source before or used make, I followed this guide since it installs from source: https://www.digitalocean.com/community/tutorials/how-to-migrate-mysql-database-to-postgres-using-pgloader#step-2-installing-pgloader

coniferous
  • 67
  • 8
  • 1
    1) What version of Debian/Ubuntu? 2) What repo are you using the Debian/Ubuntu ones or the [PGDG](https://www.postgresql.org/download/linux/ubuntu/ ones? 3) What version of Postgres? **ADD ANSWERS as update to question**. – Adrian Klaver Apr 03 '23 at 21:55
  • @AdrianKlaver, I don't understand the second question, but I've updated the question with my ubuntu and psql version – coniferous Apr 04 '23 at 03:51
  • The question is which repository you were using to install PostgreSQL and (initially) pgloader. Also, which version of pgloader got installed with `apt`? I'd open an issue with pgloader if I had trouble installing it from source (after carefully reading and following the instructions). – Laurenz Albe Apr 04 '23 at 06:14
  • I was looking to find out what package repository you are using to retrieve `pgloader`. I will assume you are using the Ubuntu repo. On my Ubuntu 22.04 install I see `pgloader 3.6.3-1ubuntu1` as available. That is later then the version referred to in the issue you linked to. Confirm what version of `pgloader` you are using and **add as update to question**. – Adrian Klaver Apr 04 '23 at 15:22
  • Thanks. I updated my post with the info @AdrianKlaver – coniferous Apr 04 '23 at 15:42
  • And @laurenz-albe – coniferous Apr 04 '23 at 15:42
  • 1
    1) For building from sources take a look at [Issue 3109](https://github.com/pretix/pretix/issues/3109): *On ubuntu (22.04.1) you will probably also have to build pgloader yourself. Download the sources from github, then apt install sbcl and make and use the resulting binary instead of the packaged one*. 2) Also from that issue: *We already added a recommendation to install pgloader from the PostgreSQL apt sources after #3090, which should work on 22.04 as well.*. 3) Said Postgresql apt repo(sources) are here [PGDG Ubuntu repo](https://www.postgresql.org/download/linux/ubuntu/). – Adrian Klaver Apr 04 '23 at 15:57
  • Thanks @AdrianKlaver. What does it mean by "use the resulting binary" though? I'm also unsure how I'd install from the psql "apt sources" – coniferous Apr 04 '23 at 16:00
  • 1
    1) For built binaries see here [Install from source](https://pgloader.readthedocs.io/en/latest/install.html): Section *Building from sources on debian*. 2) For installing from the Postgres repos you would follow the instructions on the page I linked to. **NOTE** doing so will mean you have two sources for Postgres packages. – Adrian Klaver Apr 04 '23 at 16:10
  • @AdrianKlaver 1) I already tried building the binaries like that then (see addition to my initial question at the bottom). 2) I will try uninstalling my current Postgres instance and installing via the apt sources you linked – coniferous Apr 04 '23 at 16:14
  • 1
    I just built `pgloader(3.6.9)` from source on my Ubuntu 22.04 instance using the exact instructions from the pgloader site. e.g `make save`. I noticed in your build error you had `make pgloader`. Which form of `make` did you actually use? – Adrian Klaver Apr 04 '23 at 16:27
  • @AdrianKlaver, I tried `make pgloader` because that's what the digital ocean guide used, but also been running `make save`. Getting the same result, though, with it timing out. – coniferous Apr 04 '23 at 16:47
  • @AdrianKlaver After running `tar xvf v3.6.9.tar.gz `, did you have anything in your pgloader-3.6.9/build/bin directory? Mine is empty – coniferous Apr 04 '23 at 16:53
  • 1
    1) `...build/bin/` will only have something in it after the `make` succeeds in building the `pgloader` program. The `tar` operation just unpacks the compressed file. 2) It looks to me like the `make` is having an issue downloading the `quicklisp` code necessary to build the `pgloader` program. You might look at your system logs to see if they have more information or see if the pgloader from the PGDG repo works. – Adrian Klaver Apr 04 '23 at 17:54
  • The pgloader from the PGDG repo didn't work--still gives the `ERROR mysql: 76 fell through ECASE expression.` error. Where can I find the logs you're referring to? I haven't really looked through logs other than those in the terminal before... @AdrianKlaver – coniferous Apr 04 '23 at 19:03
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/252975/discussion-between-coniferous-and-adrian-klaver). – coniferous Apr 04 '23 at 19:09

0 Answers0