How to convert a postgres database to sqlite

Question

We're working on a website, and when we develop locally (one of us from Windows), we use sqlite3, but on the server (linux) we use postgres. We'd like to be able to import the production database into our development process, so I'm wondering if there is a way to convert from a postgres database dump to something sqlite3 can understand (just feeding it the postgres's dumped SQL gave many, many errors). Or would it be easier just to install postgres on windows? Thanks.

Answer 1

I found this blog entry which guides you to do these steps:

1. Create a dump of the PostgreSQL database.

   ssh -C username@hostname.com pg_dump --data-only --inserts YOUR_DB_NAME > dump.sql
   

2. Remove/modify the dump.

   1. Remove the lines starting with SET
   2. Remove the lines starting with SELECT pg_catalog.setval
   3. Replace true for ‘t’
   4. Replace false for ‘f’

3. Add BEGIN; as first line and END; as last line

4. Recreate an empty development database. bundle exec rake db:migrate

5. Import the dump.

   sqlite3 db/development.sqlite3
   sqlite> delete from schema_migrations;
   sqlite> .read dump.sql
   

Of course connecting via ssh and creating a new db using rake are optional

Answer 2

STEP1: make a dump of your database structure and data

pg_dump --create --inserts -f myPgDump.sql \
   -d myDatabaseName -U myUserName -W myPassword

STEP2: delete everything except CREATE TABLES and INSERT statements out of myPgDump.sql (using text editor)

STEP3: initialize your SQLite database passing structure and data of your Postgres dump

sqlite3 myNewSQLiteDB.db -init myPgDump.sql

STEP4: use your database ;)

Answer 3

Taken from https://stackoverflow.com/a/31521432/1680728 (upvote there): The sequel gem makes this a very relaxing procedure:

First install Ruby, then install the gem by running gem install sequel.

In case of sqlite, it would be like this: sequel -C postgres://user@localhost/db sqlite://db/production.sqlite3

Credits to @lulalala .

Answer 4

You can use pg2sqlite for converting pg_dump output to sqlite.

#  Making dump
pg_dump -h host -U user -f database.dump database

#  Making sqlite database
pg2sqlite -d database.dump -o sqlite.db

Schemas is not supported by pg2sqlite, and if you dump contains schema then you need to remove it. You can use this script:

# sed 's/<schema name>\.//' -i  database.dump
sed 's/public\.//' -i  database.dump
pg2sqlite -d database.dump -o sqlite.db

Answer 5

Even though there are many very good helpful answers here, I just want to mark this as answered. We ended up going with the advice of the comments:

I'd just switch your development environment to PostgreSQL, developing on top of one database (especially one as loose and forgiving as SQLite) but deploying on another (especially one as strict as PostgreSQL) is generally a recipe for aggravation and swearing. – @mu is too short

To echo mu's response, DON'T DO THIS..DON'T DO THIS..DON'T DO THIS. Develop and deploy on the same thing. It's bad engineering practice to do otherwise. – @Kuberchaun

So we just installed postgres on our dev machines. It was easy to get going and worked very smoothly.

Answer 6

In case one needs a more automatized solution, here's a head start:

#!/bin/bash

$table_name=TABLENAMEHERE

PGPASSWORD="PASSWORD" /usr/bin/pg_dump --file "results_dump.sql" --host "yourhost.com" --username "username" --no-password --verbose --format=p --create --clean --disable-dollar-quoting --inserts --column-inserts --table "public.${table_name}" "memseq"

# Some clean ups
perl -0777 -i.original -pe "s/.+?(INSERT)/\1/is" results_dump.sql
perl -0777 -i.original -pe "s/--.+//is" results_dump.sql

# Remove public. prefix from table name
sed -i "s/public.${table_name}/${table_name}/g" results_dump.sql

# fix binary blobs
sed -i "s/'\\\\x/x'/g" results_dump.sql

# use transactions to make it faster
echo 'BEGIN;' | cat - results_dump.sql > temp && mv temp results_dump.sql
echo 'END;' >> results_dump.sql

# clean the current table 
sqlite3 results.sqlite "DELETE FROM ${table_name};"

# finally apply changes
sqlite3 results.sqlite3 < results_dump.sql && \
rm results_dump.sql && \
rm results_dump.sql.original

Answer 7

when I faced with same issue I did not find any useful advices on Internet. My source PostgreSQL db had very complicated schema.
You just need to remove from your db-file manually everything besides table creating
More details - here

Answer 8

It was VERY easy for me to do using the taps gem as described here: http://railscasts.com/episodes/342-migrating-to-postgresql

And I've started using the Postgres.app on my Mac (no install needed, drop the app in your Applications directory, although might have to add one line to your PATH envirnment variable as described in the documentation), with Induction.app as a GUI tool to view/query the database.