4

I'm new. Bear with me.

I'm developing a Flask application using SQLAlchemy as an ORM and up until today I have been using SQLite for convenience. I'm now putting the application live on Digital Ocean and want to use MySQL instead of SQLite.

I have MySQL installed on my ubuntu VPS on Digital Ocean and it seems like it is configured properly. However, obviously I have to create the database tables, so I can save the data inside.

Question: Is there a way for me to migrate my models.py, so the database tables are created from what I have written in models.py or do I have to create all the database tables myself manually in MySQL?

You can see the application live here: http://workflowforum.dk/ and I have made a small test to see if there is a database connection here: http://workflowforum.dk/testdb

Models.py (Only user model):

from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy 
from flask.ext.script import Manager
from flask.ext.migrate import Migrate, MigrateCommand
from datetime import datetime, date
from hashlib import md5
from bcrypt import hashpw, gensalt

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:password@localhost/database'
db = SQLAlchemy(app)

migrate = Migrate(app, db)

manager = Manager(app)
manager.add_command('db', MigrateCommand)

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    slug = db.Column(db.String(80))
    username = db.Column(db.String(80), unique=True)
    email = db.Column(db.String(80))
    password = db.Column(db.String(80), unique=False)
    admin = db.Column(db.Boolean(), default=False)
    join_date = db.Column(db.DateTime)
    last_seen = db.Column(db.DateTime)
    topics = db.relationship('Topic')
    posts = db.relationship('Post')
    picture = db.Column(db.Boolean(), default=False)
    title = db.Column(db.String(80))
    company = db.Column(db.String(80))
    summary = db.Column(db.String(80))

class Category(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    description = db.Column(db.String(180), unique=False)
    topics = db.relationship('Topic', backref="category")

class Topic(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    slug = db.Column(db.String(255), unique=True)
    title = db.Column(db.String(80), unique=False)
    description = db.Column(db.Text, unique=False)
    pub_date = db.Column(db.DateTime)
    last_update = db.Column(db.DateTime)
    user_id = db.Column(db.String(80), db.ForeignKey('user.id'))
    category_id = db.Column(db.Integer, db.ForeignKey('category.id'))
    views = db.Column(db.Integer, default=0)
    locked = db.Column(db.Boolean(), default=False)
    pinned = db.Column(db.Boolean(), default=False)
    user = db.relationship('User')
    posts = db.relationship('Post')

Views.py (Only database test):

@app.route('/testdb')
    def testdb():
    if db.session.query("1").from_statement("SELECT 1").all():
        return 'It works.'
    else:
        return 'Something is broken.'

UPDATE after Lukas comment:

When trying to db.create_all() I get this traceback:

sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1005, "Can't create table 'pwforum.topic' (errno: 150)") [SQL: u'\nCREATE TABLE topic (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tslug VARCHAR(255), \n\ttitle VARCHAR(80), \n\tdescription TEXT, \n\tpub_date DATETIME, \n\tlast_update DATETIME, \n\tuser_id VARCHAR(80), \n\tcategory_id INTEGER, \n\tviews INTEGER, \n\tlocked BOOL, \n\tpinned BOOL, \n\tPRIMARY KEY (id), \n\tUNIQUE (slug), \n\tFOREIGN KEY(user_id) REFERENCES user (id), \n\tFOREIGN KEY(category_id) REFERENCES category (id), \n\tCHECK (locked IN (0, 1)), \n\tCHECK (pinned IN (0, 1))\n)\n\n']
Andy
  • 531
  • 1
  • 4
  • 19
  • You should be able to simply do `db.create_all()` (where `db = SQLAlchemy(app)` to have SQLAlchemy create all the necessary tables based on the metadata of your models. See [`Metadata.create_all()`](http://docs.sqlalchemy.org/en/rel_1_0/core/metadata.html#sqlalchemy.schema.MetaData.create_all) and the [Describing Databases with MetaData](http://docs.sqlalchemy.org/en/rel_1_0/core/metadata.html) docs in general. – Lukas Graf Aug 07 '15 at 19:35
  • Hey Lukas. Thx! It raises an error: sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1005, "Can't create table 'pwforum.topic' (errno: 150)") [SQL: u'\nCREATE TABLE topic (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tslug VARCHAR(255), \n\ttitle VARCHAR(80), \n\tdescription TEXT, \n\tpub_date DATETIME, \n\tlast_update DATETIME, \n\tuser_id VARCHAR(80), \n\tcategory_id INTEGER, \n\tviews INTEGER, \n\tlocked BOOL, \n\tpinned BOOL, \n\tPRIMARY KEY (id), \n\tUNIQUE (slug), \n\tFOREIGN KEY(user_id) REFERENCES user (id), \n\tFOREIGN KEY(category_id) REFERENCES category (id), – Andy Aug 07 '15 at 19:40
  • That's an error with one of your foreign key constraints. See [this question](http://stackoverflow.com/questions/2799021/mysql-error-1005-hy000-cant-create-table-foo-sql-12c-4-errno-150) on how to debug this, particularly using `SHOW ENGINE INNODB STATUS` to get the `LATEST FOREIGN KEY ERROR`. – Lukas Graf Aug 07 '15 at 19:49
  • You probably also need to include the models for `Topic` and `Category` in your question for someone to be able to help you debug this. – Lukas Graf Aug 07 '15 at 19:57
  • Helpful links once again Lukas. Thanks! It seems like there are multiple things that could be wrong from the other thread. I ran `SHOW ENGINE INNODB STATUS` and got: 150807 15:46:55 Error in foreign key constraint of table pwforum/topic: FOREIGN KEY(user_id) REFERENCES user (id), FOREIGN KEY(category_id) REFERENCES category (id), CHECK (locked IN (0, 1)), CHECK (pinned IN (0, 1)) ): Any idea what could be wrong with the foreign keys? I'm a beginner in databases. Apologize and thanks for taking the time! Have updated original question with topic model. – Andy Aug 07 '15 at 19:59
  • 1
    Thanks Lukas. `Topic` and `Category` models have been added. – Andy Aug 07 '15 at 20:01
  • 1
    From a quick glance, `user.id` has a different data type (Integer) than `topic.user_id` (String(80)). A foreign key column needs to have the same data type as the referenced key. – mata Aug 07 '15 at 20:06
  • I think @mata is on to something ;-) With that change, the `create_all()` succeeds for me for that part of the OP's model (references to the `posts` table commented out). – Lukas Graf Aug 07 '15 at 20:12
  • Mata and Lukas if you were here in Copenhagen I would give you both a hug. It works like a charm. I do have a follow up question. Now I have a MySQL running on Digital Ocean with "real data". Should I keep my SQLite for testing purposes on my local machine? Change it to MySQL as well? In case I should have MySQL on Digital Ocean and SQLite on local machine - how do I have two different settings for it in my views.py app.config? (`app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:password@localhost/database'`) – Andy Aug 07 '15 at 20:21
  • 1
    I'd say that if you use mysql in production, you should also use it in testing, testing should be as close to production as possible. As you've seen, if something works with sqlite, it doesn't mean that it will also work on mysql - specially things concerning data types as sqlite doesn't really care for them. – mata Aug 07 '15 at 20:32
  • I agree. Recent post on HN on this very subject: [Don't test (only) with SQLite when you use Postgres in Production](https://news.ycombinator.com/item?id=10002142). Also consider this: In order to debug a tricky issue locally, you may well want to copy (parts of) the production data to your local machine to investigate the issue in more detail. Having your testing environment mirror your production environment closely is crucial for that. – Lukas Graf Aug 07 '15 at 20:41

1 Answers1

1

The db.create_all suggestion in the comments is usually what you do when you don't use a migration framework. But it looks like you are using Flask-Migrate as a database migration framework here. The problem with doing create_all is that your migration scripts are skipped, so any fixes that you need to make to convert your sqlite models to MySQL will not be reflected in the migration scripts.

My recommendation is that you create an empty MySQL database, configure it in your Flask app, and then generate the MySQL tables simply by calling:

$ ./manage.py db upgrade

When you do this, Flask-Migrate will start running the migration scripts one by one and in order.

If you get failures, that it is probably because some of your migration scripts have changes that are compatible with sqlite but not with MySQL. You will need to fix all these problems in your migration scripts, until you get all of them to run cleanly, at which point you will have a complete MySQL database to use in your application.

Miguel Grinberg
  • 65,299
  • 14
  • 133
  • 152