19

I changed my Django application to use SQLAlchemy, and it works now.

But I'm wondering where I should put these lines:

engine = sqlalchemy.create_engine(settings.DATABASE_URL)
Session = sqlalchemy.orm.sessionmaker(bind=engine)
session = Session()

The reason I'm asking is because I want to use SQLAlchemy at many place, and I don't think its correct/powerful/well-written to call this three lines everytime I need to use the database.

The place I will require SA is :

  • In my views, of course
  • In some middleware I wrote
  • In my models. Like in get_all_tags for a BlogPost Model.

What I think would be correct, is to get the session, by re-connecting to the database if the session is closed, or just returning the current, connected session if exists.

How can I use SQLAlchemy correctly with my Django apps?

Thanks for your help!

Note: I already followed this tutorial to implement SA into my Django application, but this one doesn't tell me exactly where to put those 3 lines (http://lethain.com/entry/2008/jul/23/replacing-django-s-orm-with-sqlalchemy/).

Cyril N.
  • 38,875
  • 36
  • 142
  • 243
  • Please forgive me for asking why you'd switch away from Django's model system in the first place. (And I am curious to know why.) – Chris Morgan Jul 07 '11 at 06:41
  • This project will be used by many users and they have access to the template engine (also changed to Jinja2). I changed those original objects (orm &template) because I'm thinking about moving from Django to a micro framework if this project works well, and I don't want to break the user code (templates) because the orm has changed and the way you use the objects are different. Is my answer correct to you? – Cyril N. Jul 07 '11 at 07:52
  • 1
    if your *models* need access to the session, then your models are **wrong**, sessions are at a different layer of abstraction than models – SingleNegationElimination Jul 07 '11 at 07:55
  • I probably misunderstood something. So, how can I list all the tags from a BlogPost's model without using session and not having tags reference in this model? – Cyril N. Jul 07 '11 at 07:59
  • @cx42net: if you're thinking of switching to some other framework then you'll be changing everything. I would advise you to either decide to use Django or decide to use some other framework and then move cleanly and entirely to whichever you choose. If it's aught but a tiny project, I would recommend doing it all with Django rather than using a micro-framework. And "access to the template engine" is quite possible in Django, if that's a consideration you wish to make. – Chris Morgan Jul 07 '11 at 08:07
  • @cx42net: who's calling `get_all_tags`? how come the caller can't create a session? – SingleNegationElimination Jul 07 '11 at 08:21
  • 2
    @Chris Morgan: using sqlalchemy doesn't necessetate switching out the whole framework. Django is only loosely coupled with it's orm. Sqlalchemy is a little more powerful than django's orm, but some other parts of django integrate nicely with the native orm, so it's hard to say without knowing more about the specific application if this would be a good choice. – SingleNegationElimination Jul 07 '11 at 08:26
  • The get_all_tags would be called in a template, from the BlogPost model instamce. That's why the caller can't create the session (or I'm missing something) – Cyril N. Jul 07 '11 at 08:36
  • Think of my project as a Tumblr alternative. The front web site (the one seen by everyone) will only read the database to list blog post, but all my users can make their own theme. An admin area will allow them to add/esit/delete their posts. – Cyril N. Jul 07 '11 at 08:39
  • @TokenMacGuy: I know it doesn't, but I'm suggesting that it's likely to be better to do everything one way or the other. – Chris Morgan Jul 07 '11 at 08:39
  • @cx42net one great advantage of Django ORM is the builtin admin. You will often want to heavily customise it by the end (with styles and templates), but it's a whole lot faster to start with and often does quite well verbatim. – Chris Morgan Jul 07 '11 at 08:40
  • is get all tags a relationship on the blog post? of all tags, a post has some of them? if so, sqlalchemy handles that for you. You don't need special access to the session for that. just use `post.tags`, or whatever you've configured the `relationship` to use. – SingleNegationElimination Jul 07 '11 at 08:40
  • BlogPost & BlogTags are n:n related. So i can do that yeah. But I will sometime need to access to the database in my model ( to count something not directly related for example). The get_all_tags was a wrong example. – Cyril N. Jul 07 '11 at 08:43
  • I won't use Django admin, the admin part is independantly written. – Cyril N. Jul 07 '11 at 08:44
  • search for where to put your [startup code](http://stackoverflow.com/questions/2781383/where-to-put-django-startup-code) and **maybe** this module can help you: [sqlahelper](http://pypi.python.org/pypi/SQLAHelper) – kusut Jul 07 '11 at 07:39
  • beware, `session = Session()` is *not* startup code. – SingleNegationElimination Jul 07 '11 at 07:40
  • Yes, the thing is : i'd like to know where is the best way to connect my application with the db BUT also how can I access the current session from anywhere in my code, as listed on my question. Also, what is suggested on the link is to connect to the database when application starts, but what happen if this application loose the connection after? Does sql alchemy reconnect automatically or do I have to restart my app? – Cyril N. Jul 07 '11 at 07:56
  • being able to 'access the session anywhere' is probably going to bite you later on. sessions are more than just the database connection, they also represent the "Unit of Work". web apps usually have many distinct units of work in their lifetimes, and often have many in flight simultaneously (possibly in the same *thread*). Only you the programmer really know what sql statements belong to what requests, and you express that to sqlalchemy by way of the session. – SingleNegationElimination Jul 07 '11 at 08:12
  • 1
    I have just found [django-sqlalchemy](https://code.google.com/p/django-sqlalchemy/wiki/Roadmap) project. I don't know if it suits to your needs, but it is definitely worth of reading about it. – noisy Jul 02 '13 at 14:33

1 Answers1

26

for the first two, engine and Session, you can put them in settings.py; they are, configuration, after all.

Actually creating a session requires slightly more care, since a session is essentially a 'transaction'. The simplest thing to do is to create it in each view function when needed, and commit them just before returning. If you'd like a little bit more magic than that, or if you want/need to use the session outside of the view function, you should instead define some middleware, something like

class MySQLAlchemySessionMiddleware(object):
    def process_request(self, request):
        request.db_session = settings.Session()

    def process_response(self, request, response):
        try:
            session = request.db_session
        except AttributeError:
            return response
        try:
            session.commit()
            return response
        except:
            session.rollback()
            raise

    def process_exception(self, request, exception):
        try:
            session = request.db_session
        except AttributeError:
            return
        session.rollback()

Then, every view will have a db_session attribute in their requests, which they can use as they see fit, and anything that was added will get commited when the response is finished.

Don't forget to add the middleware to MIDDLEWARE_CLASSES

SingleNegationElimination
  • 151,563
  • 33
  • 264
  • 304
  • Nice one. But I have 2 questions : my code will only read (select) the database. Do I have to commit then? Does creating a mew session at each request will open a new database connection? Or will it adapt (opening a new connection to the db if none exists, re-using one if it exists)? Because I'm affraid about the heavy load if every request create a new db connection. Thanks for your help and your reply. – Cyril N. Jul 07 '11 at 08:03
  • 3
    connection pooling is controlled by the parameters to `sqlalchemy.create_engine`. the parameter of interest is `poolclass`, which has a default value of `QueuePool`, which will create and reuse a maximum of 5 (by default) connections. If you don't want or need the automagic commit behavior, then you can change the recipe arround, maybe eliminate `process_response`, or set a flag on the request (or response) to indicate whether a commit is needed. For the most control over how resources are consumed, just call `Session()` whenever you need a session. – SingleNegationElimination Jul 07 '11 at 08:08
  • If I understand correctly, that mean that if I call 6 times Session(), it will open only 5 connections to the database, and then reuse one of them ? (for the sixth one) – Cyril N. Jul 07 '11 at 11:55
  • 3
    It may only open one connection, if the sessions are garbage collected before the next one is created. but your understanding is sound. You can create thousands of sessions, but at most 5 connections will be open to the database (or whichever value you select) – SingleNegationElimination Jul 07 '11 at 12:20