Django 1.6 TransactionManagementError: database doesn't behave properly when autocommit is off

Question

I'm trying to update a project from Django 1.5.5 to Django 1.6 however I've been getting this error everywhere.

Traceback (most recent call last):

File "project/virtualenv/lib/python2.7/site-packages/django/core/handlers/base.py", line 114, in get_response
  response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 215, in wrapper
  return self.admin_view(view, cacheable)(*args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view
  response = view_func(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
  response = view_func(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 197, in inner
  return self.login(request)

File "project/virtualenv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
  response = view_func(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 330, in login
  return login(request, **defaults)

File "project/virtualenv/lib/python2.7/site-packages/django/views/decorators/debug.py", line 75, in sensitive_post_parameters_wrapper
  return view(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view
  response = view_func(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
  response = view_func(request, *args, **kwargs)

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/auth/views.py", line 43, in login
  auth_login(request, form.get_user())

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 83, in login
  request.session.cycle_key()

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py", line 277, in cycle_key
  self.create()

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py", line 40, in create
  self.save(must_create=True)

File "project/virtualenv/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py", line 62, in save
  with transaction.atomic(using=using):

File "project/virtualenv/lib/python2.7/site-packages/django/db/transaction.py", line 244, in __enter__
  "Your database backend doesn't behave properly when "

TransactionManagementError: Your database backend doesn't behave properly when autocommit is off. Turn it on before using 'atomic'.

I've removed TransactionMiddleware from MIDDLEWARE_CLASSES and replaced it with ATOMIC_REQUESTS = True. (Same error even if I don't do this step)

Can someone please shed some light on this?

Are you trying to use a command that uses an atomic save (ie, get_or_create()), within a code block that has been decorated with a @transaction — OldTinfoil, Nov 20 '13 at 12:48
I'm just using Django's generic views and doing get_or_create in south datamigrations. I haven't explicitly decorated any views with @transaction myself, but I'm not sure if Django automatically does this. Even so, Django still throws the same error when doing the datamigration. — SunnySydeUp, Nov 20 '13 at 23:35
How are you using get_and_create() within the South Migrations? I pretty sure South uses @transactions for optimisation (if it doesn't, I'd be very surprised). If you replace the get_or_create() with the full expanded definition (ie, the try/catch block) and that should play nicely... — OldTinfoil, Nov 21 '13 at 18:26
In the `forwards` function, I would do: `orm["auth.Group"].objects.get_or_create(name="blah")`. I'll give your suggestion a go when I get the chance. — SunnySydeUp, Nov 21 '13 at 22:09
Sorry for the long delayed reply, been busy with other stuff. While adding a try/catch block in the migration files makes this particular error go away, many other errors pop up such as 'Site matching query does not exist' when I run django tests. Pure speculation, but I don't think objects can be created at all while in testing. However, I've got `TransactionManagementError` to go away when running the server on a browser by calling `django.db.transcation.set_autocommit(True)` in the shell prior to `runserver`. I'm unsure of how to do it for for the test database though. — SunnySydeUp, Dec 05 '13 at 02:10
Hey no worries bud, as long as you've got something working ;). You'd probably be referencing objects in the database that didn't exist yet. — OldTinfoil, Dec 06 '13 at 12:06
Except the `Site`s should be added from the initial_data fixture. — SunnySydeUp, Dec 06 '13 at 23:13

score 22 · Answer 1 · answered Dec 20 '13 at 16:28

22

I ran into this with sqlite3 db, using Django 1.6. Here are the solutions.

django.middleware.transaction.TransactionMiddleware has been deprecated. If you don't have this in your settings.py file, you should not get the error.
Accidentally, I found that including ATOMIC_REQUESTS: True works around the error if you had left django.middleware.transaction.TransactionMiddleware in your middlewares list.

E.g.

DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.sqlite3',
    'NAME': 'sqlite3-db',
    'ATOMIC_REQUESTS': True
  }
}

answered Dec 20 '13 at 16:28

Overclocked

1,187
1
11
22

1

Did the second solution work for you when you ran tests? – SunnySydeUp Dec 21 '13 at 00:46
Didn't try, since I opted for the 1st solution. – Overclocked Dec 21 '13 at 13:13
Well TransactionMiddleware has been deprecated in favour of ATOMIC_REQUESTS. And I have stated in my question that I have already removed it from the list of middlewares. – SunnySydeUp Dec 27 '13 at 22:42
18

@Overclocked: I don't think this works, I don't have teh TransactionMiddleware and both with and without the `ATOMIC_REQUESTS` option I get this same error from the sessions backend. – Wolph Feb 20 '14 at 15:33
@Wolph Did you figure out the issue? I am stuck with your exact issue for a while now. – karthikr Jul 26 '14 at 04:55
@karthikr: I simply downgraded Django to a version that still worked. Was on a deadline and didn't have time to figure it out. – Wolph Jul 26 '14 at 08:05
Thanks, disabling atomic requests helped – alexche8 Apr 26 '17 at 10:34

score 14 · Answer 2 · answered Apr 05 '14 at 01:30

I had the same issue in my forwards migration (interestingly, didn't happen in my backwards migration), and don't have TransactionMiddleware in my settings. My workaround was to avoid using the get_or_create method and instead do the same thing more verbosely. From the Django docs:

try:
    obj = Person.objects.get(first_name='John', last_name='Lennon')
except Person.DoesNotExist:
    obj = Person(first_name='John', last_name='Lennon', birthday=date(1940, 10, 9))
    obj.save()

You can then create your own pseudo-get_or_create method like this:

def fake_get_or_create(model, *args, **kwargs):
    try:
        obj = model.objects.get(**kwargs)
    except model.DoesNotExist:
        obj = model(**kwargs)
        obj.save()
    return obj

Which you can use by doing the following

obj = fake_get_or_create(Person, first_name='John', last_name='Lennon')

This solution worked for me when upgrading to django 1.6.5 while using South migrations. — rhigdon, Jun 18 '14 at 19:04

score 7 · Answer 3 · answered Apr 03 '14 at 03:09

7

I ran into the same problem when using sqlite3. I found out that I was using transaction.commit_on_success. On changing that to transaction.atomic(), the problem was resolved.

answered Apr 03 '14 at 03:09

Ramashish Baranwal

7,166
2
18
14

This is the only thing that worked for me here. Thanks! – Max Apr 29 '14 at 14:34

score 1 · Answer 4 · answered Dec 27 '13 at 16:55

I believe the error is due to limitations of Sqlite3. To resolve this, I had to switch from Sqlite3 to a more robust database engine like postgrsql_psycopg2.

The code throwing the error (transaction.py:244) provides a clue in the comment:

        if not connection.get_autocommit():
            # Some database adapters (namely sqlite3) don't handle
            # transactions and savepoints properly when autocommit is off.
            # Turning autocommit back on isn't an option; it would trigger
            # a premature commit. Give up if that happens.
            if connection.features.autocommits_when_autocommit_is_off:
                raise TransactionManagementError(
                    "Your database backend doesn't behave properly when "
                    "autocommit is off. Turn it on before using 'atomic'.")

Looking at the latest South Documentation (0.8.4) sheds more light on issues with Sqlite3: http://south.readthedocs.org/en/latest/databaseapi.html#database-specific-issues

SQLite doesn’t natively support much schema altering at all, but South has workarounds to allow deletion/altering of columns. Unique indexes are still unsupported, however; South will silently ignore any such commands.

In my case, I have unique indexes in my models which appear to be unsupported.

Thanks for the suggestion. But unfortunately, I'm stuck with sqlite in the meantime. — SunnySydeUp, Dec 28 '13 at 00:10

score 1 · Answer 5 · answered Aug 30 '14 at 15:38

1

add these to your migrations

def forwards(self, orm):
    if connection.vendor == 'sqlite':
        set_autocommit(True)

and it will set the auto commit to true for migrations :)

answered Aug 30 '14 at 15:38

Yousuf Jawwad

3,037
7
33
60

Django 1.6 TransactionManagementError: database doesn't behave properly when autocommit is off

5 Answers5

Linked