Django Migration Error: Column does not exist

Question

Python 3, Django 1.8.5, Postgres

I have a model Sites that has been working fine. I recently tried to add a field, airport_code, and migrate the data.

class Site(BaseModel):

  objects = SiteManager()

  name = models.CharField(max_length=200, unique=True)
  domain = models.CharField(max_length=200, unique=True)
  weather = models.CharField(max_length=10)
  nearby_sites = models.ManyToManyField('self', symmetrical=False, blank=True)
  users = models.ManyToManyField(settings.AUTH_USER_MODEL, blank=True)
  facebook = models.URLField(max_length=200)
  twitter = models.URLField(max_length=200)
  header_override = models.TextField(blank=True)
  email_header_override = models.TextField(blank=True)
  timely_site_tag_id = models.IntegerField()
  timely_featured_tag_id = models.IntegerField()
  timely_domain = models.CharField(max_length=255)
  sitemap_public_id = models.CharField(max_length=255)
  state = models.CharField(max_length=24)
  airport_code = JSONField()

However, when I ran makemigrations I got an error:

django.db.utils.ProgrammingError: column sites_site.airport_code does not exist LINE 1: ..._site"."sitemap_public_id", "sites_site"."state", "sites_sit...

Of course, this doesn't make sense, because the column obviously does not exist when I'm trying to create it within the migration.

I have seen many questions about this bug on Stack Overflow that are unanswered, or have a solution to manually create the migration file, or destroy and rebuild the database. This is not an okay solution.

Answer 1

After you run makemigrations, be sure to go through the stack trace step by step.

In my case, I noticed it traced through a call to a Form contained within a forms.py in a completely different app, which happened to have a call to the model I was trying to create a new migration for.

Moving the Form class out of forms.py into the views.py fixed the issue.

Answer 2

This bug was resolved for me by commenting out the django debug toolbar from INSTALLED_APPS in settings.py. I am not sure why debug toolbar is the culprit, but after I commented it out, I was able to run makemigrations and migrate with no issue.

Hoping this helps someone, as I spent twelve hours trying to figure it out.

Answer 3

I ran into this issue as well and the answer by @Nexus helped. I thought I'd provide details of my specific case here for better illustrate the cause of the issue. It seems like a potential bug to me.

I have a model Brand as follows:

class Brand(BaseModelClass):
    name = CharField(max_length=256, unique=True)
    website = ForeignKey(URL, on_delete=CASCADE, null=True, blank=True)

I was running a python manage.py makemigrations after adding a Boolean field as follows:

class Brand(BaseModelClass):
    name = CharField(max_length=256, unique=True)
    website = ForeignKey(URL, on_delete=CASCADE, null=True, blank=True)
    trusted = Boolean(default=True)

When running the makemigrations command, I recieved an error similar to OP's:

django.db.utils.ProgrammingError: column appname_brand.trusted does not exist

Per @Nexus' suggestion, I went through the stacktrace, line-by-line, assuming that it wasn't some core issue with Django. As it turns out, in one of apps forms.py file I had the following:

choices={(str(brand.id), brand.name) for brand in Brand.objects.all()}

The solution was to simply comment-out that line, run manage.py makemigrations, then run manage.py migrate. Afterwards, I uncommented the line and everything and my forms' functionality worked just as before.

Answer 4

In my case, that was because I had a unique_together constraint that was set.

When I wanted to remove a field, the auto-generated migration tried to remove the field before removing the unique_together constraint.

What I had to do was manually move up the migrations.AlterUniqueTogether constraint in the migration file, so that django removes first the constraint before trying to remove the field.

I hope this can help someone.

Answer 5

Make sure you are not doing any queries when loading the application!, as eg. in:

class A:
  field = fn_that_makes_query()

When running migrate or makemigrations, Django performs system checks, which loads the entire application, so if during this process any queries are made which use added/altered db fields you run into inconsitencies, because you are trying to access db fileds that are not there yet.

Answer 6

I too got same issue when i executed a cmd like python manage.py makemigrations app1. I resolved my issue by commenting the custom orms which is running in another app2 under views.

Example:

inside app1 models.py

class ModelA(models.Model):
    subject = models.CharField(max_length=1)
    course = models.CharField(max_length=1)

inside app2 view.py

# obj = ModelA.object.all()
# get_subjct = [s.subject for s in obj]

So here i have commented above code and executed the makemigrations and migrate then uncommented.

Working fine...

Answer 7

I got the same problem (column not exist) but when I try to run migrate not with makemigrations

• Cause: I removed the migration files and replaced them with single pretending intial migration file 0001 before running the migration for the last change

• Solution:

  1. Drop tables involved in that migration of that app (consider a backup workaround if any)
  2. Delete the rows responsible of the migration of that app from the table django_migrations in which migrations are recorded, This is how Django knows which migrations have been applied and which still need to be applied.

And here is how solve this problem:

• log in as postgres user (my user is called posgres):

  sudo -i -u postgres

• Open an sql terminal and connect to your database:

  psql -d database_name

• List your table and spot the tables related to that app:

  \dt

• Drop them (consider drop order with relations):

  DROP TABLE tablename ;

• List migration record, you will see migrations applied classified like so:

id | app | name | applied
--+------+--------+---------+

SELECT * FROM django_migrations;

• Delete rows of migrations of that app (you can delete by id or by app, with app don't forget 'quotes'):

  DELETE FROM django_migrations WHERE app='your_app';

• log out and run your migrations merely (maybe run makemigrations in your case):

  python manage.py migrate --settings=your.settings.module_if_any

Note: it is possible that in your case will not have to drop all the tables of that app and not all the migrations, just the ones of the models causing the problem.

I wish this can help.

Answer 8

Run into this problem after the migration of my postgres database to a differnt server. Somehow I messed up the database and could not update my model with the new class UserProfile.

I solved the problem creating initial migration for existing schema:

1. Empty the django_migrations table: delete from django_migrations;with a command DELETE FROM django_migrations WHERE app='my_app';
2. For every app, delete its migrations folder: rm -rf <app>/migrations/
3. Reset the migrations for the "built-in" apps: python manage.py migrate --fake
4. For each app run: python manage.py makemigrations <app>. Take care of dependencies (models with ForeignKey's should run after their parent model).
5. Finally: python manage.py migrate --fake-initial

Got it here: https://stackoverflow.com/a/29898483

PS I'm not sure that this was relevant to the resolution of the problem but, first, I dropped the table in postgresql that caused an error and commented out the UserProfile class in models.

in shell:

 sudo -su postgres
 psql databse_name   
 DROP TABLE table_name;

models.py:

#class UserProfile(models.Model):
    #user = models.OneToOneField(settings.AUTH_USER_MODEL, unique=True, primary_key=True, on_delete=models.CASCADE, related_name='user_profile')
    #avatar = ThumbnailerImageField(upload_to='profile_images', blank=True)
    #country = models.CharField(max_length = 128)

Answer 9

I ran into this problem recently after upgrading to Django 1.11. I wanted to permanently address the issue so I wouldn't have to comment / uncomment code every time I ran a migration on the table, so my approach:

from django.db.utils import ProgrammingError as AvoidDataMigrationError

try:
    ... do stuff that breaks migrations
except AvoidDataMigrationError:
    pass

I rename the exception during import to AvoidDataMigrationError so it's clear why it's there.

Answer 10

Just now had the same error when I tried to migrate a SingletonModel to actually contain the necessary fields.

Reason for the error was that my Model A used some fields of this SingletonModel (as configurable values). And during instantation of model A during the migration process it obviously couldn't guarantee that my migration was safe.

A colleague had a wonderful idea. Make the default value for the field a function call, and therefor lazy.

Example:

class A (models.Model):
    default_value = models.DecimalField(default: lambda: SingletonModel.get_solo().value, ...)

So therefor, my advice: Try and make the offending call (seen in stacktrace) a lazy one.

Answer 11

I got the same issue, here is my case: in the app MyApp I add a new field to the model:

class Advisors(models.Model):
    AdvID = models.IntegerField(primary_key=True)
    Name = models.CharField(max_length=200,null=False)
    ParentID = models.IntegerField(null=True) # <--- the NEW field I add

so what I did is: in the urls.py of MyProject, NOT MyApp, comment out the url() linked to MyApp and then do makemigrations and migrate everything goes well; in MyApp/urls.py file:

urlpatterns = [
    url(r'^admin/', admin.site.urls, name='admin'),
    url(r'^$', views.HomePage.as_view(),name='home'),
    #comment out the following line, after migrate success, bring it back;
    # url(r'^myapp/', include('myapp.urls',namespace='research')), <--- 
]

Answer 12

Late to the party, but there is some info I want to share. It helped me a lot! :)

I am using django-solo to store app config in database, and I got the same issue as Alex when adding new field to configuration model. Stacktrace pointed me to the line where I'm getting config from database: conf = SiteConfiguration().get_solo().

There is a closed issue for django-solo project on Github. The idea is to make model loading lazy (exactly as MrKickkiller pointed before).

So for modern Django version (in my case 3.0) this code works perfectly:

from django.utils.functional import SimpleLazyObject
conf = SimpleLazyObject(SiteConfiguration.get_solo)

Answer 13

Stuck into this issue recently.

In my case, I added a reference to a non-existing field in the code, then I came to the model file and added the new field, then tried to run makemigrations command which thrown the above error.

So I went to the stack trace all the way up and found the newly added reference was the problem. commented that out, ran makemigrations and voila.

Answer 14

In my case it happens because of my custom AdminSite has MyModel.objects.filter on application start, i have disabled it for makemigrations and migrate database.

Answer 15

I faced this problem. for solving problem follow the step.
1.open Database and table.
2.Create a required(sites_site.airport_codes in question ) column which was not exits in your table with default value.

3.run the command

python manage.py makemigrations

python manage.py migrate <app name>

python manage.py runserver

Answer 16

Got the same issue, thanks to this thread to point me out the way by exploring the backtrace.

In a ListView declaration, I was declaring a queryset with a filter pointing to the model I was trying to update, without overiding the get_query_set() function.

    class BacklogListView(ListView):
        model = Version
        context_object_name = 'vhs'
        template_name = 'backlog_list.html'    
        queryset = VersionHistory.objects.filter(extract=Extract.objects.last())

fixed by:

    class BacklogListView(ListView):
        model = Version
        context_object_name = 'vhs'
        template_name = 'backlog_list.html'
    
        def get_queryset(self):
            queryset = VersionHistory.objects.filter(extract=Extract.objects.last())

If it can help someone...

Answer 17

You might have a RunPython using the new state of the model you need before modifying your database, to fix that, always use apps.get_model

def forwards_func(apps, schema_editor):
    # We get the model from the versioned app registry;
    # if we directly import it, it'll be the wrong version
    SiteModel = apps.get_model('my_app', 'Site')

    # DONT
    from my_app.models import Site

def reverse_func(apps, schema_editor):
    ...

Answer 18

This issue happens when you try to migrate an app X which has a related attribute to an app Y before Y migrate.

So, to solve it you have to make the migrations of Y first. You can specify the order migrations by using the django dependencies property

You can also, run:

python manage.py migrate --run-syncdb

then for every error caused app (like django.db.utils.ProgrammingError: relation "AppName_attribute" does not exist)run:

python manage.py migrate AppName

P.S:

If you need to reset the db schema, you can use:

manage.py reset_schema

And don't forget to remove all migrations files:

find . | grep -E "(__pycache__|\.pyc|\.pyo$|migrations)" | xargs rm -rf

Answer 19

For my case i am getting the error when migrating. I solved the issue using the below steps.

After makemigration the created file was.

# Example:
class Migration(migrations.Migration):

    dependencies = [
        ('app1', '0030_modify'),
    ]

    operations = [
        migrations.RemoveField(
            model_name='users',
            name='phone1',
        ),
        migrations.RemoveField(
            model_name='users',
            name='phone2',
        ),
        migrations.AlterUniqueTogether(
            name='users',
            unique_together={('email', 'phone')},
        ),
        
    ]

I solved the issue by moving the unique_together migration first, then field removals.

# Example:
class Migration(migrations.Migration):

    dependencies = [
        ('app1', '0030_modify'),
    ]

    operations = [
        migrations.AlterUniqueTogether(
            name='users',
            unique_together={('email', 'phone')},
        ),
        migrations.RemoveField(
            model_name='users',
            name='phone1',
        ),
        migrations.RemoveField(
            model_name='users',
            name='phone2',
        ),
    ]

Hope it will solve your problem