Migrate data from original User model to Custom User model in django 1.8

Question

I am created a custom user model. There were already have users in the auth database. So, I using the data migration to migrate the data to my custom user model.

This is how I did the data migration in the auto migration file (which I found from here):

UPDATED

from __future__ import unicode_literals

from django.db import models, migrations


class Migration(migrations.Migration):

    dependencies = [
        ('userauth', '0002_auto_20150721_0605'),
    ]

    operations = [
        migrations.RunSQL('INSERT INTO userauth_userauth SELECT * FROM auth_user'),
        migrations.RunSQL('INSERT INTO userauth_userauth_groups SELECT * FROM auth_user_groups'),
        migrations.RunSQL('INSERT INTO userauth_userauth_user_permissions SELECT * FROM auth_user_user_permissions'),
    ]

models.py

class UserManager(BaseUserManager):
        def _create_user(self, username, email, password, is_staff, is_superuser, **extra_fields):
            now = timezone.now()
            if not username:
              raise ValueError(_('The given username must be set'))
            email = self.normalize_email(email)
            user = self.model(username=username, email=email,
                     is_staff=is_staff, is_active=False,
                     is_superuser=is_superuser, last_login=now,
                     date_joined=now, **extra_fields)
            user.set_password(password)
            user.save(using=self._db)
            if not is_staff:
                group = Group.objects.get(name='normal')
                user.groups.add(group)
            return user

        def create_user(self, username, email=None, password=None, **extra_fields):
            return self._create_user(username, email, password, False, False,
                     **extra_fields)

        def create_superuser(self, username, email, password, **extra_fields):
            user=self._create_user(username, email, password, True, True,
                         **extra_fields)
            user.is_active=True
            user.save(using=self._db)
            return user


    class UserAuth(AbstractBaseUser, PermissionsMixin):
        #original fields
        username = models.CharField(_('username'), max_length=30, unique=True,
        help_text=_('Required. 30 characters or fewer. Letters, numbers and @/./+/-/_ characters'),
        validators=[
          validators.RegexValidator(re.compile('^[\w.@+-]+$'), _('Enter a valid username.'), _('invalid'))
        ])
        first_name = models.CharField(_('first name'), max_length=30, blank=True, null=True)
        last_name = models.CharField(_('last name'), max_length=30, blank=True, null=True)
        email = models.EmailField(_('email address'), max_length=255, unique=True)
        is_staff = models.BooleanField(_('staff status'), default=False,
        help_text=_('Designates whether the user can log into this admin site.'))
        is_active = models.BooleanField(_('active'), default=False,
        help_text=_('Designates whether this user should be treated as active. Unselect this instead of deleting accounts.'))
        date_joined = models.DateTimeField(_('date joined'), default=timezone.now)
      #additional fields
        full_name =  models.CharField(_('Full Name'), max_length=600, blank=True, null=True)
        profileImage = models.ImageField(upload_to="upload",blank=True,null=True,
        help_text = _("Please upload your picture"))
        BioUser =  models.TextField(blank=True,null=True)
        Social_link = models.URLField(blank=True,null=True)

        objects = UserManager()
        USERNAME_FIELD = 'username'
        REQUIRED_FIELDS = []

        class Meta:
            verbose_name = _('user')
            verbose_name_plural = _('users')

        def get_full_name(self):
            return self.full_name

        def get_short_name(self):
            return self.first_name

        def email_user(self, subject, message, from_email=None):
            send_mail(subject, message, from_email, [self.email])

The problem after the migration, I can't create new user. I get this error:

duplicate key value violates unique constraint "userauth_userauth_pkey" DETAIL: Key (id)=(3) already exists.

Seem the table is out of sync. How do I fix this?

Answer 1

On Postgres, the way Django handles creating unique primary keys for its database records is by using a database sequence from which it gets new primary keys. Looking in my own database, I see that if I have a table named x, then Django creates the sequence under the name x_id_seq. So the sequence for your userauth_userauth table would be userauth_userauth_id_seq.

Now, the way you did your migration you went with raw SQL statements. This completely bypasses Django's ORM, which means that the sequences for the new tables were not touched by the migration. What you should do after performing such a raw migration is to set the primary key sequence to a number that won't clash with what is already in the database. Borrowing from this answer, then you should issue:

select setval('userauth_userauth_id_seq', max(id)) 
       from userauth_userauth;

And perform the same kind of operation for the other tables: set their own sequences to the max value if their id field. (If you wonder, the next value that will used will be obtained through nextval and will be equal to one more than the value of the sequence before nextval is called.)

In a comment you wondered why creating new users eventually worked. Probably what happened is that you were trying to create new users and it went like this:

1. Django got a new primary key from the appropriate sequence. Here the sequence gets incremented.

2. Django tried to save the new user, which failed because the number it got in the previous step was not unique.

If you do this enough times, your sequence will get incremented for each try because irrespective of transactions, Postgres does not rollback sequences. The documentation says:

Important: Because sequences are non-transactional, changes made by setval are not undone if the transaction rolls back.

So eventually, the sequence increases past the maximum primary key already in the table, and from that point onwards it works.

Answer 2

What database are you using? It sounds like your database has a counter for the primary key that is generating ids like 3. Since you created new rows with primary keys, you may need to manually reset the DB counter. For an example in postgres, see How to reset postgres' primary key sequence when it falls out of sync?