1

So below I have some code that tests the functionality where someone creates a post and that post has a hash_tag which is "#video" in this case. The code takes the Post body and uses regex to find any word that starts with "#". If it does then it creates or gets that HashTag from the HashTag table. Then sets that list of HashTag to the hash_tags attribute under Post.

For some reason the CreatePostSerializer serializer is throwing an exception that doesn't make sense. The serializer is throwing the exception ValidationError({'hash_tags': [ErrorDetail(string='Invalid pk "[\'video\']" - object does not exist.', code='does_not_exist')]}). The reason this doesn't make sense is because when I debug and set a breakpoint right after except Exception as e under views.py this is what I get

>>>e
ValidationError({'hash_tags': [ErrorDetail(string='Invalid pk "[\'video\']" - object does not exist.', code='does_not_exist')]})
>>>HashTag.objects.get(pk='video')
<HashTag: HashTag object (video)>
>>>request.data['hash_tags']
['video']

So the >>> represents what I input into the debugger. I'm essentially stopped at the line return Response... and we can see e is the ValidationError I mentioned, but we can see that the object it claims doesn't exist does indeed exist. Why is the serializer throwing a "ValidationError - object does not exist" when it does?

Note: I have another test that does exactly the same thing and passes except no video file is being passed this leads me to believe that Django is doing something different in the case that the incoming body is multi-part. I also tried in the instance that there is only one hash tag to set hash_tags=<single hash tag> rather than a list and it worked. This is a hack though and cleaner solution is preferred.

helpers.py

import re

def extract_hashtags(text):
    regex = "#(\w+)"
    return re.findall(regex, text)

test.py

def test_real_image_upload_w_hash_tag(self):
    image_file = retrieve_test_image_upload_file()
    hash_tag = 'video'
    response = self.client.post(reverse('post'),
                                data={'body': f'Some text and an image #{hash_tag}',
                                      'images': [image_file]},
                                **{'HTTP_AUTHORIZATION': f'bearer {self.access_token}'})
    self.assertEqual(response.status_code, status.HTTP_201_CREATED)

views.py

def set_request_data_for_post(request, user_uuid: str):
    request.data['creator'] = user_uuid
    post_text = request.data['body']
    hash_tags_list = extract_hashtags(post_text)
    hash_tags = [HashTag.objects.get_or_create(hash_tag=ht)[0].hash_tag for ht in hash_tags_list]

    if len(hash_tags) > 0:
        request.data['hash_tags'] = hash_tags

    return request

def create_post(request):
    user_uuid = str(request.user.uuid)
    request = set_request_data_for_post(request=request, user_uuid=user_uuid)

    try:
        serializer = CreatePostSerializer(data=request.data)
        if serializer.is_valid(raise_exception=True):
            post_obj = serializer.save()
    except Exception as e:
        return Response(dict(error=str(e),
                             user_message=error_message_generic),
                        status=status.HTTP_400_BAD_REQUEST)

    return Response(serializer.data, status=status.HTTP_201_CREATED)

serializer.py

from rest_framework import serializers
from cheers.models import Post

class CreatePostSerializer(serializers.ModelSerializer):
    class Meta:
        model = Post
        fields = ('creator', 'body', 'uuid', 'created', 'updated_at', 'hash_tags')

model.py

class Post(models.Model):
    # ulid does ordered uuid creation
    uuid = models.UUIDField(primary_key=True, default=generate_ulid_as_uuid, editable=False)
    created = models.DateTimeField('Created at', auto_now_add=True)
    updated_at = models.DateTimeField('Last updated at', auto_now=True, blank=True, null=True)
    creator = models.ForeignKey(
        User, on_delete=models.CASCADE, related_name="post_creator")
    body = models.CharField(max_length=POST_MAX_LEN, validators=[MinLengthValidator(POST_MIN_LEN)])
    hash_tags = models.ManyToManyField(HashTag, blank=True)

class HashTag(models.Model):
    hash_tag = models.CharField(max_length=HASH_TAG_MAX_LEN, primary_key=True, validators=[
        MinLengthValidator(HASH_TAG_MIN_LEN)])
  • Try `request.data['hash_tags'] = hash_tags` → `request.data.setlist('hash_tags', hash_tags)`. – aaron Dec 03 '21 at 07:57
  • Does that answer your question? – aaron Dec 04 '21 at 14:42
  • @aaron sorry I was busy. I'll give it a try soon and let you know! –  Dec 05 '21 at 00:28
  • yep looks like it works! I noticed it gives an error if hash_tags is `None`. Is there a way to allow this? I also use this serializer for patching the data so sometimes I want to clear the hashtags if the edit has no `hash_tags.` If I do `request.data['hash_tags]=None` or `[]` on the set partial it throws an exception. –  Dec 05 '21 at 00:38
  • @aaron hmm, but it seems like my other test that was succeeding is now failing. The one without the image. –  Dec 05 '21 at 00:41
  • @aaron so the issue seems to be when the request doesn't include an image it's a dictionary otherwise it's an ordered dict that seems to be what the problem is stemming from. –  Dec 05 '21 at 01:01
  • Can you create a repro project on GitHub? – aaron Dec 05 '21 at 01:32
  • @aaron I do have a repo, but it's private there are security keys on it and what not. –  Dec 05 '21 at 01:33
  • Create a project without them. – aaron Dec 05 '21 at 03:10
  • @aaron I think I know what the issue is. Has to do with setting the data if it's a QueryDict or regular Dict. If you add a file Django converts request to QueryDict, which bugs out if you set wit key value i.e. `query_dict[key] = value` for manyTomany attributes –  Dec 05 '21 at 03:35

1 Answers1

1

under your test/__init__.py you have to add these lines

from django.db.backends.postgresql.features import DatabaseFeatures

DatabaseFeatures.can_defer_constraint_checks = False

There's some weird internal bug where if you operate on one table a lot with a lot of different TestCase classes then it'll do a DB check at the end after it's been torn down and it'll cause an error.

I'm also using factory boy (https://factoryboy.readthedocs.io/en/stable/orms.html) to generate my test DB, which is the main reason this issue arises. The reason I believe this is because I switched out factory boy for just using <model>.objects.create() and my tests stopped failing.

  • This seems like a bandage that solves the immediate error but covers up deeper issues in your test cases. – Code-Apprentice Dec 20 '21 at 20:01
  • what makes you call it a "bandage"? Apparently this constraint_check is unnecessary. –  Dec 21 '21 at 00:42
  • I think the underlying issue stems from factory boy combined with test suite in Django. I extended my answer. –  Dec 21 '21 at 00:44
  • Disabling features often is a bandage. Here you are disabling foreign key constraints. I'm glad it works for you, but hopefully it doesn't mask some deeper issue with your tests. – Code-Apprentice Dec 25 '21 at 04:48
  • @Code-Apprentice is that what it does? When this was suggested to me they just told me it defers constraint checks to the end of TestCase rather than end of each test function. That's definitely not good if it does that. –  Dec 25 '21 at 04:49
  • I'm probably not using the correct wording. Either way disabling or deferring feels like a bandage since it modifies the behavior of the database. – Code-Apprentice Dec 25 '21 at 04:51
  • well the main issue stems from FactoryBoy to generate test data. I'll just use `model.objects.create` instead. Thanks for the heads up you probably saved me a lot of headache later. Add as an answer and I'll vote it up. –  Dec 25 '21 at 05:10
  • Magic! This somehow solved a `ForeignKeyViolation` with pytest fixtures having many-to-many relations (Django 4.2.4, pytest 7.4.0, pytest-django 4.5.2). Thanks! – utapyngo Aug 17 '23 at 03:37