Unable to retrieve database objects in Flask test case session

Question

I am testing an API created using flask-restful, sqlalchemy, flask-sqlalchemy and factory-boy.

I have encountered a strange issue where objects created before GET requests using factory are available but objects created before post/put calls are unavailable.

I have set up a test case class based on flask-testing:

# tests/common.py
from flask_testing import TestCase as FlaskTestCase

from app import create_app
from database import db
from config import TestConfig


class TestCase(FlaskTestCase):
    def create_app(self):
        return create_app(TestConfig)

    def setUp(self):
    db.create_all()

    def tearDown(self):
        db.session.remove()
        db.drop_all()

Test factory:

# tests/factory.py

import factory

from database import db

from ..models import Item

class ItemFactory(factory.alchemy.SQLAlchemyModelFactory):
    class Meta:
        model = Item
        sqlalchemy_session = db.session

Resources:

# resources.py
from flask import request
from flask_restful import Resource

from database import db
from .models import Item
from .serializers import ItemSerializer

class ItemResource(Resource):
    def get(self, symbol):
        obj = db.session(Item).filter(Item.symbol == symbol).first_or_404()
    return ItemSerializer(obj).data

    def put(self, symbol):
        params = request.get_json(silent=True)
        query = db.session(Item).filter(Item.symbol == symbol).update(params)

        db.session.commit()

        obj = db.session(Item).filter(Item.symbol == symbol).first_or_404()
        return ItemSerializer(obj).data

Tests:

# tests/test_resources.py
import json

from tests.common import TestCase
from tests.factory import ItemFactory


def parse_response(response):
    return json.loads(response.get_data().decode())


class ResourcesTest(TestCase):
    def test_get_item(self):
        symbol = 'TEST'
        ItemFactory(symbol=symbol)

        response = self.client.get('/api/v1/items/%s' % symbol)
        results = parse_response(response)

        self.assertEqual(response.status_code, 200)
        self.assertEqual(results['symbol'], symbol)

    def test_update_item(self):
        symbol = 'TEST'
        new_symbol = 'TEST_NEW'
        ItemFactory(symbol=symbol)

        response = self.client.put('/api/v1/items/%s' % symbol, json={'symbol': new_symbol})
        results = parse_response(response)

        self.assertEqual(response.status_code, 200)

In test_update_item I receive a 404 instead. Before we enter self.client.put, checking the database shows that the new Item was created. However, when we reach the put method in the resource, db.session(Item).query.all() returns an empty array.

From flask-testing documentation, I spotted this paragraph:

Another gotcha is that Flask-SQLAlchemy also removes the session instance at the end of every request (as should any thread safe application using SQLAlchemy with scoped_session). Therefore the session is cleared along with any objects added to it every time you call client.get() or another client method.

I believe that the issue lies with the way sessions are treated but have not been able to come out with any solutions to ensure that my tests pass. Another interesting observation was that for self.client.put or self.client.post, if I change the data being posted as self.client.put('/some/url', data={'symbol': symbol}), the objects will be available in the session.

score 1 · Answer 1 · answered Jun 25 '18 at 04:19

Ok, found a convoluted way to solve this issue. I think calling request.get_json instantiates a session else where. The solution would be to override the way json data is posted.

In tests/common.py:

import json

from flask_testing import TestCase as FlaskTestCase
from flask.testing import FlaskClient

from app import create_app
from database import db
from config import TestConfig


class TestClient(FlaskClient):
    def open(self, *args, **kwargs):
        if 'json' in kwargs:
            kwargs['data'] = json.dumps(kwargs.pop('json'))
            kwargs['content_type'] = 'application/json'
        return super(TestClient, self).open(*args, **kwargs)


class TestCase(FlaskTestCase):
    def create_app(self):
        app = create_app(TestConfig)
        app.test_client_class = TestClient
        return app

    def setUp(self):
        db.create_all()

    def tearDown(self):
        db.session.remove()
        db.drop_all()

So basically we are converting any json kwargs in post/put calls in the test client into data and also setting the content_type to json. Stumbled upon this here (https://stackoverflow.com/a/40688088).

score 0 · Answer 2 · answered Jun 26 '18 at 10:18

You may need to take a look at the SQLAlchemy session handling.

The factory_boy documentation describes a few options: https://factoryboy.readthedocs.io/en/latest/orms.html#managing-sessions

It looks like the factory creates objects in a SQLAlchemy session, but that session is not written to the database. When your code goes into the flask side, that context might be using another connection to the database (see the flask-testing doc you quoted), thus not seeing the objects that haven't been committed yet.

Unable to retrieve database objects in Flask test case session

2 Answers2