1

Im hitting an issue after configuring sqlalchemy ORM with sqlite in multithreaded environment. Here is what I'm trying to achieve. Im trying to subscribe to mqtt topic using Paho mqtt python library. The mqttsubscribe class is running in separate thread. The MqttSubscribe class callbacks the function I registered while initializing the class. Whenever there is message im trying to write to database using models.py defined. Here is the sample code of models.py.

import os
import datetime
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy.pool import StaticPool, NullPool
from sqlalchemy.engine import Engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine, event, Column, Integer, String, DateTime, Boolean, ForeignKey, and_, desc

Base = declarative_base()
Session = sessionmaker()
session = ''

# Sqlite doesnt support foreign key by default
# Below code is logic is taken from
# https://stackoverflow.com/questions/31794195/
# how-to-correctly-add-foreign-key-constraints-to-sqlite-db-using-sqlalchemy
@event.listens_for(Engine, "connect")
def set_sqlite_pragma(connection, connection_record):
    cursor = connection.cursor()
    cursor.execute("PRAGMA foreign_keys=ON")
    cursor.close()

def init_db_connection(engine, echo=True):
    global session
    engine = create_engine(engine, echo=echo)
    Session.configure(bind=engine, connect_args={'check_same_thread':False},
        poolclass=NullPool)
    session = Session()
    Base.metadata.create_all(engine)
    return session

class Person(Base):
    __tablename__ = 'Person'

    id = Column(Integer, primary_key=True)
    first_name = Column(String(50))
    last_name = Column(String(50))
    created_on = Column(DateTime, nullable=False, default=datetime.datetime.utcnow())
    updated_on = Column(DateTime, nullable=False, default=datetime.datetime.utcnow(),\
        onupdate=datetime.datetime.utcnow())

    def __init__(self, first_name, last_name):
        self.last_name = last_name
        self.first_name = first_name


    @classmethod
    def create(cls, first_name, last_name):
        result = cls(first_name, last_name)
        session.add(result)
        session.commit()
        return result

    def __repr__(self):
        return "<Person (name=%s last_name=%s)>" % (self.first_name, self.last_name)

Im trying to initialize the db_connnection using this method in another library by importing model.py and init_db_connection

init_db_connection('sqlite:////tmp/foo.db', echo=False)
current_time = datetime.datetime.utcnow()
time.sleep(1)
print('Current_time: {}'.format(current_time))
person = Person.create('Andy','Boss')
print('Person name: {}'.format(person.first_name))

The callback function is able to write to the database from different thread. Im facing problem while reading the database. Im hitting this error

self = <sqlalchemy.orm.session.SessionTransaction object at 0x756550b0>, prepared_ok = False, rollback_ok = False, deactive_ok = False
closed_msg = 'This transaction is closed'

    def _assert_active(self, prepared_ok=False,
                       rollback_ok=False,
                       deactive_ok=False,
                       closed_msg="This transaction is closed"):
        if self._state is COMMITTED:
            raise sa_exc.InvalidRequestError(
                "This session is in 'committed' state; no further "
                "SQL can be emitted within this transaction."
            )
        elif self._state is PREPARED:
            if not prepared_ok:
                raise sa_exc.InvalidRequestError(
>                   "This session is in 'prepared' state; no further "
                    "SQL can be emitted within this transaction."
E                   sqlalchemy.exc.InvalidRequestError: This session is in 'prepared' state; no further SQL can be emitted within this transaction.

sanity/lib/python3.5/site-packages/sqlalchemy/orm/session.py:264: InvalidRequestError

Should I be using scoped_session instead of session to overcome this issue?. I dont whether sqlite is right database for such environment.

user128956
  • 123
  • 1
  • 14
  • 1
    As you've noticed, a `Session` is not thread safe. A scoped session might help, but that depends on your app. – Ilja Everilä Oct 06 '18 at 06:18
  • thanks a lot for information. I modified the code using scoped session. Im opening and closing for each thread that way im not trying to reuse the global session. – user128956 Oct 07 '18 at 02:46

0 Answers0