stored procedures with sqlAlchemy

Question

How can I call stored procedures of sql server with sqlAlchemy?

Answer 1

Engines and Connections have an execute() method you can use for arbitrary sql statements, and so do Sessions. For example:

results = sess.execute('myproc ?, ?', [param1, param2])

You can use outparam() to create output parameters if you need to (or for bind parameters use bindparam() with the isoutparam=True option)

Answer 2

context: I use flask-sqlalchemy with MySQL and without ORM-mapping. Usually, I use:

# in the init method
_db = SqlAlchemy(app)

#... somewhere in my code ...
_db.session.execute(query)

---

Calling stored procedures is not supported out of the box: the callproc is not generic, but specific to the mysql connector.

For stored procedures without out params, it is possible to execute a query like

_db.session.execute(sqlalchemy.text("CALL my_proc(:param)"), param='something')

as usual. Things get more complicated when you have out params...

---

One way to use out params is to access the underlying connector is through engine.raw_connection(). For example:

conn = _db.engine.raw_connection()
# do the call. The actual parameter does not matter, could be ['lala'] as well
results = conn.cursor().callproc('my_proc_with_one_out_param', [0])
conn.close()   # commit
print(results) # will print (<out param result>)

This is nice since we are able to access the out parameter, BUT this connection is not managed by the flask session. This means that it won't be committed/aborted as with the other managed queries... (problematic only if your procedure has side-effect).

Finally, I ended up doing this:

# do the call and store the result in a local mysql variabl
# the name does not matter, as long as it is prefixed by @
_db.session.execute('CALL my_proc_with_one_out_param(@out)')
# do another query to get back the result
result = _db.session.execute('SELECT @out').fetchone()

The result will be a tuple with one value: the out param. This is not ideal, but the least dangerous: if another query fails during the session, the procedure call will be aborted (rollback) as well.

Answer 3

The easiest way to call a stored procedure in MySQL using SQLAlchemy is by using callproc method of Engine.raw_connection(). call_proc will require the procedure name and parameters required for the stored procedure being called.

def call_procedure(function_name, params):
       connection = cloudsql.Engine.raw_connection()
       try:
           cursor = connection.cursor()
           cursor.callproc(function_name, params)
           results = list(cursor.fetchall())
           cursor.close()
           connection.commit()
           return results
       finally:
           connection.close()

Answer 4

Just execute procedure object created with func:

from sqlalchemy import create_engine, func
from sqlalchemy.orm import sessionmaker

engine = create_engine('sqlite://', echo=True)
print engine.execute(func.upper('abc')).scalar() # Using engine
session = sessionmaker(bind=engine)()
print session.execute(func.upper('abc')).scalar() # Using session

Answer 5

Supposing you already have session created with sessionmaker(), you can use following function:

def exec_procedure(session, proc_name, params):
    sql_params = ",".join(["@{0}={1}".format(name, value) for name, value in params.items()])
    sql_string = """
        DECLARE @return_value int;
        EXEC    @return_value = [dbo].[{proc_name}] {params};
        SELECT 'Return Value' = @return_value;
    """.format(proc_name=proc_name, params=sql_params)

    return session.execute(sql_string).fetchall()

Now you can execute your stored procedure 'MyProc' with parameters simply like that:

params = {
    'Foo': foo_value,
    'Bar': bar_value
}
exec_procedure(session, 'MyProc', params)

Answer 6

Out of desperate need for a project of mine, I wrote a function that handles Stored Procedure calls.

Here you go:

import sqlalchemy as sql

def execute_db_store_procedure(database, types, sql_store_procedure, *sp_args):
    """ Execute the store procedure and return the response table.

    Attention: No injection checking!!!

    Does work with the CALL syntax as of yet (TODO: other databases).

    Attributes:
        database            -- the database
        types               -- tuple of strings of SQLAlchemy type names.
                               Each type describes the type of the argument
                               with the same number.
                               List: http://docs.sqlalchemy.org/en/rel_0_7/core/types.html
        sql_store_procudure -- string of the stored procedure to be executed
        sp_args             -- arguments passed to the stored procedure
    """
    if not len(types) == len(sp_args):
        raise ValueError("types tuple must be the length of the sp args.")

    # Construch the type list for the given types
    # See
    # http://docs.sqlalchemy.org/en/latest/core/sqlelement.html?highlight=expression.text#sqlalchemy.sql.expression.text
    # sp_args (and their types) are numbered from 0 to len(sp_args)-1
    type_list = [sql.sql.expression.bindparam(
                    str(no), type_=getattr(sql.types, typ)())
                        for no, typ in zip(range(len(types)), types)]

    try:
        # Adapts to the number of arguments given to the function
        sp_call = sql.text("CALL `%s`(%s)" % (
                sql_store_procedure,
                ", ".join([":%s" % n for n in range(len(sp_args))])),
            bindparams=type_list
        )
        #raise ValueError("%s\n%s" % (sp_call, type_list))
        with database.engine.begin() as connection:
            return connection.execute(
                sp_call,
                # Don't do this at home, kids...
                **dict((str(no), arg)
                    for (no, arg) in zip(range(len(sp_args)), sp_args)))
    except sql.exc.DatabaseError:
        raise

It works with the CALL syntax, so MySQL should work as expected. MSSQL uses EXEC instead of call and a little differennt syntax, I guess. So making it server agnostic is up to you but shouldn’t be too hard.

Answer 7

Another workaround:

query = f'call Procedure ("{@param1}", "{@param2}", "{@param3}")'    
sqlEngine = sqlalchemy.create_engine(jdbc)
conn = sqlEngine.connect() 
df = pd.read_sql(query,conn,index_col=None)

Answer 8

I had a stored procedure for postgresql with following signature -

    CREATE OR REPLACE PROCEDURE inc_run_count(
        _host text,
        _org text,
        _repo text,
        _rule_ids text[]
    )

After quite a few error and trial, I found this is how to call the procedure from python3.

    def update_db_rule_count(rule_ids: List[str], host: str, org: str, repo: str):
        param_dict = {"host": host, "org": org, "repo": repo, "rule_ids": f'{{ {",".join(rule_ids)} }}'}


        with sessionmaker(autocommit=False, autoflush=False, bind=ro_engine) as analytics_db:
            analytics_db.execute('call inc_run_count(:host, :org, :repo, :rule_ids)', param_dict)
            analytics_db.commit()