13

Amazon suggests to not include big libraries/dependencies in lambda functions. As far as I know, SQLAlchemy is quite a big python library. Do you think it is a good idea to use it in lambda functions? An option would be to include it as a Lambda Layer and use it across all related Lambda functions.

Anyways, what is the best practise?

funtik
  • 1,688
  • 3
  • 11
  • 27
  • See https://aws.amazon.com/blogs/database/use-python-sqlalchemy-orm-to-interact-with-an-amazon-aurora-database-from-a-serverless-application/ – Kai Nacke Nov 14 '21 at 01:19

3 Answers3

1

Serverless functions are meant to be small self-contained functions. SQLAlchemy is an ORM, which allows you to manipulate database objects like objects in python. If you're just writing a few serverless functions that do you're average CRUD operations on a database you're better off writing the SQL by composing the strings and directly executing that through your database driver (which you'll have to install anyways, even if you're using sqlalchemy). If you're building your own framework on top of AWS Lambda then perhaps consider sqlalchemy.

Steven Black
  • 1,988
  • 1
  • 15
  • 25
  • 9
    Please be aware that "writing the SQL by composing the strings" is dangerous and may lead to SQL injection issues. – blahblah Aug 26 '20 at 11:48
0

From what I read, SQLAlchemy performs in memory caching of data it has read and uses that for future calls. Based on what you are doing, it would be good to check out the SQLAlchemy caching strategy so another Lambda does not change the data from under the first lambda with SQLAlchemy.

HandyManDan
  • 148
  • 1
  • 6
-4
  • Do you think it is a good idea to use SQLAlchemy in lambda functions?

Yes but you might want to consider AWS Dynamodb or other options first. If SQL is your requirement then you can go for an RDS instance with an SQL compatible backend. It all depends on your specific needs in this case. It is not a general requirement.

  • Anyways, what is the best practice?

If you just need to store some data and connect the data storage to AWS Lambda then DynamoDB is usually a good choice. SQL can be preferred if you already have a relational database or if you need to perform certain SQL queries which are easier to do with SQL than a non-relational model.

Niklas Rosencrantz
  • 25,640
  • 75
  • 229
  • 424
  • Thanks for the answer! Although your answer gives a great comparison for AWS DynamoDB and SQL RDS, it doesn't quite answer my question. It is more about using SQLAlchemy as a Python library in Lambda functions, when SQL database is already chosen. Can I count your answer as "It is ok to use this library"? – funtik Oct 12 '19 at 14:08
  • 2
    @fd.huseynov Yes but please consider if you really need SQLAlchemy or if it would make things more difficult. If you chose MySQL then you can solve everything with pymysql usually (https://pypi.org/project/PyMySQL/) and adding SQLAlchemy would add complexity. However, if you have a class library in Python and you wish to use that with an object-relational-model then SQLAlchemy might be a good choice. It will depend on your situation and your project requirements. I had a quite large project based on an SQL RDS connected to AWS Lambda and we did not use SQLAlchemy. It is certainly not a must. – Niklas Rosencrantz Oct 12 '19 at 14:32
  • 4
    This doesn't answer the question, he's asking about whether including an ORM in a lambda NOT asking about what database to use. – Steven Black Jul 14 '20 at 00:18