57

I am writing a lambda function for file extraction and need to store a file while performing this function so need to store that file in aws lambda function . Is it possible to store store a file on lambda .

Rahul Panwar
  • 635
  • 1
  • 5
  • 8
  • As long as you need this file only in the same invocation, there isn't a problem to do that. – MaiKaY Jun 20 '18 at 11:09
  • 1
    Not only in the same invocation, the file will be there for subsequent invocations of the same function. However it will be scoped locally just to this instance. – mewa Jun 20 '18 at 11:15
  • 1
    Just store it in `/tmp`. The container _might_ be reused, so delete the file if you don't need it again, otherwise subsequent executions of the function might fill up the available disk space (500MB). – John Rotenstein Jun 20 '18 at 12:40
  • 2
    Since the file is anyway not persistent there isn’t usually a good reason to use one, just use the regular memory (RAM). In most languages you can also use a virtual file that won’t be saved to disk. – Ronyis Jun 20 '18 at 18:38
  • 2
    I believe it's an old post however it would be useful to know if there is an automated mechanism supported in AWS Lambda to free up `/tmp` after every Lambda execution? Or in case it has to be done manually, how do we do it or how to track the space, thanks @JohnRotenstein – omdurg Sep 10 '20 at 18:19
  • @omdurg The Lambda function needs to free up the `/tmp/` directory itself. This is intentional because there are some good reasons why functions might want to cache data there for future executions. Basically, if the Lambda function creates a file that it doesn't want to keep, it is responsible for deleting it before execution ends. – John Rotenstein Sep 10 '20 at 21:49

1 Answers1

74

Yes, quoting from the AWS Lambda FAQ

Each Lambda function receives 500MB of non-persistent disk space in its own /tmp directory.

https://aws.amazon.com/lambda/faqs/

Note that as the commenters point out, it is only reliably there for a single execution of your function. You don't have control over the execution environment, so subsequent runs may be in different instances of your lambda function.

If you need the data to be more persistent you should use AWS S3.

poida
  • 3,403
  • 26
  • 26
  • 2
    I understood that if Lambda function automatically creates/stores file in /tmp/, then it would automatically free up /tmp/, although one more point might help better, in cases when we manually mention /tmp/ like `iterator.to_csv(f'/tmp/{i}.csv')`, will it automatically clean /tmp/ after every execution or do we have to release the /tmp/ using some method, thanks@poida – omdurg Sep 11 '20 at 17:55
  • 4
    @omdurg Not sure what you mean by "automatically creates/stores file in /tmp/" but, in general, the Lambda service doesn't clean up /tmp between Lambda function executions. Any time your Lambda function is warm-started, it inherits /tmp from the previous execution, along with whatever was written there by the previous execution. Lambda will eventually decide to recycle a runtime environment, at which point /tmp is wiped/discarded. If you don't want /tmp diskspace to persist across invocations, delete whatever you wrote there before ending your Lambda function. – jarmod Feb 01 '21 at 19:09
  • note this can now be increased to 10gb https://aws.amazon.com/blogs/aws/aws-lambda-now-supports-up-to-10-gb-ephemeral-storage/ – Andy Jun 08 '23 at 09:03