0

I have a lot of imports at the beginning of my python script. Additionally I have some functions which are transformed into compiled C code by using numba. You can find the very small imitation of my code structure below.

import numpy as np
import pandas as pd
from numba import jit,njit

def mysum(a,b):
    return(a+b)

mysum_jit = jit() (mysum)

mysum_jit(5,6)

I want to put those codes on AWS Lambda feature. However, since both the importing and compilation parts are creating huge overheads at first running and I just want to call mysum_jit() function only whenever an event is triggering my script, I am not sure if the AWS Lambda is the right option for me.

Is there a way to solve such an issue?

Thanks in advance.

datatech
  • 147
  • 8
  • Consider [using Lambda Layers](https://stackoverflow.com/questions/53824556/how-to-install-numpy-and-pandas-for-aws-lambdas) for these huge dependencies. Or, better still, try to avoid using such large packages in a low latency microservice. – jarmod Oct 14 '21 at 22:24
  • Do you just want to move the `import` and call to `jit` into a function that's only called when necessary? – Anon Coward Oct 14 '21 at 22:24
  • @AnonCoward Actually, alI want is to call them (imports and compilation) only once as like calling some global codes at the very beginning of a running program on server. By the way, I am not an AWS Lambda user right now. I am just about to be. Now I have to be sure about its features. – datatech Oct 14 '21 at 22:36
  • Lambdas are ephemeral. You can't make any assumption about how long your python host will run. If you need long running tasks, it may not be the answer for you. – Anon Coward Oct 14 '21 at 22:37
  • @jarmod I think I couldn't ask my question in a right way. In our local system, if a library is imported ones, then we don't need to reimport it again. Similarly ones a function is defined, it can be used in anywhere in a session. However, what I see in AWS Lambda, you always call the same imports and define the same functions in every request which creates overhead. (Please correct me if I am wrong. It is just my understanding about AWS Lambda) – datatech Oct 14 '21 at 23:06
  • The imports in a Lambda function will be re-run if the Lambda function cold starts. You can learn more at [part 1](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/) and [part 2](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-2/). You can improve things with more RAM, Provisioned Concurrency, and Lambda Layers. Evaluate those, then decide if you value the guaranteed lower latency (and higher cost) of 24x7 compute vs. higher latency (but lower cost) of Lambda. – jarmod Oct 14 '21 at 23:22

0 Answers0