4

I am trying to add SpaCy as a dependency to my Python Lambda. I am doing this by installing SpaCy as a standalone dependency inside a directory named dependencies using pip3 install spacy --no-deps -t . This is because I can't load the entire Spacy dependency inside the \tmp directory of my Lambda.

I am able to successfully upload the folder to s3 and download it during the Lambda invocation. When I try to import spacy, I get this error: [ERROR] Runtime.ImportModuleError: Unable to import module : No module named 'srsly.ujson.ujson'.

I manually installed srsly inside dependencies\ and I have all the files that are listed as per this link. This was referenced by this link. One of the responses says, "it seems like Python can't load it, because it's not compiled?". How would I compile a dependency which has a .c file in it?

One other question which I found on SO is this question, but I have already manually installed srsly. How to I import the module? Thanks.

I manually check in my code for the presence of ujson before importing spacy like this:

if os.path.exists('/tmp/dependencies/srsly/ujson/ujson.c'):
    print('ujson exists')

and the print statement gets printed.

Nagarajan Shanmuganathan
  • 418
  • 7
  • 29
  • 2
    What version of spacy are you using? And where do you run pip install? I'm facing a similar issue and I think that the problem is that I run pip install on my machine, but lambda runtime uses aws linux and C dependencies that were compliled on my machine do not work there. – Alexey Smirnov Jul 31 '19 at 16:24
  • @AlexeySmirnov I am running pip install inside a folder `/tmp/python` as `pip3 install spacy --no-deps -t .` . Yes, I think the issue might also be with python3.6 vs python3.7. – Nagarajan Shanmuganathan Jul 31 '19 at 17:01
  • 1
    If you run pip install on your local machine, then zips the dependencies, and then uploads for lambda to use it means that some binaries were compiled using you OS compiler which is different from lambda runtime os - amazonlinux – Alexey Smirnov Aug 01 '19 at 08:48

2 Answers2

1

For me pip uninstalling and installing srsly again worked fine.. sometimes its just the compatibility issue with your python version so make sure correct python/srsly versions are present

Bhumika Mahajan
  • 11
  • 1
0

Well it is a bit strange, but my solution for this problem was to create an aditional "ujson" folder in the srsly folder and then move all the ujson generated code to the folder "ujson" previously created

João Ramos
  • 75
  • 1
  • 6