In Snowflake: How to access internally staged pre-trained model from UDF, syntax dilemma?

Question

What is the syntax to reference a staged zip file from UDF? Specifically, I created UDF in Snowpark and it needs to load s-bert sentence_transformers pre-trained model (I downloaded the model, zipped it, and uploaded it to internal stage).

The "SentenceTransformer" method (see the code line below) takes a parameter that can either be the name of the model -- in which case, the pre-trained model will be downloaded form the web; or it can take a directory path to a folder that contains already downloaded pre-trained model files.

Downloading the files from the Web with UDF is not an option in Snowflake.
So, what is the directory path to the internally staged file that I can use as a parameter to the SentenceTransformer method so it can access already downloaded zip model? "@stagename/filename.zip" is not working.

@udf(....)
def create_embedding()..:
     ....
     model = SentenceTransformer('all-MiniLM-L6-v2')   #THIS IS THE LINE IN THE QUESTION
     ....
     ....

score 1 · Answer 1 · answered Aug 25 '22 at 03:02

UDFs need to specify specific files when creating them (for now):

https://docs.snowflake.com/en/developer-guide/udf/python/udf-python-creating.html#loading-a-file-from-a-stage-into-a-python-udf

Check the example from the docs, which uses imports, snowflake_import_directory to open(import_dir + 'file.txt'):

create or replace function my_udf()
  returns string
  language python
  runtime_version=3.8
  imports=('@my_stage/file.txt')
  handler='compute'
as
$$
import sys
IMPORT_DIRECTORY_NAME = "snowflake_import_directory"
import_dir = sys._xoptions[IMPORT_DIRECTORY_NAME]

def compute():
  with open(import_dir + 'file.txt', 'r') as file:
    return file.read()
$$;

In Snowflake: How to access internally staged pre-trained model from UDF, syntax dilemma?

1 Answers1