0

I have a basic word2vec embedding code I've written, referring to this. Now the 'final_embeddings' are what are relevant to me in this question. I want to run two different types of classification which use this. So instead of saving it as a .ipynb file in Jupyter, I saved it as a script called 'LanguageModel.py' so that I can just call in the final embeddings and use them as a parameter for classification. However, using 'import LanguageModel' means that it just runs that whole code again. I wanted it as a separate file because I dont want to run that bit again and again for each run. How exactly should I frame all my code so that the variable 'final_embeddings' is saved after just one run, and I just call it again and again for whatever purpose when I need it?

Arjun Mohan
  • 63
  • 3
  • 10
  • Don't use a module to persist data. You can use a module to handle the generation and saving of your data structure, but don't use that module to generate it again and again. Save the data in a structure that lets Python load it easily again; `pickle` is probably your best bet. See the duplicate. – Martijn Pieters Oct 12 '17 at 07:31
  • I'm sorry, I dont follow this properly. So it just saves the state as is in a '.pkl' file? I didn't understand the bit about the reloading. How does it reload it? As if I'd already run the code in the notebook previously and I can just use the variables normally as if it were a part of my code? Please bear with the potentially stupid question – Arjun Mohan Oct 12 '17 at 07:39
  • If you still have the notebook open then the data is part of the notebook. If you want to load the data into a new notebook, just use `pickle.load()` to load the data from your pickle file. – Martijn Pieters Oct 12 '17 at 07:40
  • Okay, I understand now. A last question now. Upon reading the pickle documentation, it has nothing about numpy arrays, which is the data type of what I want to load. Does this mean I cannot use `pickle.load()` for loading numpy arrays? – Arjun Mohan Oct 12 '17 at 07:46
  • Numpy arrays can be pickled: [How does Python 3 know how to pickle extension types, especially Numpy arrays?](//stackoverflow.com/q/45391834) – Martijn Pieters Oct 12 '17 at 07:50
  • thanks for your time, that was helpful – Arjun Mohan Oct 12 '17 at 07:54

0 Answers0