How to use tf-hub models locally

Question

I,ve been trying to use a BERT model from tf-hub https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2.

import tensorflow_hub as hub
bert_layer = hub.keras_layer('./bert_en_uncased_L-12_H-768_A-12_2', trainable=True)

But problem is that it is downloading data after every run.

So i downloaded the .tar file from tf-hub https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2

Now i'm trying to use this downloaded tar file(after untar)

I've followed this tutorial https://medium.com/@xianbao.qian/how-to-run-tf-hub-locally-without-internet-connection-4506b850a915

But it didn't work out well and there is no further infromation or script is provided in this blog post

if someone can provide complete script to use the dowloaded model locally(without internet) or can improve the above blog post(Medium).

I've also tried

untarredFilePath = './bert_en_uncased_L-12_H-768_A-12_2'
bert_lyr = hub.load(untarredFilePath)
print(bert_lyr)

Output

<tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject object at 0x7f05c46e6a10>

Doesn't seems to work.

or is there any other method to do so..??

This article helped me in setting it up- https://medium.com/@xianbao.qian/how-to-run-tf-hub-locally-without-internet-connection-4506b850a915 . Hopefully this will help you also — Shashank Shekhar Shukla, Jul 01 '20 at 11:25
I've already mentioned about this article in problem. Can you provide further steps or your complete python script so that I can make more sense of it including your folder structure e.g. where is your script and where is untarred model file. Thank You — devspartan, Jul 01 '20 at 13:12
I just posted an answer with the script that i was using to cache the models locally. Hope this helps. — Shashank Shekhar Shukla, Jul 01 '20 at 20:37

Frederik Bode · Answer 1 · 2020-07-01T13:41:41.280

1

Hmm I cannot reproduce your problem. What worked for me:

script.sh

# download the model file using the 'wget' program
wget "https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2?tf-hub-format=compressed"

# rename the downloaded file name to 'tar_file.tar.gz'
mv 2\?tf-hub-format\=compressed tar_file.tar.gz

# extract tar_file.tar.gz to the local directory 
tar -zxvf tar_file.tar.gz

# turn off internet

# run a test script
python3 test.py

# running the last command prints some tensorflow warnings, and then '<tensorflow_hub.keras_layer.KerasLayer object at 0x7fd702a7d8d0>'

test.py

import tensorflow_hub as hub
print(hub.KerasLayer('.'))

edited Jul 01 '20 at 13:41

answered Jul 01 '20 at 11:23

Frederik Bode

2,632
1
10
17

is that a python script or what? – devspartan Jul 01 '20 at 13:02
1

it's a `bash` script, so you run it using `bash script.sh` – Frederik Bode Jul 01 '20 at 13:08
Can you provide complete script of the above code snippet. Or any script so that i can make sense of it – devspartan Jul 01 '20 at 13:13
1

This is a complete script, I will add some comments to explain what happens. – Frederik Bode Jul 01 '20 at 13:37

score 1 · Answer 2 · answered Jul 01 '20 at 20:36

1

I wrote this script using this medium article(https://medium.com/@xianbao.qian/how-to-run-tf-hub-locally-without-internet-connection-4506b850a915) as reference. I am creating a cache directory within my project and the tensorflow model is cached locally in this cached directory and I am able to load the model locally. Hope this helps you.

import os
os.environ["TFHUB_CACHE_DIR"] = r'C:\Users\USERX\PycharmProjects\PROJECTX\tf_hub'

import tensorflow as tf
import tensorflow_hub as hub
import hashlib

handle = "https://tfhub.dev/google/universal-sentence-encoder/4"
hashlib.sha1(handle.encode("utf8")).hexdigest()


embed = hub.load("https://tfhub.dev/google/universal-sentence-encoder/4")
def get_sentence_embeddings(paragraph_array):
    embeddings=embed(paragraph_array)
    return embeddings

answered Jul 01 '20 at 20:36

Shashank Shekhar Shukla

690
9
12

Your answer did help me to make sense of that article and model almost did load. while loading i got this error : ```Unsuccessful TensorSliceReader constructor: Failed to find any matching files for C:\Users\devendra\PycharmProjects\NeuralMachineTranslation\tf_hub/276b9d19b6b0a264a5526d08219dfe30923ac454/variables/variables [[node save/RestoreV2 (defined at /globalEnvironment/venv/lib/python3.7/site-packages/tensorflow_hub/module_v2.py:102) ]] [[save/RestoreV2/_398]] ``` – devspartan Jul 03 '20 at 04:08
I've tried with other models also but i got the same error seems this, it is missing any variable file in variable folder or anything else. Have you any idea how to cope with it? – devspartan Jul 03 '20 at 04:14
It seems like a common problem when loading and restoring model check points in tensorflow. I tried searching a workaround for this problem maybe this https://github.com/tensorflow/tensorflow/issues/6082#issuecomment-289650535 will help you. – Shashank Shekhar Shukla Jul 03 '20 at 20:17

score 1 · Answer 3 · answered Jul 06 '20 at 15:02

After getting information from tf-hub team they provide this solution. Let's say you have downloaded the .tar.gz file from official tf-hub model page from download button. You have extracted it. You got a folder which contain assets, variable and model. You put it in your working directory.

In script just add path to that folder:

import tensroflow-hub as hub

model_path ='./bert_en_uncased_L-12_H-768_A-12_2' # in my case
# one thing the path you have to provide is for folder which contain assets, variable and model
# not of the model.pb itself

lyr = hub.KerasLayer(model_path, trainable=True)

Hope it should work for you as well. Give it a try

score 1 · Answer 4 · answered Jul 08 '20 at 11:43

1

The tensorflow_hub library caches downloaded and uncompressed models on disk to avoid repeated uploads. The documentation at tensorflow.org/hub/caching has been expanded to discuss this and other cases.

answered Jul 08 '20 at 11:43

arnoegw

1,218
6
13

How to use tf-hub models locally

4 Answers4