How to download models and weights from tensorflow.js

Question

I'm trying to download a pretrained tensorflow.js model including weights, to be used offline in python in the standard version of tensorflow as part of a project that is not on an early stage by any means, so switching to tensorflow.js is not a possibility. But I cant just figure out how to download those models and if its necessary to to do some conversion to the model.

I'm aware that in javascript I can access the models and use them by calling them like this but how do I actually get the .ckpt files or the model frozen if thats the case?

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.3"></script>

<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/posenet@0.2.3"></script>

My final objective is to get the frozen model files, and get the outputs like is done in the normal version of tensorflow. Also this will be used in an offline environment, so any online reference would not be useful.

Thanks for your replies

I wonder why you want to create your model in js and export it in python. Why prevent you from creating the model directly in python ? — edkeveked, Jan 06 '19 at 23:07
no I dont wanna create a new one I wanna download a trained one, sorry I wasnt clear about it — Boris, Jan 07 '19 at 01:45
@edkeveked — isn't it perfectly clear that the OP wants to use the fully-trained model from a Python environment and hence is asking how to download the model weights directly? — Jivan, Feb 19 '20 at 23:39
@Boris have you found a solution? I'm looking for the exact same thing here. — Jivan, Feb 19 '20 at 23:39

score 3 · Answer 1 · edited Sep 02 '19 at 10:09

3

It is possible to save the model topology and its weights by calling the method save of the model.

const model = tf.sequential();
model.add(tf.layers.dense(
     {units: 1, inputShape: [10], activation: 'sigmoid'}));
const saveResult = await model.save('downloads://mymodel');
// This will trigger downloading of two files:
//   'mymodel.json' and 'mymodel.weights.bin'.
console.log(saveResult);

There are different scheme strings depending on where to save the model and its weights (localStorage, IndexDB, ...). doc

edited Sep 02 '19 at 10:09

Francisco Presencia

8,732
6
46
90

answered Jan 06 '19 at 23:09

edkeveked

17,989
10
55
93

`const model = posenet; const saveResult = model.save('downloads://mymodel'); console.log(saveResult);` Tried with something like that but doesnt work, how to assign posenet, for example, to the variable model? – Boris Jan 07 '19 at 02:08
1

If you're looking for a way to get the scripts offline, just copy the scripts here https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.3 and paste it in a local js file. And change the following: ` – edkeveked Jan 08 '19 at 18:55
1

thanks, but I'm not looking to get the scripts offline, I'm trying to get the pretrained models including weights, that are hosted as demos for tensorflow.js, offline – Boris Jan 09 '19 at 19:28
3

I wonder if you're looking for the url of the weights file to download. Here is the manifest.json of one posenet architecture: https://storage.googleapis.com/tfjs-models/weights/posenet/mobilenet_v1_100/manifest.json. Here is a way to load the files in case you want to setup your own server: https://github.com/oveddan/posenet-for-installations/. I can't tell however if the license of the project allows to host those files locally – edkeveked Jan 10 '19 at 23:07
yeah, that helps but still no idea how to download the weights of that model – Boris Jan 12 '19 at 02:26
wait, I didnt checked the second link, thanks weights downloaded! now the question remaining is how to convert those into standard tensorflow weights – Boris Jan 13 '19 at 00:36
Here you have an answer to how to load a model and the weights: https://stackoverflow.com/questions/53639919/load-tensorflow-js-model-from-local-file-system-in-javascript/53712229#53712229 – edkeveked Jan 13 '19 at 21:20
@Boris have you found a way to load these weights from a Python TensorFlow installation? – Jivan Feb 19 '20 at 23:40
@edkeveked the answer is showing how to load the models from JavaScript. The OP wants to load them from Python. – Jivan Feb 19 '20 at 23:45
@Jivan you can ask your own question with more details – edkeveked Feb 20 '20 at 10:34
just wanted to download the weights from one of the tensorflow.js examples and use it on python tensorflow, but never figured out how to do it, never got an answer on the second part – Boris Feb 21 '20 at 00:45

Ben Butterworth · Answer 2 · 2020-08-27T15:34:13.593

I went to https://storage.googleapis.com/tfjs-models/ and found the directory listing all the files. I found the relevant files (I wanted all the mobilenet float, as opposed to quantized mobileNet), and populated this file_uris list.

base_uri = "https://storage.googleapis.com/tfjs-models/"
file_uris = [
    "savedmodel/posenet/mobilenet/float/050/group1-shard1of1.bin",
    "savedmodel/posenet/mobilenet/float/050/model-stride16.json",
    "savedmodel/posenet/mobilenet/float/050/model-stride8.json",
    "savedmodel/posenet/mobilenet/float/075/group1-shard1of2.bin",
    "savedmodel/posenet/mobilenet/float/075/group1-shard2of2.bin",
    "savedmodel/posenet/mobilenet/float/075/model-stride16.json",
    "savedmodel/posenet/mobilenet/float/075/model-stride8.json",
    "savedmodel/posenet/mobilenet/float/100/group1-shard1of4.bin",
    "savedmodel/posenet/mobilenet/float/100/group1-shard2of4.bin",
    "savedmodel/posenet/mobilenet/float/100/group1-shard3of4.bin",
    "savedmodel/posenet/mobilenet/float/100/model-stride16.json",
    "savedmodel/posenet/mobilenet/float/100/model-stride8.json"
]

Then I used python to iteratively download the files into their same folders.

from urllib.request import urlretrieve
import requests
from pathlib import Path

for file_uri in file_uris:
    uri = base_uri + file_uri
    save_path = "/".join(file_uri.split("/")[:-1])
    Path(save_path).mkdir(parents=True, exist_ok=True)
    urlretrieve(uri, file_uri)
    print(path, file_uri)

I enjoyed Jupyter Lab (Jupyter Notebook is also good) when experimenting with this code.

With this, you'll get a folder with bin files (the weight) and the json files (the graph model). Unfortunately, these are graph models, so they cannot be converted into SavedModels, and so they are absolutely useless to you. Let me know if someone find a way of running these tfjs graph model files in regular TensorFlow (preferably 2.0+).

You can also download zip files with the 'entire' model from TFHub, for example, a 2 byte quantized ResNet PoseNet is available here.

How to download models and weights from tensorflow.js

2 Answers2

Linked