Overall goal:
I used some Python code from mmpose which can identify animals in a picture, and then deduce their pose. Great. Now, my goal is to be able to bring this to the browser with TensorFlow.js. I understand this question might require many steps.
What I've managed so far:
I used the file top_down_img_demo_with_mmdet.py which came in the demo/ folder of mmpose. Detecting objects works like a charm, the key line being mmdet_results = inference_detector(det_model, image_name) (from mmdet.apis) which returns bounding boxes of what's found. Next, it runs inference_top_down_pose_model (from mmpose.apis) which returns an array of all the coordinates of key points on the animal. Perfect. From there, it draws out to a file. Now, shifting over to TensorFlow.js, I've included their COCO-SSD model, so I can get bounding boxes of animals. Works fine.
What I need help with:
As I understand it, to use the .pth file (big) used in the animal pose identification, it must be ported to another format (.pt, maybe with an intermediate onnx stop) and then loaded as a model in TensorFlow.js where it can run its pose-detection magic in-browser. Two problems: 1) most instructions seem to expect me to know data about the model, which I don't. Kernel size? Stride? Do I need this info? If so, how do I get it? 2) it's honestly not clear what my real end-goal should be. If I end up with a .pt file, is it a simple few lines to load it as a model in TensorFlow.js and run an image through it?
TL;DR: I've got a working Python program that finds animal pose using a big .pth file. How do I achieve the same in-browser (e.g. TensorFlow.js)
What didn't work
This top answer does not run, since "model" is not defined. Adding model = torch.load('./hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth') still failed with AttributeError: 'dict' object has no attribute 'training'
This GitHub project spits out a tiny saved_model.pb file, less then 0.1% the size of the .pth file, so that can't be right.
This answer gave a huge wall of text, array values off my screen, which it said were weights anyway, not a new model file.
This article expects me to know the structure of the model.
Thank you all. Honestly, even comments about apparent misunderstandings I have about the process would be very valuable to me. Cheers.
- Chris
