Load onnx model in opencv dnn

Question

I get the following error while loading onnx model (downloaded from here) in OpenCV-C++ (4.6.0-dev) using the given code. What am I doing wrong? Please help me to solve this problem.

[ERROR:0@0.003] global /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2564) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 243 [ 0 0 0 51 ] [ERROR:0@0.003] global /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp (1042) handleNode DNN/ONNX: ERROR during processing node with 1 inputs and 1 outputs: [Shape]:(onnx_node!Shape_70) from domain='ai.onnx' terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.6.0-dev) /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1064: error: (-2:Unspecified error) in function 'handleNode'

Node [Shape@ai.onnx]:(onnx_node!Shape_70) parse error: OpenCV(4.6.0-dev) /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp:2565: error: (-215:Assertion failed) !isDynamicShape in function 'parseShape'

Aborted (core dumped)

#include <iostream>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/dnn/dnn.hpp>

using namespace std;
using namespace cv;
using namespace dnn;

int main()
{
    // load the neural network model
    cv::dnn::Net net = cv::dnn::readNetFromONNX("yunet.onnx");
}  

I can successfully load the model in Python using both of the following two methods: Method 1:

import onnx
onnx_model = onnx.load('yunet.onnx')
onnx.checker.check_model(onnx_model)

Method 2:

#net = cv2.dnn.readNet('yunet.onnx')
net = cv2.dnn.readNetFromONNX('yunet.onnx')

Am I missing something during building the OpenCV from source?

Answer 1

It seems opencv does not support onnx models that have dynamic input shapes, check this link. Try to build the latest version of opencv. Also, check this link . It has been mentioned to use a fixed input shape for Yunet. If previous suggestions did not work, use the following method.

If you're using tf2 and if your weights are in .h5 form you will be able to dispose of onnx troubles . you can generate .pb from your .h5 and then easily use in your c++ program.to aim generating .pb use the following code:

after that by using opencv you will be able to import your model and then enjoy!

import tensorflow as tf
from tensorflow import keras
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
from tensorflow.keras.models import load_model
import numpy as np                    
#path of the directory where you want to save your model
frozen_out_path = '/path.../freez/'             # name of the .pb file
frozen_graph_filename = "freez_graph_6cls_try1" #“frozen_graph”

model = load_model("/cls_vgg16_6_cl.h5")
# model = ""# Your model# Convert Keras model to ConcreteFunction
full_model = tf.function(lambda x: model(x))
full_model = full_model.get_concrete_function(
tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype))# Get frozen ConcreteFunction
frozen_func = convert_variables_to_constants_v2(full_model)
frozen_func.graph.as_graph_def()
layers = [op.name for op in frozen_func.graph.get_operations()]
print("-" * 60)
print("Frozen model layers: ")
for layer in layers:
   print(layer)
   print("-" * 60)
print("Frozen model inputs: ")
print(frozen_func.inputs)
print("Frozen model outputs: ")
print(frozen_func.outputs)# Save frozen graph to disk
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
                  logdir=frozen_out_path,
                  name=f"{frozen_graph_filename}.pb",
                  as_text=False)# Save its text representation
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
                logdir=frozen_out_path,
                name=f"{frozen_graph_filename}.pbtxt",
                as_text=True)

Answer 2

The model you are using has dynamic input shape. OpenCV DNN does not support ONNX models with dynamic input shape [Ref]. However, you can load an ONNX model with fixed input shape and infer with other input shapes using OpenCV DNN.

You can download face_detection_yunet_2022mar.onnx, which is the fixed input shape version of the model you are using.

You can successfully load and use the model with the following code in OpenCV 4.6.0-dev.

cv::String fd_modelPath = "face_detection_yunet_2022mar.onnx";
float scoreThreshold = 0.9;
float nmsThreshold = 0.3;
int topK = 5000;

cv::Ptr<cv::FaceDetectorYN> detector = cv::FaceDetectorYN::create(fd_modelPath, "", cv::Size(320, 320), scoreThreshold, nmsThreshold, topK);

HOW TO CONVERT DYNAMIC INPUT SHAPE TO FIXED ? Please read this page.

HOW TO READ SAVEDMODEL FORMAT IN OPENCV DNN?

# In python 3.6 to 3.10

# load frozengraph.pb
net = cv2.dnn.readNet(path2frozengraph)
# You can easily convert your model to frozengraph using
# https://github.com/fisakhan/convert_model_x2y

# preprocess input if needed
imgn = (img-127.5)/128.0# preprocess input if needed
imgn = imgn.astype(np.float32)# convert to float32 if needed

# opencv dnn way of pre-processing (must be NCHW in cv2 dnn) 
input_blob = cv2.dnn.blobFromImage(imgn, 1, (100,100), 0, swapRB=False, crop=False)

# inference
net.setInput(input_blob)
feature_vector_dnn = net.forward()