ERR with YOLOv4 to TensoRT

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello,
I have a problem with pytorch-YOLOv4

I try to transer my weight to engine and it work fine with batchsize = 1
When i try with batch 2 or 4 it is still export file but some error appeard
-I follow this guilde GitHub - Tianxiaomo/pytorch-YOLOv4: PyTorch ,ONNX and TensorRT implementation of YOLOv4

-My command to exe is:
sudo python3 demo_darknet2onnx.py yolov4-custom.cfg yolov4-custom_best_0510.weights img2774.jpg 2

Please help me!

Loading weights from yolov4-custom_best_0510.weights… Done!
Export the onnx model …
/home/xavier/.local/lib/python3.6/site-packages/numpy/core/function_base.py:113: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can’t record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
num = operator.index(num)
/home/xavier/Son/transfer_yolov4/pytorch-YOLOv4/tool/yolo_layer.py:196: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
bx = bxy[:, ii : ii + 1] + torch.tensor(grid_x, device=device, dtype=torch.float32) # grid_x.to(device=device, dtype=torch.float32)
/home/xavier/Son/transfer_yolov4/pytorch-YOLOv4/tool/yolo_layer.py:198: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
by = bxy[:, ii + 1 : ii + 2] + torch.tensor(grid_y, device=device, dtype=torch.float32) # grid_y.to(device=device, dtype=torch.float32)

3

Anyone can help me?
Thank you :(

4

Hi @khuomdt3,
What’s the error? Can you generate the onnx model with these warning?
And, is this related to DeepStream? I think it may be better to report this issue on the github project.

Thanks!

1 Like
5

Very thank you to your reply!

  • “What’s the error?”
    As usually, when i transfer model to TRT engine the success result apeared name of the generated onnx but when i pick batch_size=2 the result are:

Loading weights from yolov4-custom_best_0510.weights… Done!
Export the onnx model …
/home/xavier/.local/lib/python3.6/site-packages/numpy/core/function_base.py:113: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can’t record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
num = operator.index(num)
/home/xavier/Son/transfer_yolov4/pytorch-YOLOv4/tool/yolo_layer.py:196: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
bx = bxy[:, ii : ii + 1] + torch.tensor(grid_x, device=device, dtype=torch.float32) # grid_x.to(device=device, dtype=torch.float32)
/home/xavier/Son/transfer_yolov4/pytorch-YOLOv4/tool/yolo_layer.py:198: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
by = bxy[:, ii + 1 : ii + 2] + torch.tensor(grid_y, device=device, dtype=torch.float32) # grid_y.to(device=device, dtype=torch.float32)

  • " Can you generate the onnx model with these warning?"
    Yes, it’s still generated. 2 onnx are generated!
  • Then when i try to transer new onnx (batch_size = 2) to TRT engine, it’s still success. But when i run it with deepstream-app, the terminal said my model’s batch is 2 and it was trying to rebuild and stop!
    -“And, is this related to DeepStream? I think it may be better to report this issue on the github project.”
    I will post this err/warn to github too.
6

Could you share the detailed log?

Thanks!

1 Like
7
  • “Could you share the detailed log?”
    This one!

0:00:07.315880930 14349 0xeb06040 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1518> [UID = 1]: Backend has maxBatchSize 1 whereas 2 has been requested
0:00:07.315962690 14349 0xeb06040 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1689> [UID = 1]: deserialized backend context :/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/yolov4_2_3_416_416_fp16.engine failed to match config params, trying rebuild
0:00:07.401163990 14349 0xeb06040 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files
Yolo type is not defined from config file name:
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:07.405474992 14349 0xeb06040 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1611> [UID = 1]: build engine file failed
0:00:07.405585648 14349 0xeb06040 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1697> [UID = 1]: build backend context failed
0:00:07.405632432 14349 0xeb06040 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1024> [UID = 1]: generate backend failed, check config file settings
0:00:07.406148432 14349 0xeb06040 WARN nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:07.406194896 14349 0xeb06040 WARN nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/config_infer_primary_yoloV4.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: main:651: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance

  • i’m trying to transfer yolov4 to enigine with these command right now:

python3 demo_darknet2onnx.py yolov4-custom.cfg yolov4-custom_best_0510.weights img2774.jpg 2

Still have warning!

./trtexec --onnx=yolov4_2_3_320_416_static.onnx --batch=2 --maxBatch=2 --saveEngine=yolov4_2_3_320_416_fp16__official.engine --workspace=4096 --fp16

=> Deepstream accept and run batch_size=2 now but can not show up. I’m testing and fixing. Thank you
This topic’s error is no more error but warnning!

  • Note:Last time i used:

./trtexec --onnx=yolov4_2_3_320_416_static.onnx --explicitBatch saveEngine=yolov4_2_3_320_416_fp16__official.engine --workspace=4096 --fp16

8

There is error above,

Have you checked project - GitHub - NVIDIA-AI-IOT/yolo_deepstream: yolo model qat and deploy with deepstream&tensorrt ?

Thanks!

1 Like
9

Thank you!
I fixed the config and re-transfer model with batch=2
I’m working with yolo. i ignore the warnning/err