RTSP stream not work "Stream format not found, dropping the frame"

longvuvan083 · December 1, 2023, 12:50pm

Hardware Jetson AGX Jetpack JP502.
GStreamer 1.16.3

Hello, i have an camera that stream the video via udp port 5600.

"#include <gst/gst.h>
#include <gst/rtsp-server/rtsp-server.h>
#include <iostream>
#include <string>
int main(int argc, char* argv[]) {
    gst_init(&argc, &argv);

    GMainLoop* main_loop = g_main_loop_new(NULL, false);

    GstRTSPServer* server = gst_rtsp_server_new();
    g_object_set(server, "service", "8553", NULL);
    
    std::string launch_string = "udpsrc port=5600 caps=application/x-rtp,encoding-name=(string)H264 ! rtph264depay ! rtph264pay name=pay0";
    std::cout << "start streaming to port 8553/live !!! \n";
    GstRTSPMediaFactory* factory = gst_rtsp_media_factory_new();
    gst_rtsp_media_factory_set_launch(factory, launch_string.c_str());
    gst_rtsp_media_factory_set_shared(factory, true);
    
    GstRTSPMountPoints* mount_points = gst_rtsp_server_get_mount_points(server);
    gst_rtsp_mount_points_add_factory(mount_points, "/live", factory);
    g_object_unref(mount_points);

    gst_rtsp_server_attach(server, NULL);

    g_main_loop_run(main_loop);
}

"
the code is working fine as i can capture the video stream in VLC with the address rtsp://127.0.0.1.:8553/live
however, i have another c++ app running on same system, that i expect to read the input rtsp address.
the gstream pipline as follow :

pipe_string = "rtspsrc location=\"";
pipe_string += address;
 pipe_string += "\" latency=0 ! rtph264depay ! h264parse ! queue ! nvv4l2decoder ! queue ! nvvideoconvert ! video/x-raw,format=BGRx";

however i got the bunch of error “Stream format not found, dropping the frame”.
my pipeline is working fine with other video source from rtsp camera. but for this rtsp stream, it’s not work. idk why.

Can you guys give me some advices?

longvuvan083 · December 1, 2023, 12:57pm

I used this test pipeline
export GST_DEBUG=3
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink

but i got the warning

0:00:11.317535899 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff60060480 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame
0:00:11.416085454 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff60060360 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame
0:00:11.514314164 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff5003eea0 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame

longvuvan083 · December 1, 2023, 1:13pm

Okay i found the solution for this
because i forgot to run the sudo jetson_clocks, then the stream working fine

longvuvan083 · December 4, 2023, 3:16pm

hmm,no i still got that problem after that. the stream is only work one time and not work anymore. the frame dropped error still there

fanzh · December 5, 2023, 1:52am

can VLC play the RTSP source?
could you share the result “gst-discoverer-1.0 -v rtsp://your rtsp address”? if no gst-discoverer-1.0, you can use “ffprobe rtsp://your rtsp address” instead.

longvuvan083 · December 5, 2023, 5:35am

Hi @fanzh here is the result of gst-discoverer command

Analyzing rtsp://192.168.1.4:8559/live
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
reference in DPB was never decoded
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
Done discovering rtsp://192.168.1.4:8559/live

Topology:
  unknown: application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile-level-id=(string)64002a, sprop-parameter-sets=(string)"Z2QAKqyyAPAET8uAtQEBAUAAAAMAQAAAHiPGDJI\=\,aOvMsiw\=", a-tool=(string)GStreamer, a-type=(string)broadcast, a-ts-refclk=(string)local, a-mediaclk=(string)sender, ssrc=(uint)654825449, clock-base=(uint)2585653043, seqnum-base=(uint)7889, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Tags:
        video codec: H.264 (High Profile)
      
      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Additional info:
        None
      Stream ID: b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:96
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 60/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags: 
      video codec: H.264 (High Profile)

longvuvan083 · December 5, 2023, 5:44am

vlc is working fine as i said on above. but it’s not working in my app.

My system are setup like this

(Simulation UDP Stream used gstreamer) via port 5600 —> (My RTSP c++ server 8554/live) ----> (c++ application)

fanzh · December 5, 2023, 6:00am

Thanks for the update!

could you share more logs? please do the following setting first,
export GST_DEBUG=6
then execute
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink >1.log 2>1.og,
then use “ctrl+c” to end after 10 seconds, then share 1.log. you can use zip to compress the log.
could you use this command-line to dump some stream data?

gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse !   mux. mpegtsmux name=mux ! filesink location=output.ts

how did you generate "(Simulation UDP Stream used gstreamer) via port 5600 "?

longvuvan083 · December 5, 2023, 6:30am

Hi @fanzh , Thank you for the support

here is the 1.log and 1.og for above gst-launch command
log.zip (7.6 MB)
I can use the commend to save some stream data in output.ts file

gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse !   mux. mpegtsmux name=mux ! filesink location=output.ts

i got the UDP simulation stream from PX4 gazebo camera plugin source , so i think the stream source is fine
Gazebo Simulation · PX4 Developer Guide

fanzh · December 5, 2023, 7:48am

do you mean the decoding state is sometimes good and sometimes bad?
could you share that output? I will have a try. You can use forum private email. please click forum avatar-> personal messages->new message.

longvuvan083 · December 5, 2023, 8:44am

i mean last time i get it to work somehow after activate jetson-clock. but only for that time, i tried again but not work. so i think that is not the solution so far.
yes, i was shared the rtsp output to you. pls check the DM

fanzh · December 6, 2023, 6:45am

AYK, decoder will decode successfully only after getting IDR frame. the rtspserver does not do transcoding. I suppose the camera 's h264 IDR idrinterval is too big. can you lower idrinterval to have a try?

longvuvan083 · December 8, 2023, 5:27am

Thank you, i tried to reduce the frame update rate from 60fps to 30fps. but it’s seem not work.

fanzh · December 8, 2023, 5:33am

not frame rare(fps), it is IDR frame interval.

longvuvan083 · December 8, 2023, 6:07am

I tried but cannot find anywhere in the codebase of the camera to change the IDR frame. However, I have an idea that the UDP stream of the camera is working fine, and my app can capture the data from the udpsrc source.

In my application, two clients are needed to capture the camera data, but currently, the udpsrc can only supply the stream to one client. Therefore, I think creating a UDP stream to multiple UDP ports can be a solution

fanzh · December 8, 2023, 7:29am

thanks for the update!

will playing rtsp://127.0.0.1:8553/live fail if waiting one minute?
could you do this test to get decoder low level log 2.log?

export GST_DEBUG=1 && touch /tmp/tegrav4l2_logs  && export TEGRA_TVMR_GENERAL_LOGS=1 && export TEGRA_VIDEO_LOGS=1 
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink >2.log 2>2.log
mv /tmp/tegrav4l2_logs /tmp/tegrav4l2_logs_1

from the output.ts you provided, there is only one IDR frame during 2 minutes. could you use this command-line to get 3 minutes h264 data? Thank you!

gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=test.h264

longvuvan083 · December 13, 2023, 11:11am

Hi fanzh, sorry for late reply.

no, the RTSP stream keeps working fine VLC
here is the attached 2.log and test.zip
2.log (452.5 KB)
test.zip (2.2 MB)

fanzh · December 14, 2023, 1:38am

Thanks for the sharing! could provide more device information? Thanks! here are some command-line.
CUDA version nvcc -V
dpkg -l |grep TensorRT
dpkg -l | grep gstreamer
cat /etc/nv_tegra_release or jetson_release
apt-cache show nvidia-jetpack

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)

longvuvan083 · December 14, 2023, 3:02am

@fanzh thank you, here is my device information

CUDA Version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_May__4_00:02:26_PDT_2022
Cuda compilation tools, release 11.4, V11.4.239
Build cuda_11.4.r11.4/compiler.31294910_0

Hardware Jetson AGX Xavier

image758×328 72.4 KB
TensorRT version: 8.4.1-1+cuda11.4

image1218×262 106 KB
nvidia-jetpack: (apt-cache show nvidia-jetpack)
Version: 5.0.2-b231
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-jetpack-runtime (= 5.0.2-b231), nvidia-jetpack-dev (= 5.0.2-b231)
Deepstream Version 6.1

fanzh · December 14, 2023, 5:56am

I can decode the test.h264 by this command-line
gst-launch-1.0 filesrc location=test.h264 ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

please the following command-line to play the rtsp source.
no rendering:

gst-launch-1.0 rtspsrc location=rtsp://10.19.227.166/media/video1! rtph264depay ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

with rendering:

gst-launch-1.0 rtspsrc location=rtsp://10.19.227.166/media/video1! rtph264depay ! h264parse config-interval=-1 ! nvv4l2decoder ! nv3dsink

Topic		Replies	Views
Deepstream_test1_rtsp_in_rtsp_out.py not working DeepStream SDK	16	598	April 14, 2023
RTSP in Deepstream DeepStream SDK	10	1114	October 12, 2021
Uridecodebin cannot decode an RTSP video stream DeepStream SDK nvbugs	23	6939	September 6, 2022
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection DeepStream SDK gstreamer	20	2086	July 11, 2022
Issues decoding RTSP stream using nvv4l2decoder with Jetpack 4.4 Jetson Nano rtsp , nvbugs	27	8244	October 18, 2021
Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1) DeepStream SDK	15	1631	October 12, 2021
Nvv4l2decoder does not work with certain rtsp video stream DeepStream SDK	13	2647	June 13, 2022
Gst-play-1.0, gst-launch-1.0 unable to display RTSP stream on Xavier AGX Jetson AGX Xavier rtsp , camera , gstreamer	8	2402	December 22, 2023
Nvv4l2decoder error DeepStream SDK	8	1800	October 12, 2021
Deepstream failed to decode rtsp feed withnvv4l2decoder DeepStream SDK	12	740	April 16, 2021

RTSP stream not work "Stream format not found, dropping the frame"

Related topics