RTSP in/out with deepstream python improvement with tracker and MQTT

Accelerated Computing Intelligent Video Analytics DeepStream SDK
5

For clarification, I use this code then modified it. I also currently learning gstreamer functions and its pipeline. Mind that I primarily use python on here.

For the updates, I’ve already modifed the code using MQTT and tracker. However, for MQTT, I copied the whole osd_sink_pad_buffer_probe function and differentiate them. So it ended up like this:

in main

...
osdsinkpad1 = nvosd[0].get_static_pad("sink")
    osdsinkpad2 = nvosd[1].get_static_pad("sink")
    if not osdsinkpad1:
        sys.stderr.write(" Unable to get sink pad of nvosd \n")
    
    osdsinkpad1.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe1, 0)
    osdsinkpad2.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe2, 0)
...

while in each osd_sink_pad_buffer_probe i placed MQTT program. For the results, the MQTT can publish properly to the broker for now. However, because i placed directly on the probe. The publish has no fixed period which I assume the publish speed is the same as stream FPS (I haven’t tried FPS check on the code, but I won’t discuss the FPS for now).

For the specifics:

def osd_sink_pad_buffer_probe(pad,info,u_data): #i have two different function names with same function but different mqtt topic
    frame_number=0
    #Intiallizing object counter with 0.
    obj_counter = {
        PGIE_CLASS_ID_VEHICLE:0,
        PGIE_CLASS_ID_PERSON:0,
        PGIE_CLASS_ID_BICYCLE:0,
        PGIE_CLASS_ID_ROADSIGN:0
    }
    num_rects=0

    gst_buffer = info.get_buffer()
    if not gst_buffer:
        print("Unable to get GstBuffer ")
        return

    # Retrieve batch metadata from the gst_buffer
    # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        frame_number=frame_meta.frame_num
        num_rects = frame_meta.num_obj_meta
        l_obj=frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break
            obj_counter[obj_meta.class_id] += 1
            try: 
                l_obj=l_obj.next
            except StopIteration:
                break

        # Acquiring a display meta object. The memory ownership remains in
        # the C code so downstream plugins can still access it. Otherwise
        # the garbage collector will claim it when this probe function exits.
        display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
        display_meta.num_labels = 1
        py_nvosd_text_params = display_meta.text_params[0]
        # Setting display text to be shown on screen
        # Note that the pyds module allocates a buffer for the string, and the
        # memory will not be claimed by the garbage collector.
        # Reading the display_text field here will return the C address of the
        # allocated string. Use pyds.get_string() to get the string content.
        py_nvosd_text_params.display_text = "Frame Number={} Number of Objects={} Vehicle_count={} Person_count={}".format(frame_number, num_rects, obj_counter[PGIE_CLASS_ID_VEHICLE], obj_counter[PGIE_CLASS_ID_PERSON])

        # Now set the offsets where the string should appear
        py_nvosd_text_params.x_offset = 10
        py_nvosd_text_params.y_offset = 12

        # Font , font-color and font-size
        py_nvosd_text_params.font_params.font_name = "Serif"
        py_nvosd_text_params.font_params.font_size = 10
        # set(red, green, blue, alpha); set to White
        py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)

        # Text background color
        py_nvosd_text_params.set_bg_clr = 1
        # set(red, green, blue, alpha); set to Black
        py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
        # Using pyds.get_string() to get display_text as string
        print(pyds.get_string(py_nvosd_text_params.display_text))
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)

        ### i insterted MQTT program here
        
        try:
            l_frame=l_frame.next
        except StopIteration:
            break

	### i copied tracker (from deepstream-test2) here

    return Gst.PadProbeReturn.OK