I am currently evaluating the feasibility of using the Holoscan Sensor Bridge IP to send a compressed video stream over to AGX Orin. One of the main challenges I am facing is that, by its nature, the size of the compressed frame coming out from the FPGA is not determined until after the entire frame is generated and I would like to avoid buffering the entire frame before sending it.
With this in mind, I wanted to get clarification on the following:
Would effectively changing the frame size mid-transmission or terminating a frame early via end-of-frame packet be possible using the Holoscan Sensor Bridge IP core? If so, how would one go about doing so?
Alternatively, what would be a recommended workaround for a schema like this under Holoscan infrastructure?
Technically it’s doable. But we currently don’t have an example application to demonstrate this. Here’s how you can RDMA the compressed pixel data into the GPU memory.
There are two indicators of frame end. One is when the data counter reaches the pre-configured frame size (bytes per frame) in the HSB FPGA IP, and two is the toggling of tlast signal of the sensor interface AXIS bus. Ideally these two events should happen at the same time for uncompressed frames. For variable frame size, you can configure the frame size (bytes per frame) to the largest byte size, if any frame is compressed to a smaller frame size value, you can toggle the tlast at an earlier time, the tlast will reset the frame byte counter and also generate an interrupt packet to the CPU to indicate the entire frame is sent. Once the data is in GPU memory, you can write a decompression pipeline to decode the pixel data for later processing.