GStreamer Video Capture and H264 Encoding with SEI

Nicolas Dufresne nicolas at ndufresne.ca
Wed Nov 4 14:47:43 UTC 2020


Le mardi 03 novembre 2020 à 23:09 -0600, steven414 a écrit :
> Hi,
> 
> I am trying to build an application which captures video from a Camera
> (1920*1088).
> 
> The region 1920*1080 will be encoded into H264. While the region 1920*8 is
> so-called embedded data, which should be extracted and added into SEI frame.
> Then one original raw frame should become one I or P frame and one SEI frame
> (embedded data). 
> 
> Currently I have built the pipeline to do the H264 encoding part like below:
> camerasrc -> videoconvert -> h264enc -> h264parse -> rtph264pay -> udpsink
> 
> Now I wonder:
> 1. How to create a corresponding SEI frame from the 1920*8 cropped region. 

As this is custom way of embedding data, you'll have to add you own
filter, and you'll have to do that before videoconvert, as videoconvert
might endup stripping off these padding lines.

So basically, implement a video fitler (passthrough) that will analyze
the binary coming out of the camera. If the information is really per
frame, I'd use a custom BufferMeta to embedded the information,
otherwise I'd use serialized downstream element event.

> 2. How to insert the SEI frame into H264 frame-by-frame.

For that, you need a second filter, this one will read your BufferMeta,
and transform it into vendor specific SEI messages. You can push the
SEI NAL indepedently, though it will be up to you to place it properly.
Using alignment=au, you simply need to identify the first VLC and
prepend. If performance is a problem, consider using stream-format=avc,
this way you can avoid the costy scanning process. The payloader do
support avcc.

> 
> 
> <http://gstreamer-devel.966125.n4.nabble.com/file/t379728/Capture.png> 
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel




More information about the gstreamer-devel mailing list