Question about integrating proprietary h264 encoders in gstreamer

[Carlos Rafael Giani]

On 2015-02-03 15:03, Ottavio Campana wrote:
> Well, I'd love not to have to deal with this "monster" wrapper. In fact, my first try was to wrap the source and the encoder in two different elements, using GstBaseSrc and GstVideoEncoder classes, because this solution gives much better flexibility.
>
> The problem that I faced was that I am having horrible performance, because I need to copy the acquired frame into a GstBuffer per passing it to the GstVideoEncoder and this makes the whole thing terribly slow.
>
> I checked v4l2src, as far as I understand it also copies the acquired frame before re-enqueuing it in the driver, which is something I want to avoid.
>
>
> My perfect solution would be a source that passes the pointer of the acquired frame to the successive elements without copying them and that re enqueues the uncompressed frame in the driver once all the processing is done. But I haven't found yet an example to do that, therefore I think it is not possible.
>
>
> Ottavio
>
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

No, this is possible. It involves several steps though. I do similar 
things to wrap DMA buffers in GstBuffers.

A GstBuffer is made of metadata (timestamps etc.), GstMeta blocks (not 
to be confused with the aforementioned metadata), and GstMemory blocks.
GstMemory blocks are the actual memory the GstBuffer represents. GstMeta 
blocks are additional, free-form meta information on top.

By default, GstMemory blocks are allocated with the internal sysmem 
allocator. If you run gst_buffer_new_allocate() and pass NULL for the 
allocator, a buffer with one GstMemory sysmem block is created. I needed 
to have this allocate DMA memory, so I wrote my own allocator, derived 
from the GstAllocator class. In the encoder, you can then either walk 
the GstMemory blocks, and see if one of them uses the allocator you 
defined, or (this is what I do) you add your own custom defined GstMeta 
block, and in the encoder, check for its presence.

In the case of gstreamer-imx en/decoders, I add a GstImxPhysMemMeta 
block to the buffers. When for example the encoder receives buffers, and 
determines that this meta is present, it knows that this is physically 
contiguous memory (= DMA memory). The meta also contains extra 
information I need for the encoder (specifically, the DMA block's 
physical address). Only if this meta is *not* present, it copies the 
frame with the CPU.

Have a look at:
https://github.com/Freescale/gstreamer-imx/blob/master/src/vpu/encoder/base_enc.c#L607
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_allocator.c 
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_allocator.h
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_meta.c 
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_meta.h
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_buffer_pool.c 
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/phys_mem_buffer_pool.h

Your src element would have to use a custom decide_allocation function 
to make sure your src element uses your custom buffer pool. Here is an 
example: 
https://github.com/Freescale/gstreamer-imx/blob/master/src/common/blitter_video_transform.c#L1032