Encoding a DMABuf into a video stream with VAAPI

[Victor Jaquez]

On Mon, 20 Feb 2023 at 18:00, Michael via gstreamer-devel wrote:
>  
> 
> Hello,
> 
>  
> 
> I want to encode a framebuffer I got via dmabuf into a video stream. I have
> a dmabuf file descriptor which contains the framebuffer.
> 
> I got the filedescriptor from the intel i915 driver via ioctl
> VFIO_DEVICE_QUERY_GFX_PLANE. 
> 
> The dmabuf has the format BGRx and modifier is 0 respectively changes to
> I915_FORMAT_MOD_X_TILED.

I don't follow, the modifier is 0 or I915_FORMAT_MOD_X_TILED??

Right now, no drm modifier is negotiated. There are on-going efforts in this
direction, but they aren't finished yet.

> 
>  
> 
> Now I want to encode it per zero copy in gstreamer into a video stream
> (h264, h265 etc...). 
> 
> I push the single frames per appsrc into the gstreamer pipline. Since I use
> intel hardware I thought it makes sense to use VAAPI.
> 
> The problem is that the sink pads of vaapi only support video/x-raw and
> video/x-raw(memory:VASurface) and I have video/x-raw(memory:DMABuf).

What version of gstreamer-vaapi are you using?

Right now it support dmabuf as input. But also, it would be better if you can
use vah264enc/vah264enc in gst-plugins-bad which will replace those in
gstreamer-vaapi.

Still, both don't consume RGB frames. You'll need to add vapostproc /
vaapipostproc to do the color conversion. vapostproc / vaapipostproc also
process DMAbuf as input, and wil negotiate VA native format downstream with the
encoder.

> 
>  
> 
> Is there any way to convert video/x-raw(memory:DMABuf) to
> video/x-raw(memory:VASurface) (zero copy) or import the DMABuf directly as
> video/x-raw(memory:VASurface)?
> 
>  
> 
> Alternatively is there a plugin which is better suited than vaapi?
> 
>  
> 
> My code to push the frames into gstreamer currently looks like this:
> 
>  
> 
>     GstMemory* mem = gst_dmabuf_allocator_alloc(vedpy->gdata.allocator,
> dmabuf->fd, dmabuf->width * dmabuf->height * (dmabuf->stride/1024));
> 
>     vedpy->gdata.buffer = gst_buffer_new();
> 
>     gst_buffer_append_memory(vedpy->gdata.buffer, mem );
> 
>     
> 
>     gsize offset[GST_VIDEO_MAX_PLANES] = {0, 0, 0, 0};
> 
>     gint stride[GST_VIDEO_MAX_PLANES] = {dmabuf->stride, 0, 0, 0};
> 
>     gst_buffer_add_video_meta_full( vedpy->gdata.buffer,
> GST_VIDEO_FRAME_FLAG_NONE,
> 
>                                     GST_VIDEO_FORMAT_ENCODED,
> 
>                                     dmabuf->width, dmabuf->height, 1,
> offset, stride);
> 
>     GstFlowReturn ret;
> 
>     vfio_encode_dpy *vedpy = container_of(dcl, vfio_encode_dpy, dcl);
> 
>     g_signal_emit_by_name (vedpy->gdata.source, "push-buffer",
> vedpy->gdata.buffer, &ret);
> 
>     ...
> 
>    
> 
> And my pipline:
> 
>  
> 
> char launch_stream[]  = "appsrc name=source ! "
> 
>     "
> video/x-raw(memory:DMABuf),width=1024,height=768,framerate=0/1,format={BGRx,
> BGRx:0x0100000000000001} ! "
> 
>     " vaapipostproc !"
> 
>     " vaapih265enc ! 
> 
>     ...
> 
>  
> 
> which obviously does not work, because vaapipostproc cannot be linked with
> the filter.

Why not? They should link.  A debug log would be needed.