ZeroCopy + AppSink + VAAPI + Qt

[Nicolas Dufresne]

Le jeudi 30 janvier 2020 à 09:45 +0000, Timtchenko, Michael a écrit :
> Hi folks,
> 
> i'm trying to implement an efficient, zero copy image handling on a EGL-System 
> without a window manager. Unfortunately this seems to be a quite hard job. :-/
> 
> My application uses the following pipeline:
> 
> "udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-
> rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" "
>                                          "! rtph264depay "
>                                          "! vaapih264dec "
>                                          "! vaapipostproc name=postproc "
>                                          "! video/x-raw, format=RGBA "
>                                          "! appsink name=sink emit-
> signals=true sync=false max-buffers=1 drop=true"

In this context, you are asking your GPU to upload to system memory. As the GPU
does not operate in LINEAR pixel formats, this requires a copy (detiling,
decompression, etc.). If you goals is to manually import your decoded image to
GL, I would suggest requesting DMAbuf, VASurface, or the deprecated
GstVideoGLTextureUploadMeta.

  video/x-raw\(memory:DMABuf\)
  video/x-raw\(memory:VASurface\)
  video/x-raw\(meta:GstVideoGLTextureUploadMeta\)

And then use the appropriate API to zero-copy import that to EGL. With the
preferred method (DMABuf), you need to get the GstMemory and get the DMABuf FD
using gst_dmabuf_memory_get_fd(). The rest is EGL calls, so left to you.

> 
> As you can see, i'm using vaapi elements as well as the appsink. I'm getting
> notified by signal, when a new frame is received.
> Afterwards i'm pulling the frame by the following code:
> 
> #####
> 
> GstSample* sample = gst_app_sink_pull_sample( m_data->appSink );
> GstBuffer* buffer = gst_sample_get_buffer( sample );
> 
> GstMapInfo* map = new GstMapInfo();
> gst_buffer_map( buffer, map, GST_MAP_READ );
> 
> and access the data by
> 
> map->data()
> 
> ####
> 
> This is the status quo, but my goal is to put/bind the data into a texture and
> render it within the qt screnegraph without any additional copy actions. To
> achieve this,
> I need to get a already rendered texture or a direct access to the framebuffer
> object. Currently i'm  uploading the data with the OpenGL-function
> glTexImage2D which isn't very efficient at all.
> 
> In (1) they said, that this can be done by:
> 
> [...Using memory:VASurface you can map it as a normal gst video frame:
> gst_video_frame_map ...]
> 
> Unfortunately i don't have a clue, how i can access a texture or a
> framebufferobject
> from the VASurface. Is that even possible?
> 
> Is there someone who can share his knowledge? Any help is welcome! :)
> 
> Thanks in advance,
> Michael
> 
> (1) 
> http://gstreamer-devel.966125.n4.nabble.com/How-to-use-VAAPI-appsink-with-zero-copy-mmap-td4688670.html
> 
> 
> Bitte beachten / Please note!
> 
> ******************************************************************************
> *
> 
> AGCO GmbH
> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
> Registergericht Amtsgericht Kempten HRB 10327
> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr.
> Heribert Reiter, Ekkehart Glaeser
> Vorsitzender des Aufsichtsrates: Torsten Dehner
> 
> ******************************************************************************
> *
> 
> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und
> kann vertrauliches
> bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin
> enthaltene Ansicht
> oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die
> Ansicht oder
> Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail
> irrtümlich
> erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift
> oder jeglicher
> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender
> übernehmen die
> Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren
> angehängte
> Dateien (sofern vorhanden) auf Viren zu überprüfen.
> 
> ******************************************************************************
> *
> 
> This email is intended solely for the use of the individual to whom it is
> addressed and may contain
> confidential and/or privileged material. Any views or opinions presented are
> solely those of the
> author and do not necessarily represent those of AGCO. If you are not the
> intended recipient, be
> advised that you have received this email in error and that any use,
> dissemination, forwarding,
> printing or copying of this email is strictly prohibited. Neither AGCO nor the
> sender accepts any
> responsibility for viruses and it is your responsibility to scan and virus
> check the email and its
> attachment(s) (if any).
> 
> ******************************************************************************
> *
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel