Vulkan Example

[Matthew Waters]

Hi,

The GstVulkanImageMemory has field that specify the latest memory
barrier required by the next processing.  Have a look at what any of the
other vulkan element do with the incoming GstVulkanImageMemory.  They
only do memory barrier operations as currently everything occurs on the
same Vulkan queue.  Multiple Vulkan queue processing is a TODO item for
where that may make sense and there semaphores or events would be required.

If you need semaphores for whatever you are doing, you need to implement
that yourself.  Semaphores aren't needed for every use case, especially
when running on the same vulkan queue.  Pre-creating VkImage's is also
limiting for other scenarios and in general for GStreamer usage.

VkImage's area created as needed and you could either get a completely
new VkImage each frame, or go through a small number of them.  The exact
details depend on the upstream elements and e.g. whether they use a
buffer pool or not.

You need to insert the relevant barrier operation to ensure writes have
finished before you read from the VkImage as required by the Vulkan
specification.  If you are accessing the Vulkan image from another
queue, then you also need to perform a queue transfer.

You can keep a reference to the GstBuffer for as long as you like,
however that may cause more VKImage's to be created to accomodate your
processing.

Cheers
-Matt

On 3/3/22 13:10, Joel Winarske wrote:
> I am now able to get GstVulkanImageMemory/vkImage from a sample:
> https://gist.github.com/jwinarske/dda0f7154a0bf34c3e7b6606907ad0f7
>
> What is the state of the vkImage when I read it from the sample? Ready
> to use and safe until the next incoming sample?
> Do I need to do anything with the barrier?
> How many vkImages are created, and in use?
>
> It might be good to have an API where one can pass in an array of
> pre-created vkImages for use by the pipeline.  Then semaphores can be
> used externally for optimal threading and the presentation logic.
>
> Cheers,
> Joel
>
> Thanks,
> Joel
>
>
> On Wed, Mar 2, 2022 at 12:12 AM Matthew Waters <ystreet00 at gmail.com>
> wrote:
>
>     appsink is your friend and you can retrieve the VkImage's from
>     upstream and you can do what you like with them.
>
>     There's no requirement to use vulkansink however you would need to
>     make sure to interop with the relevant Vulkan objects between the
>     application and GStreamer which may require some API additions in
>     GStreamer for that case.  The design is similar to OpenGL
>     integration but there are some differences with how Vulkan itself
>     works that need to be accounted for when writing your own glue code.
>
>     Cheers
>     -Matt
>
>     On 2/3/22 10:02, Joel Winarske wrote:
>>     I'm looking to use GST with an external swap chain.  Is there
>>     currently a way to get a callback or signaled semaphore (common)
>>     when a VkImage is ready for external use?  I'm thinking I may
>>     need to implement a variant of vulkansink.
>>
>>     The external swap chain is running multiple pipelines, and
>>     handling display presentation logic.
>>
>>     On Mon, Feb 7, 2022 at 6:40 PM Matthew Waters
>>     <ystreet00 at gmail.com> wrote:
>>
>>         Apart from running with vulkan elements with gst-launch-1.0,
>>         yes. That's
>>         the only application example for Vulkan.  Note that Vulkan
>>         does work on
>>         macOS/iOS through the MoltenVK layer.
>>
>>         On 8/2/22 12:38, Joel Winarske via gstreamer-devel wrote:
>>         > Is this the only Vulkan example?
>>         >
>>         >
>>         https://gitlab.freedesktop.org/gstreamer/gst-examples/-/tree/master/vulkan
>>         >
>>         > Thanks,
>>         > Joel
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220303/83ec1ca9/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220303/83ec1ca9/attachment-0001.sig>