Confused about GST_PAD_SET_PROXY_CAPS and negotiation

[Guillaume POIRIER]

Hi Tim, Hi everyone,

On Sat, Jan 24, 2015 at 4:47 PM, Tim Müller <tim at centricular.com> wrote:
> On Fri, 2015-01-23 at 17:46 +0100, Guillaume POIRIER wrote:
>> I'm trying to implement a video filter in GStreamer that would take
>> RGB as input (sink) and would output ARGB64 (src).
>> I naively thought that since my src and sink only support a fixed
>> format, I didn't need to perform any negotiation... is that right?
>
> That's not quite right. Your input and output formats may be fixed, but
> other things such as width, height, framerate, pixel aspect ratio etc.
> are not fixed (presumably).


I didn't think about that, but it makes sense.



>> Since my code didn't do what I wanted, I tried to add negotiation by
>> calling gst_event_new_caps() + gst_pad_send_event() on my sink and src
>> pads... but this didn't seem to work (gst_pad_send_event() returned
>> FALSE each time). How come? Who should have handled these events? Not
>> me I presume?
>
> What class does your element derive from? GstElement? GstBaseTransform?
> GstVideoFilter? Something else? (Should probably be GstVideoFilter)


My element derives from GstElement. I'm not quite confident with
GObjects for the moment, so I didn't dare go to far away from the
plugin template. I somehow thought that GstVideoFilter may be doing
too many things that I didn't need or want (QoS, frameskip, and
whatnot) which may get in my way... I guess it's time for me to get
more familiar with GObjects to learn writing a plugin that derives
from GstVideoFilter!



> The simplest implementation possible would be: when you receive a
> GST_EVENT_CAPS on your sink pad, you parse the caps, copy it (to make it
> writable), change the format from RGB to ARGB64, and then push a new
> caps event with the new caps out on your src pad. This ignores all other
> aspects of negotiation (such as queries, and buffer pools etc.). Just
> conceptually it can work like this.
>
> For more proper negotiation you would need to proxy things (caps) from
> downstream to upstream, whilst only changing the format.
>
> For example:
>
>  ... ! videoconvert ! videoscale ! yourfilter !
> video/x-raw,format=ARGB64,width=1920,height=1080 ! ...
>
> in this case the upstream elements need to know that you will only
> accept width=1920,height=1080 (since you can't scale), so you would have
> to return video/x-raw,format=RGB,width=1920,height=1080 to a caps query
> from upstream here.


Hey, thanks a lot for the detailed answer: so far I didn't manage to
handle negotiation events, but since I didn't know if I was going the
right direction GStreamer-wise, I didn't know what to fix. Now I know
(or at least you have narrowed it down for me). :-)


>> Then, just calling GST_PAD_SET_PROXY_CAPS() on each of my pads did
>> allow me to get a pipeline that seemed to have negotiated a format,
>> ... but the verbose output of gst-launch-1.0 does not show that my src
>> pad has been seen as ARGB64. When I look at the documentation of
>> GST_PAD_SET_PROXY_CAPS() states 'Set this if the element always
>> outputs data in the exact same format as it receives as input."
>
> This won't work in your case, since your output caps are different than
> your input caps.
>
>
>> My question is: what is the simplest way to have a filter take fixed
>> format in input, what do I need to do negotiation-wise (if any)?
>
> Derive from GstVideoFilter, and override
> GstBaseTransformClass::transform_caps() in your class_init function.
>
> In your transform_caps function you check the direction of the
> transform, and depending on it, you copy the caps (to make them
> writable), and loop over all caps structures, and for each set the
> "format" field to "RGB" or "ARGB64" (depending on the direction), but
> leave all other fields untouched.
>
> The base class should then handle everything else for you.


Many thanks! I will do that and hopefully quickly get something working.

Regards,

Guillaume