[videocrop/compositor] Change output of videocrop just in time using GstVideoCropMeta metadata

[Nicolas Dufresne]

Le dimanche 29 mai 2016 à 10:10 -0700, Arjen Veenhuizen a écrit :
> I use videocrop to crop a specific region per frame. This region (x, y,
> width, height), however, also changes for each frame. I cannot rely on out
> of band pad-probes and change the videocrop element properties (left, right,
> top, bottom) in a pad probe callback due to overhead so I have (and want to)
> do this in-band. Therefore I have modified the videocrop element to accept
> GstVideoCropMeta metadata and use that to define the output per frame. The
> videocrop element will append GstvideoCropMeta metadata to its output buffer
> as well instructing the modified compositor element on how to position and
> scale its input frame on its sinkpad. 

That's a great change, hope you'll share this, it would be really nice.

> 
> Problem 1: 
> If I understand things correctly, I am already too late to set the output
> frame width/height when I am parsing the input buffer in the videocrop
> element. If I would still output a frame with the required width/height (as
> signalled in the GstVideoCropMeta metadata), I would potentially start
> writing in unallocated memory (because my output frame is simply not big
> enough) causing visual distortions and sooner or later a segfault. As a
> partial work-around, I signal the width and height of the next output frame
> one frame in advance so that the src pad can negotiate the proper caps on
> time but this is far from ideal. Is there a way to properly solve this
> problem?

It's a general bug in the way videocrop negotiate. The final
negotiation can be moved to when the frame is to be produces. This
would at the same time remove few races that occurs when we change
properties during the negotiation process.

> 
> Problem 2:
> In case the input framerate is equal to the output framerate, things run
> relatively smoothly using the workaround above. However, if the input
> framerate is lower than the output framerate, I notice all kinds of visual
> glitches in the output video. I am not an expert on the exact inner-workings
> of the compositor element, but this is my assessment so far:

Interesting, seems like a seperate bug. Looks like a buffer being
released, but still used.

> 
> I change the output caps of videocrop src pad 10 times per second, but the
> compositor element queries the sink pad linked to the videocrop element 25
> times per second(?). Since I change the src pad caps just after a frame is
> output on the src pad, the compositor element would (at the next interval)
> query an incorrectly sized frame (because the src pad properties already
> changed but no new frame was rendered on the videocrop src pad). This way,
> the videocrop element expects a bigger/smaller frame on its sink pad than
> actually available. Is this assessment correct or am I making a mistake?

A VideoCropMeta should be applied independently from the properties. In
fact, the properties should apply on top of the meta. You might have a
bug in your implementation here.

Hope this helps, if you need more precise guidance, share some code.

regards,
Nicolas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160529/c177be6b/attachment.sig>