Overlay with python, appsrc, cairo and a full cast of errors

[Sebastian Dröge]

On So, 2014-03-02 at 17:54 +1100, Michael Tyson wrote:
> Hello,
> 
> I’m putting together a python app that receives video data and pops an overlay on top of it, rendered using Cairo. The general idea seems to be to use appsrc, and to watch for 'need-data' and then render and push a buffer.
> 
> Little demonstration app: https://gist.github.com/michaeltyson/9302751
> 
> This appears to work, sorta-kinda, but:
> 
> 1. The buffer fill() call takes *ten seconds* at 720p
> 2. The whole thing only works with a really basic pipeline (appsrc | whatevervideosink)
> 
> Two questions:
> 
> 1. Gst.Buffer.fill takes 10 seconds for 720p. What can I do about that? I’d imagine allocating a buffer/buffer pool in advance and rendering straight into the buffer’s memory would be better, but Gst.MapInfo (as returned from Gst.Buffer.map()) has a ‘data’ field of type int, as opposed to some kind of byte array! What’s with that??

I would guess that this is so impressively slow because you get the data
from cairo into Python objects, and then copy the Python objects into
the GStreamer buffer. For every single buffer.

One way to make this work faster is probably by using the cairooverlay
element and do the overlaying there with just using cairo API and never
getting the frame data into Python.

> 2. How do I get this contraption to function with videomixer? When I run the above script, I see two of the following:
> 
>     (test.py:9818): GStreamer-CRITICAL **: gst_segment_to_running_time: assertion 'segment->format == format’ failed
> 
>     Running with GST_DEBUG=6 shows a bunch of this nonsense once the pipeline’s up and running: https://gist.github.com/michaeltyson/b004988e428d423ec27d (tl;dr: "gst_videomixer2_sink_clip:<mix:sink_0> Timestamped buffers required!” / "gst_base_src_do_sync:<overlay> we have no clock")
> 
> In response to the message about timestamped buffers I’ve attempted to:
> 
> 1. Set format=time on appsrc (this stops those GStreamer-CRITICAL errors showing up, but that’s all)
> 2. Explicitly add timestamps to the buffer, like:
>         self.buffer.timestamp = self.time
>         self.buffer.duration = Gst.SECOND * self.OVERLAY_FRAME_DURATION
>         self.time += self.buffer.duration
> 3. Add/remove do-timestamp=true to appsrc.
> 4. Combinations of the above
> 
> Any pointers whatsoever would be greatly appreciated.

You'll a) need to set appsrc into TIME format and b) either use
do-timestamp=true on it or explicitly set timestamps on each buffer.

You say you tried that, can you provide debug logs about the problems
that happen when you use this combination?

-- 
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140302/4fa7b21e/attachment.pgp>