Generating frames from a paused video?

[Stefan Sauer]

On 05/20/2012 03:42 AM, Robert Schroll wrote:
> Hi all,
>
> I'm trying to put together a simple video player with overlaid
> controls.  At the moment, I'm using cairooverlay to draw the controls
> directly on the video stream.  When I receive user input, I just
> update some state variables so that when the next frame is drawn, it
> properly reflects the user's actions by prelighting controls, etc. 
> This works pretty well when the video is playing (low frame-rate
> videos have a bit of lag in the controls, but I'm willing to live with
> it), but it means nothing is updated when the video is paused.  I'm
> trying to work around this, but I'm new to gstreamer and don't know
> which ideas are feasible.  Any comments you have on which way to go
> (or if there's an entirely better way to do this) would be appreciated.
This is difficult. When the video is paused, the time is standing still
and no updates are pushed to the screen. I wonder if we could do
something clever using the recent VideoOverlayComposition api:
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideooverlaycomposition.html
Something like having a flag to mark them as 'live', so that video-sinks
would re-render them even on paused frames.

One general problem with your approach though is, that your UI update
rate is linked to the video framerate (but probably okay for you). You
could mitigate that using a videorate element in front of the sink to
fixate the fps to e.g. 30.

Stefan

>
> The ideas I've had so far:
> 1) Find a way to force cairooverlay to emit a frame down the pipeline
> when paused.  I could hook up a timer to call this occasionally when
> the movie is paused.  But I suspect being paused is an all-or-nothing
> thing for the pipeline, I can't have data flowing through only half of
> it.
>
> 2) Find some sort of equivalent of imagefreeze for videos, either as a
> source or a filter.  When activated, this would replace the video
> frames with a stream of that repeats the last frame over and over. 
> The pipeline would always be in the "playing" state, so the controls
> would always be updated.  But I don't know if such an element exists.
>
> 3) Use a videomixer to combine my desired video and a test source in
> such a way that only the former is visible.  (This could also up the
> refresh rate for low-frame rate videos.)  But I don't know if I can
> pause the sources to a video mixer individually, or what this would do
> to seeking.
>
> 4) What I'm doing right now is pausing the pipeline and then seeking
> to the current position every 50ms.  This triggers a new frame to go
> down the pipeline, on which I can draw the updated controls.  This
> works seamlessly for some videos, but for others it introduces
> half-second jumps on the pause, presumably because the seek is going
> to a keyframe or something.  There also seems to be a problem with
> seeking to the exact end of a video - I have to back up at least 1ns
> to get a new frame, but some videos seem to require more.  In short,
> this method seems sort of fragile, but it is the only one that I know
> how to make work.
>
> So, is there anyway to make (1), (2), or (3) work, or does gstreamer
> just not work that way?  Is there a way to improve (4), or are the
> problems with it that I'm not seeing yet?  Is there a better way to
> overlay controls than a cairooverlay?
>
> As I said, I'm new to gstreamer, so please don't assume I've
> considered something that seems obvious to you.  I'm working in Vala,
> but I'd be happy with example code in any language.
>
> Thanks in advance,
> Robert
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20120520/47d6fb4c/attachment.html>