Generating frames from a paused video?

Robert Schroll rschroll at gmail.com
Sat May 19 18:42:58 PDT 2012


Hi all,

I'm trying to put together a simple video player with overlaid controls. 
  At the moment, I'm using cairooverlay to draw the controls directly on 
the video stream.  When I receive user input, I just update some state 
variables so that when the next frame is drawn, it properly reflects the 
user's actions by prelighting controls, etc.  This works pretty well 
when the video is playing (low frame-rate videos have a bit of lag in 
the controls, but I'm willing to live with it), but it means nothing is 
updated when the video is paused.  I'm trying to work around this, but 
I'm new to gstreamer and don't know which ideas are feasible.  Any 
comments you have on which way to go (or if there's an entirely better 
way to do this) would be appreciated.

The ideas I've had so far:
1) Find a way to force cairooverlay to emit a frame down the pipeline 
when paused.  I could hook up a timer to call this occasionally when the 
movie is paused.  But I suspect being paused is an all-or-nothing thing 
for the pipeline, I can't have data flowing through only half of it.

2) Find some sort of equivalent of imagefreeze for videos, either as a 
source or a filter.  When activated, this would replace the video frames 
with a stream of that repeats the last frame over and over.  The 
pipeline would always be in the "playing" state, so the controls would 
always be updated.  But I don't know if such an element exists.

3) Use a videomixer to combine my desired video and a test source in 
such a way that only the former is visible.  (This could also up the 
refresh rate for low-frame rate videos.)  But I don't know if I can 
pause the sources to a video mixer individually, or what this would do 
to seeking.

4) What I'm doing right now is pausing the pipeline and then seeking to 
the current position every 50ms.  This triggers a new frame to go down 
the pipeline, on which I can draw the updated controls.  This works 
seamlessly for some videos, but for others it introduces half-second 
jumps on the pause, presumably because the seek is going to a keyframe 
or something.  There also seems to be a problem with seeking to the 
exact end of a video - I have to back up at least 1ns to get a new 
frame, but some videos seem to require more.  In short, this method 
seems sort of fragile, but it is the only one that I know how to make work.

So, is there anyway to make (1), (2), or (3) work, or does gstreamer 
just not work that way?  Is there a way to improve (4), or are the 
problems with it that I'm not seeing yet?  Is there a better way to 
overlay controls than a cairooverlay?

As I said, I'm new to gstreamer, so please don't assume I've considered 
something that seems obvious to you.  I'm working in Vala, but I'd be 
happy with example code in any language.

Thanks in advance,
Robert


More information about the gstreamer-devel mailing list