[gst-devel] Reduce GstXvImageSink rendering when its window is invisible

[David Schleef]

On Mon, Dec 13, 2010 at 10:55:14AM +0800, Le, Yongnian wrote:
> Hi,
> 
> I am working on multimedia application development using gstreamer
> framework. There might be many cases that some window is invisible
> (for example other window is in full screen), its related XvImageSink
> is still busy on calling gst_xvimagesink_show_frame for video
> rendering. I am wondering that we could have optimization to detect
> additional "Obscured" xevent and reduce the rendering frequency to
> very low value, like 1 fps, if such window is obscured. In this way,
> we can save precious resources for most applications, like CPU and
> power, at the same time, having minimal impact to end user. Does that
> approach make sense? What do you guys think of it? Thanks a lot for
> your feedback and comments.

Right now, the UI toolkit (Gtk+, Qt) already keeps track of windows
being mapped or unmapped, and in some cases, also the obscured or
partially-obscured information.  So it would be straightforward to
extend the GstVideoSink base class to have a property that encourages
degraded rendering.

It is tempting to put the obscurity detection directly in the
video sink, however, leaving this to the application allows for
the possibility that the app is obscuring the video on purpose,
yet doesn't want the quality degraded.  Perhaps both methods are
useful.

In general, however, rendering is a tiny fraction of the cost of
playing video, so it would be good to also push this information
upstream, either as part of a QoS event (e.g., proportion=0.0 for
video that will be obscured), or (better) an extension of the QoS
event.  That way, video decoders can avoid decoding the video.

And, for bonus points, as there is a trend in the industry toward
stream switching for video over the internet, the QoS information
could be used to select a stream with low video bit rate, or
audio-only stream.



David