[gst-devel] scale-renegotiation

[Ronald Bultje]

Hi,

given the fact that we still don't properly support software-scaling in
gst-player or audioscaling for soundcards that only run on 48kHz (see
bugzilla, we have one such open bug plus several dups), I want to
propose a system to do this. Julien, me and other interested people can
then implement it and we're all happy.

1) what should the app do:
it should add a videoscale plugin or an audioscale plugin or whatever
plugin in the pipeline.

2) what should the plugins do:
tell each other when to use software scaling and when to be an identity.

3) what should the core do:
nothing.

I will mainly describe how (2) will work, since 1 and 3 are obvious. My
example pipeline will be gst-launch filesrc location=file.avi ! avidemux
.video_00 ! { queue ! ffdec_mpeg4 ! videoscale ! ximagesink }.

The AVI file has a size, and therefore sets size via capsnego on the
following plugins. I will assume that ximagesink creates its own
XWindow, which can either be a toplevel window or can be a window
reparented to the XID given by an application through the XOverlay
interface. It might also draw on the given XID directly (it doesn't
matter).
When the window is resized (e.g. widget resize, toplevel window resize
via the window manager, etc.), an XEvent will be queued in the x socket
connection and it can be read by ximagesink. We assume that ximagesink
reads from this queue at the *end* of the chain function (this is
important!). When such an event comes in, it will send a GstEvent into
the pipeline with the "wanted" new caps (not size!). If there's a
videoscale element in the pipeline, it reads the event, redoes capsnego
and all will be happy. The sinkconnect() function of ximagesink will
resize the window. If there's no element handling the resize event,
nothing happens and ximagesink will have to deal with it in some other
way. Currently, it puts the unresized video in the middle of the window,
and this is probably the best idea.

Audioscale will work exactly the same way, except that it scales audio
rather than video. this could also be used for other processes where a
different caps than the current one is wanted, I can't find an example
right now, but there's surely more situations than just scaling.

This is tricky because the scale elements will have to do a part of the
caps negotiation process themselves, i.e. check the incoming caps of the
event for validity. Normally, the capsnego process does this itself.
Maybe there's some way to automate this, but I didn't find it yet.
A better way to do this would be to redo capsnego directly with a
chained caps from the ximagesink plugin: it sends back a chained caps
where the first one is the caps with the new size (wanted), and the
second is the current one. Problem here is - currently - that this will
only work if the videoscale plugin is directly connected to ximagesink.
If there's plugins in between, e.g. a fps-changer, it won't work. If
someone knows of a way to fix this: I prefer this method. ;).

Comments?

Ronald

-- 
Ronald Bultje <rbultje at ronald.bitfreak.net>
Linux Video/Multimedia developer