[Swfdec] Integrating Swfdec and GStreamer

[David Schleef]

On Thu, Mar 22, 2007 at 10:44:48AM +0100, in7y118 at public.uni-hamburg.de wrote:
> - Flash files can load (multiple) files (at once). Currently there's  
> no good way to request data from elsewhere. I've been told there is an  
> event to request a different URI that is used by some demuxers, but  
> that requires the current file to be complete and does not support  
> more than one file.
> - If Flash wants to open new files, it needs the URL of the input for  
> local URL resolving and for security management. Last I looked the URL  
> was not part of the (meta)data in the stream.

These two reasons are why swfdec should really be 'swfplaybin'.

> - Rendering a Flash movie requires a lot of drawing operations. Swfdec  
> uses cairo for this. The only available solution for cairo in  
> GStreamer is to use image buffers. Rendering to images is pretty much  
> the slowest rendering method cairo offers.

I've done some experimental stuff related to replacing x[v]imagesink
with a cairosink, and passing a cairo context upstream for the
decoder to draw on.  It's a hack, and doesn't work through queues
and such, but it has the potential to morph into a decent solution.
And it seamlessly used Xrender as a backend.

> - A lot of Flash files take 100% CPU and require frame dropping to get  
> smooth audio playback and OK video. Back when I dropped the plugin,  
> the QoS framework did not exist. I haven't looked if the GStreamer QoS  
> would be up to the task now.

It works quite well for video playback.

> - Modern Flash relies a lot on user input. Particular in 0.8 there was  
> a lot of buffering taking place inside pipelines, which made user  
> input feel very unresponsive. I have no idea how much that is still  
> the case in 0.10.

Yeah, mouse control in "vts ! navigationtest ! xvs" still feels a bit
"sloppy".  This comes from two problems in navigationtest: waiting
until the next buffer comes from videotestsrc to update the picture
with the new nav info, and not flushing the existing (and now stale)
buffer out of the xvimagesink.  Thus, if buffers have timestamps 1.0,
2.0, 3.0, etc., an event at time 1.2 gets processed by navigationtest
at approximately time 2.0+epsilon to be applied to buffer 3.0.

Also, gstreamer navigation still has the severe problem of not passing
around enough information for keypresses.  It *might* get "a" and "A"
correct, but perhaps not "#" and "£", or tab and shift-tab.



dave...