[gst-devel] GStreamer and gnome 2.2
Thomas Vander Stichele
thomas at urgent.rug.ac.be
Tue Oct 29 04:36:02 CET 2002
taaz asked me to send some info regarding the inclusion of GStreamer into
Gnome 2.2 and what that means for us.
Firstly, the current line of thought seems to be that GStreamer is going
to be included in one form or another in Gnome 2.2. It mostly depends on
whether apps use it ;)
Second, there are four or five categories of "platforms" in Gnome 2.2.
Each one has different requirements and contents.
- contains core libraries used by lots of apps
- ABI/API stable for all of Gnome 2.x lifecycle
- contains core apps, as well as some of the libraries used by them
- probably has to be UI- and feature-stable during each 2.x cycle
- less stringent ABI requirement, but don't do silly stuff
Fifth Toe (& Multimedia)
- cool fringe gnome apps
- likely candidates could be galeon, gnomemeeting, and probably others
- could contain GStreamer-based apps like rhythmbox and have a
- a newly created platform for cool geek toys.
Now, where would GStreamer fit in ?
For now, we probably shouldn't try to get it in the Development Platform.
First of all, there's no real need, second, the requirements are a lot more
stringent and while we believe we may be able to fulfill them, it's probably
better not to stifle our development just because we are going into Gnome.
More likely than not, we will still uncover issues for which we have
better solutions when people really test the library more.
(This is not necessarily always bad ;) xine is just completing a complete
library API rewrite in the middle of their 0.9.x cycle I've been told)
Powertools is also not the place for us. Currently it contains fringe
hardcore hacker tools and eyecandy ;)
So that leaves us with two platforms. Of these, we were originally going
to go into fifth toe/multimedia.
Now, originally rhythmbox was going to go into FT(/MM), but for various
reasons jorn has decided to make rhythmbox more mature before putting it
into the core desktop.
The current line of thought is that we are going to go into Desktop Platform,
where we would be supporting (at the very least) iain's gnome media package.
The choice of if GStreamer would go into the Desktop Platform has to be made
in (roughly) the next two days. Feature freeze and module list is
for Thursday 31st (which probably is Wednesday since it's on Oceanic time ;))
It depends on the applications using GStreamer that are proposed for the module
So, here's a shortlist of possibilities that could be made ready for inclusion
in Gnome 2.2 :
contains the sound recorder which iain has moved to GStreamer and
rumour has it he is happy with the transition. "no more sox" is a good
works with gstreamer, of course. It is less HIG-compliant than totem
and that is mostly a UI problem, but one that needs fixing on a pretty
short notice. the 31st is a module/feature freeze, so that still leaves
us time to fix up the UI.
currently uses xine, but HIG-compliant. Totem could be ported to use
GStreamer, Bastien is all for it (but is not going to do it himself).
I had offered to help out in that area, but I'm not sure I could get
that done on such short notice with my still limited video skills ;)
Someone here should outline the work that needs to be done to make this
happen - from what I can see it shouldn't have to be too hard to support
at least a first subset of what xine provides - ie, basic movie playback
4) a video thumbnailing application
nautilus 2.2 features pluggable thumbnailing, which basically launches
an app for each mime type it encounters. So, for example, the mime type
video/avi would use the thumbnailer "gst-snapshot %i %o" which would
create a thumbnail png of the input video.
This thumbnailing app is stupendously easy to write since basically
you can pretty much do that using a gst-launch line with snapshot at this
We have discussed changing snapshot's strategy because it depends on two
libs instead of one. We might investigate this a little so we can solve
this properly, but it's easy to make this app happen in any case.
there IS a music view in nautilus currently, but
- it only does mp3
- it contains a hacked-up version of (I believe) mpg123
- Red Hat doesn't even ship it due to licensing issues
- it is album-centered; it shows the dir, tries to pretend the dir is
an album, and even has room for display of an album cover and so on.
A bit overkill in a file browser perhaps.
I started learning some nautilus and wrote an audio view for nautilus that
basically works, but needs some improving UI-wise. It'll focus on
providing features that allow you to easily browse and scan through a bunch
I intend to make roughly the same view for video as well, but with a
fixed-size small video window in the view somewhere.
6) nautilus property pages
James Willcox has created a system to have pluggable property pages as well
in nautilus. I'm currently focussing on getting the necessary stuff in
place to have a streaminfo library that would fetch metadata and caps
on files for display in properties and so on.
My hope is to make this happen so that the property pages use these
functions. One of the planned features for this lib is to implement
a streaminfo cache on-disk similar to the nautilus thumbnail cache.
Wim convinced me that there is no problem in having GStreamer doing efficient
handling of metadata and stream info, and after implementing it in vorbisfile
I have to agree with him.
7) nautilus mouse-over play
Alex Graveley from Ximian has worked on a nautilus hack that basically
"drew" a video widget over the thumbnail of a video in the icon view
in nautilus, and after 1 sec of mouseover the video itself would play
"in the thumbnail". That's a pretty cool hack ;) I hope to somehow
get that functionality in nautilus somehow. We've discussed it and
it's actually in theory not that hard to do. We have to figure out
though how to provide this without making nautilus depend on GStreamer
at compile-time for now. James Willcox seems to be willing to
see how a mechanism could be provided for nautilus to allow the "thumbnail
widget" to be replaced with a video widget. If we can pull this off,
this will be a kick-ass eye candy feature.
is currently focusing on stability and some extra features and probably
is not going to propose itself for inclusion anywhere until the groups
have been implemented. I hope they can get it to work as soon as possible,
because rhythmbox rocks ;) but it's already been decided not to propose
rhythmbox for the module list this week. It might make a fifth toe
or multimedia release.
So, that's a pretty good list. Not all of them will make it, I think,
and I have to check up on when everything needs to be ready. The upcoming
freeze this week is a "feature and module decision" freeze. It means that
we should have a good idea of the features we want to provide in gnome 2.2,
and make sure we don't bite off more than we can chew.
So currently, what seems likely to me to get done is :
- (1) goes into desktop platform
- (2) gets proposed with commitment on our side to make it HIG-compliant
(which means somebody has to READ it in the first place here - come on,
don't be scared). We will get help from usability guys (Calum,
This probably depends on Steve Baker's ability and willingness to
make the changes and someone to assist him in translating the HIG to
gst-player. Someone should also just ask the usability guys to give
us a UI review so we can start somewhere.
- (3) seems like a more long-term plan, especially since I am currently
focussed on the other stuff
- (4) should be easy to write, and KConger (on irc) has done this over the
weekend. I'll prod him to see how it's doing. So I consider this
"going in" as well, possibly as a part of nautilus-media.
- (5) the audio view part of nautilus can be made ready in relatively short
time. The biggest things to get right here are
- writing a static autoplugger that actually knows when it can't play
- getting the streaminfo system basics in place so that it can
do queries on the audio types we currently want to support
(which I'm probably going to limit at this point to ogg, mp3, wav and
- (6) I suggested to James to make these use the new streaminfo system and add
these pages to nautilus-media, and he seemed to think that was a good
idea, so if the streaminfo works, then these pages can also go in.
- (7) depends on James' ability to a) provide the mechanism and b) convince
Alex Larsson and Dave Camp of the sensibility of it.
I don't want to push James into doing it, it depends on his availability.
I do hope he finds the time to help us out there though.
- (8) is not going in at this point and will need to be reconsidered in the
future. I hope to move the metadata handling in monkey-media over to
GStreamer so that monkey-media itself doesn't need to depend on any
decoding library and is thus fully pluggable.
Now, for the bad news. What does this mean we should do in GStreamer ?
We currently have three major problems that need thought.
1) We cannot fully trust our current pthread schedulers. We have seen
mysterious crashes on reuse of pthreads, and the glibc people confirmed
that you're not allowed to reuse the stack even after the thread using
it has been joined. For all uses we have had up to now, this isn't a big
problem, but it seems that in the nautilus music view the code I currently
use starts getting major problems when playing the 8th track.
Having stack sizes of 2MB be wasted for each thread that gets created is
thus probably not a good idea if you cannot reuse those 2MB in the same
ways to fix this :
- the optimal scheduler, which Wim claims is almost ready
- move the omega scheduler to use GThreads. David Schleef says this can
be done easily, he just needs to change some API in the core for that to
- reuse pthreads instead of joining them. This would make our use of
pthreads more efficient and avoids crashes on stack reuse.
- make stack space per thread smaller - this would imply making each
cothread stack space a bit smaller, or allowing for less elements in one
thread. Both will probably work (the 2MB is largely arbitrary) but
are workarounds, not fixes
2) Our most-used autoplugger, spider, has MAJOR issues. Among these are
- it locks up when given a file it can't play. Try playing a text file
- sometimes it outright crashes when doing a recursive function nesting
because it can't figure out it can't play the file
- it is currently unmaintained and Company has been around again but
has not himself stated any willingness to work on it after careful
prodding on our part ;)
ways to fix this :
- somebody bites the bullet and investigates the code. All that needs to be
added is an option to spider that tells it how many times it needs to
loop the typefind, and set it to default 1. After that it should
send an element error. Basically ;)
- a static autoplugger; it would use typefind to determine the file type,
then replace itself with a static pipeline a la gst-launch-ext.
Currently the most feasible option. Would even be sensible to write
a fixed audio-only autoplugger.
3) being in gnome means having to run on all of the archs and platforms
gnome supports. I have doubts that out current thread code will work
on every platform. Jacob Berkman from Ximian has uncovered issues on
mac osx, and I doubt it will reliably work on Sun.
We need access to machines to test somehow and someone needs to set us
up with that.
Also, our code will have to become more portable, define out the
arch-specific bits (preferably in one place and not all over the code),
and generally be more robust.
Sun is rumoured to be setting up a machine outside their firewall for
testing, so we should be able to get access to that.
We need a complete list of archs gnome supports and find the right
testers for it.
I am going over our current set of media and writing up a simple script
to run these media files through gstreamer so we can get usable
feedback on how well we do.
4) We want to maintain some form of ABI stability in our library. Wim says
he can get this done in about a week, which means it would be done
by the time the second round of tarballs needs to go in. Jeff Waugh
is ok with that.
5) We need to do and/or figure out the following things :
- proper library versioning from now on
- define where all of our headers will get installed
currently, it's (prefix)/include/(module)-(version)/gst
My current idea is to bump us up to 0.5.0, and put ALL headers in
include/gstreamer-0.5 for the whole of the 0.5 devel tree used by gnome
We use this tree to work up to a "stable" 0.6 tree for gnome 2.2
- it would also mean that plugin libraries also put their headers in
gstreamer-0.5 instead of gst-plugins-0.5 like now.
- the library base name would also be libgstreamer-0.5.so
Basically, we either drop the micro number or drop the whole "version
in the name" scheme
- I will check these changes with experienced hackers who've seen this
before like owen, hp, jody and others.
6) We'll be doing a prerelease today for a version to be included this 31st.
We need this for iain's wavenc plugin. Basically, I could also do
a separate tarball with the plugin and throw it in with the 0.4.1 release,
but it'd be good to have a new release. It will depend on some of the
features that have been added, and Uraeus is of course going to help out ;)
So, a big set of possible changes. Exciting times are ahead, but PLEASE
comment on this mail because it WILL involve some changes and we need to
make sure we understand what this will involve.
On the plus side, we will be providing a cool desktop with great features
and we will put the wonder that is GStreamer to the test ;)
It's also good to mention that this is a major addition to Gnome 2.2, will
likely be a controversial issue, will get lots of discussion. We might
not make Gnome 2.2 desktop platform, but that's no reason to be
dissapointed if we don't. We still have the Fifth Toe release which we
can go in, and we can use this opportunity to check if we can make the
proper commitments and put GStreamer to the test.
The Dave/Dina Project : future TV today ! - http://davedina.apestaart.org/
<-*- thomas (dot) apestaart (dot) org -*->
I'd like to tell you how I feel
I'll probably keep it till a Saturday
<-*- thomas (at) apestaart (dot) org -*->
URGent, the best radio on the Internet - 24/7 ! - http://urgent.rug.ac.be/
More information about the gstreamer-devel