[gst-devel] GStreamer and gnome 2.2

[Julien MOUTTE]

Waoow :)

A lot of things to do here :)

i can help steve on the gst-player issues with libgstplay. I guess it will sooner or later drive us to totem changes to integrate libgstplay there...

Dunno how many time we would need for that...

See ya...

Dolphy

On Tue, 29 Oct 2002 13:33:45 +0100 (CET)
Thomas Vander Stichele <thomas at urgent.rug.ac.be> wrote:

> Hi guys,
> 
> taaz asked me to send some info regarding the inclusion of GStreamer into
> Gnome 2.2 and what that means for us.
> 
> Firstly, the current line of thought seems to be that GStreamer is going
> to be included in one form or another in Gnome 2.2.  It mostly depends on
> whether apps use it ;)
> 
> Second, there are four or five categories of "platforms" in Gnome 2.2.
> Each one has different requirements and contents.
> 
> Development Platform
> 	- contains core libraries used by lots of apps
> 	- ABI/API stable for all of Gnome 2.x lifecycle
> 
> Desktop Platform
> 	- contains core apps, as well as some of the libraries used by them
> 	- probably has to be UI- and feature-stable during each 2.x cycle
> 	- less stringent ABI requirement, but don't do silly stuff
> 
> Fifth Toe (& Multimedia)
> 	- cool fringe gnome apps
> 	- likely candidates could be galeon, gnomemeeting, and probably others
> 	- could contain GStreamer-based apps like rhythmbox and have a
>           multimedia edge
> 
> Powertools
> 	- a newly created platform for cool geek toys.
> 
> 
> Now, where would GStreamer fit in ?
> 
> For now, we probably shouldn't try to get it in the Development Platform.
> First of all, there's no real need, second, the requirements are a lot more
> stringent and while we believe we may be able to fulfill them, it's probably
> better not to stifle our development just because we are going into Gnome.
> More likely than not, we will still uncover issues for which we have
> better solutions when people really test the library more.
> (This is not necessarily always bad ;) xine is just completing a complete
> library API rewrite in the middle of their 0.9.x cycle I've been told)
> 
> Powertools is also not the place for us.  Currently it contains fringe
> hardcore hacker tools and eyecandy ;)
> 
> So that leaves us with two platforms.  Of these, we were originally going
> to go into fifth toe/multimedia.
> 
> Now, originally rhythmbox was going to go into FT(/MM), but for various
> reasons jorn has decided to make rhythmbox more mature before putting it
> into the core desktop.
> 
> The current line of thought is that we are going to go into Desktop Platform,
> where we would be supporting (at the very least) iain's gnome media package.
> 
> The choice of if GStreamer would go into the Desktop Platform has to be made
> in (roughly) the next two days.  Feature freeze and module list is 
> scheduled
> for Thursday 31st (which probably is Wednesday since it's on Oceanic time ;))
> 
> It depends on the applications using GStreamer that are proposed for the module
> list.
> 
> So, here's a shortlist of possibilities that could be made ready for inclusion
> in Gnome 2.2 :
> 
> 1) gnome-media
>   contains the sound recorder which iain has moved to GStreamer and
>   rumour has it he is happy with the transition.  "no more sox" is a good
>   reason too.
> 2) gst-player
>   works with gstreamer, of course.  It is less HIG-compliant than totem
>   and that is mostly a UI problem, but one that needs fixing on a pretty
>   short notice.  the 31st is a module/feature freeze, so that still leaves
>   us time to fix up the UI.
> 3) totem
>   currently uses xine, but HIG-compliant.  Totem could be ported to use
>   GStreamer, Bastien is all for it (but is not going to do it himself).
>   I had offered to help out in that area, but I'm not sure I could get
>   that done on such short notice with my still limited video skills ;)
>   Someone here should outline the work that needs to be done to make this
>   happen - from what I can see it shouldn't have to be too hard to support
>   at least a first subset of what xine provides - ie, basic movie playback
>   using libgstplay
> 4) a video thumbnailing application
>   nautilus 2.2 features pluggable thumbnailing, which basically launches
>   an app for each mime type it encounters.  So, for example, the mime type
>   video/avi would use the thumbnailer "gst-snapshot %i %o" which would
>   create a thumbnail png of the input video.
>   This thumbnailing app is stupendously easy to write since basically
>   you can pretty much do that using a gst-launch line with snapshot at this
>   point.
>   We have discussed changing snapshot's strategy because it depends on two
>   libs instead of one.  We might investigate this a little so we can solve
>   this properly, but it's easy to make this app happen in any case. 
> 5) nautilus-media
>   there IS a music view in nautilus currently, but
>   - it only does mp3
>   - it contains a hacked-up version of (I believe) mpg123
>   - Red Hat doesn't even ship it due to licensing issues
>   - it is album-centered; it shows the dir, tries to pretend the dir is
>     an album, and even has room for display of an album cover and so on.
>     A bit overkill in a file browser perhaps.
>   I started learning some nautilus and wrote an audio view for nautilus that
>   basically works, but needs some improving UI-wise.  It'll focus on
>   providing features that allow you to easily browse and scan through a bunch
>   of songs.
>   I intend to make roughly the same view for video as well, but with a
>   fixed-size small video window in the view somewhere.
> 6) nautilus property pages
>   James Willcox has created a system to have pluggable property pages as well
>   in nautilus.  I'm currently focussing on getting the necessary stuff in
>   place to have a streaminfo library that would fetch metadata and caps
>   on files for display in properties and so on.
>   My hope is to make this happen so that the property pages use these
>   functions.  One of the planned features for this lib is to implement
>   a streaminfo cache on-disk similar to the nautilus thumbnail cache.
>   Wim convinced me that there is no problem in having GStreamer doing efficient
>   handling of metadata and stream info, and after implementing it in vorbisfile
>   I have to agree with him.
> 7) nautilus mouse-over play
>   Alex Graveley from Ximian has worked on a nautilus hack that basically
>   "drew" a video widget over the thumbnail of a video in the icon view
>   in nautilus, and after 1 sec of mouseover the video itself would play
>   "in the thumbnail".  That's a pretty cool hack ;) I hope to somehow
>   get that functionality in nautilus somehow.  We've discussed it and
>   it's actually in theory not that hard to do.  We have to figure out
>   though how to provide this without making nautilus depend on GStreamer
>   at compile-time for now.  James Willcox seems to be willing to
>   see how a mechanism could be provided for nautilus to allow the "thumbnail
>   widget" to be replaced with a video widget.  If we can pull this off,
>   this will be a kick-ass eye candy feature.
> 8) rhythmbox
>   is currently focusing on stability and some extra features and probably
>   is not going to propose itself for inclusion anywhere until the groups
>   have been implemented.  I hope they can get it to work as soon as possible,
>   because rhythmbox rocks ;) but it's already been decided not to propose
>   rhythmbox for the module list this week.  It might make a fifth toe
>   or multimedia release.
> 
> So, that's a pretty good list.  Not all of them will make it, I think,
> and I have to check up on when everything needs to be ready.  The upcoming
> freeze this week is a "feature and module decision" freeze.  It means that
> we should have a good idea of the features we want to provide in gnome 2.2,
> and make sure we don't bite off more than we can chew.
> 
> So currently, what seems likely to me to get done is :
> 
> - (1) goes into desktop platform
> - (2) gets proposed with commitment on our side to make it HIG-compliant
>       (which means somebody has to READ it in the first place here - come on,
>       don't be scared).  We will get help from usability guys (Calum, 
>       Nils, etc...)
>       This probably depends on Steve Baker's ability and willingness to
>       make the changes and someone to assist him in translating the HIG to
>       gst-player.  Someone should also just ask the usability guys to give
>       us a UI review so we can start somewhere.
> - (3) seems like a more long-term plan, especially since I am currently
>       focussed on the other stuff
> - (4) should be easy to write, and KConger (on irc) has done this over the 
>       weekend.  I'll prod him to see how it's doing.  So I consider this
>       "going in" as well, possibly as a part of nautilus-media.
> - (5) the audio view part of nautilus can be made ready in relatively short
>       time.  The biggest things to get right here are
>       - writing a static autoplugger that actually knows when it can't play
>         files
>       - getting the streaminfo system basics in place so that it can
>         do queries on the audio types we currently want to support
>         (which I'm probably going to limit at this point to ogg, mp3, wav and
>         flac)
> - (6) I suggested to James to make these use the new streaminfo system and add
>       these pages to nautilus-media, and he seemed to think that was a good
>       idea, so if the streaminfo works, then these pages can also go in.
> - (7) depends on James' ability to a) provide the mechanism and b) convince
>       Alex Larsson and Dave Camp of the sensibility of it.
>       I don't want to push James into doing it, it depends on his availability.
>       I do hope he finds the time to help us out there though.
> - (8) is not going in at this point and will need to be reconsidered in the
>       future.  I hope to move the metadata handling in monkey-media over to
>       GStreamer so that monkey-media itself doesn't need to depend on any
>       decoding library and is thus fully pluggable.
> 
> 
> Now, for the bad news.  What does this mean we should do in GStreamer ?
> 
> We currently have three major problems that need thought.
> 1) We cannot fully trust our current pthread schedulers.  We have seen
>    mysterious crashes on reuse of pthreads, and the glibc people confirmed
>    that you're not allowed to reuse the stack even after the thread using
>    it has been joined.  For all uses we have had up to now, this isn't a big
>    problem, but it seems that in the nautilus music view the code I currently
>    use starts getting major problems when playing the 8th track.
>    Having stack sizes of 2MB be wasted for each thread that gets created is
>    thus probably not a good idea if you cannot reuse those 2MB in the same
>    process ;)
> 
>    ways to fix this :
>    - the optimal scheduler, which Wim claims is almost ready
>    - move the omega scheduler to use GThreads.  David Schleef says this can
>      be done easily, he just needs to change some API in the core for that to
>      happen.
>    - reuse pthreads instead of joining them.  This would make our use of
>      pthreads more efficient and avoids crashes on stack reuse.
>    - make stack space per thread smaller - this would imply making each
>      cothread stack space a bit smaller, or allowing for less elements in one
>      thread.  Both will probably work (the 2MB is largely arbitrary) but
>      are workarounds, not fixes
> 
> 2) Our most-used autoplugger, spider, has MAJOR issues.  Among these are
>    - it locks up when given a file it can't play.  Try playing a text file
>      in gst-player.
>    - sometimes it outright crashes when doing a recursive function nesting
>      because it can't figure out it can't play the file
>    - it is currently unmaintained and Company has been around again but
>      has not himself stated any willingness to work on it after careful
>      prodding on our part ;)
> 
>    ways to fix this :
>    - somebody bites the bullet and investigates the code.  All that needs to be
>      added is an option to spider that tells it how many times it needs to
>      loop the typefind, and set it to default 1.  After that it should
>      send an element error.  Basically ;)
>    - a static autoplugger; it would use typefind to determine the file type,
>      then replace itself with a static pipeline a la gst-launch-ext.
>      Currently the most feasible option.  Would even be sensible to write
>      a fixed audio-only autoplugger.
> 
> 3) being in gnome means having to run on all of the archs and platforms
>    gnome supports.  I have doubts that out current thread code will work
>    on every platform.  Jacob Berkman from Ximian has uncovered issues on
>    mac osx, and I doubt it will reliably work on Sun.
>    We need access to machines to test somehow and someone needs to set us
>    up with that.
>    Also, our code will have to become more portable, define out the
>    arch-specific bits (preferably in one place and not all over the code),
>    and generally be more robust.
>    Sun is rumoured to be setting up a machine outside their firewall for
>    testing, so we should be able to get access to that.
>    We need a complete list of archs gnome supports and find the right 
>    testers for it.
>    I am going over our current set of media and writing up a simple script
>    to run these media files through gstreamer so we can get usable 
>    feedback on how well we do.
> 
> 4) We want to maintain some form of ABI stability in our library.  Wim says
>    he can get this done in about a week, which means it would be done
>    by the time the second round of tarballs needs to go in.  Jeff Waugh
>    is ok with that.
> 
> 5) We need to do and/or figure out the following things :
>    - proper library versioning from now on
>    - define where all of our headers will get installed
>      currently, it's (prefix)/include/(module)-(version)/gst
>      My current idea is to bump us up to 0.5.0, and put ALL headers in
>      include/gstreamer-0.5 for the whole of the 0.5 devel tree used by gnome
>      2.1.x.
>      We use this tree to work up to a "stable" 0.6 tree for gnome 2.2
>    - it would also mean that plugin libraries also put their headers in
>      gstreamer-0.5 instead of gst-plugins-0.5 like now.
>    - the library base name would also be libgstreamer-0.5.so
>      Basically, we either drop the micro number or drop the whole "version
>      in the name" scheme
>    - I will check these changes with experienced hackers who've seen this
>      before like owen, hp, jody and others.
> 
> 6) We'll be doing a prerelease today for a version to be included this 31st.
>    We need this for iain's wavenc plugin.  Basically, I could also do
>    a separate tarball with the plugin and throw it in with the 0.4.1 release,
>    but it'd be good to have a new release.  It will depend on some of the
>    features that have been added, and Uraeus is of course going to help out ;)
> 
> So, a big set of possible changes.  Exciting times are ahead, but PLEASE
> comment on this mail because it WILL involve some changes and we need to
> make sure we understand what this will involve.
> 
> On the plus side, we will be providing a cool desktop with great features
> and we will put the wonder that is GStreamer to the test ;)
> 
> It's also good to mention that this is a major addition to Gnome 2.2, will 
> likely be a controversial issue, will get lots of discussion.  We might 
> not make Gnome 2.2 desktop platform, but that's no reason to be 
> dissapointed if we don't.  We still have the Fifth Toe release which we 
> can go in, and we can use this opportunity to check if we can make the 
> proper commitments and put GStreamer to the test.
> 
> Steady on,
> Thomas
> 
> -- 
> 
> The Dave/Dina Project : future TV today ! - http://davedina.apestaart.org/
> <-*- thomas (dot) apestaart (dot) org -*->
> I'd like to tell you how I feel
> I'll probably keep it till a Saturday
> <-*- thomas  (at) apestaart (dot) org -*->
> URGent, the best radio on the Internet - 24/7 ! - http://urgent.rug.ac.be/
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel


-- 
Julien MOUTTE - jmoutte at electronic-group.com
C.T.O.
_________________________________________________________

ELECTRONIC GROUP INTERACTIVE - www.electronic-group.com
World Trade Center, Moll de BARCELONA
Edificio Norte 4 Planta
08039 BARCELONA SPAIN
Tel : +34 93600 23 23 Fax : +34 93600 23 10
_________________________________________________________