[gst-devel] caps mime types

[Ronald Bultje]

Hi all,

gstreamer currently faces a small problem - caps mime type confusion.
This means that different plugins use different names for the same mime
types. Examples of obvious confusion include image/jpeg with video/jpeg,
video/jpeg with video/mjpeg or video/mjpg and so on. Fortunately nobody
has made such mistakes yet ;-). A problem that we *have* encountered is
that several parsing plugins use themselves as mimetype. For example,
avidemux gives out caps of the mime type "video/avi", whichever format
the video is. Wim has created 'avitypes' (a plugin) for that, which
translates the "avi mime type" to the "gstreamer mime type". A seperate
plugin for such things is a bit too much, though, imho, there must be a
better solution. A second problem here is that every demuxer and muxer
would require its own XXXtypes plugin. Also, autoplugging becomes much
harder if we make plugins for each and every occasion. It's much easier
to solve this in a different way.

So, how can we solve this? There's several ways:
1) we could change avimux/avidemux (these are just the two I mention
here, but I basically mean 'any muxer/demuxer') to give out "gstreamer
like mime types" only. A problem here is that every time that avifile
(on which avimux/avidemux is based) supports a new format, we need to
hack avimux/avidemux. Besides, we also need to change the windex/winenc
plugins, since they are (the only) plugins that accept caps of type
"video/avi".
2) make all plugins in the world accept caps of both type (as example,
I'll take jpegdec) "video/jpeg" and "video/avi". A problem here is that
for every new muxer we introduce, we need to change *all* the plugins to
accept caps of the type "video/<name_of_mux_format>".
3) keep the current state where we need <name_of_mux_format>types
plugins for every muxer/demuxer.
4) integrate the <name_of_muxer_format>types behaviour into the actual
muxer/demuxer. This one has my preference, currently.

(4) would - in practice - mean that avidemux (as an example) is able to
give out "video/avi" caps, and besides that, for some "known" formats,
it can also give out caps of the type "video/raw" (RGB/I420/YUY2),
"video/jpeg" (MJPEG), "video/dv" (DV), etc. This would mean *some* work
on the side of the demuxers, but the consequence would be that we don't
need to change the muxer/demuxer plugins every time avifile supports a
new format, we don't need to change windec/winenc to still be able to
interact with the demuxer/muxer and it's then possible to interact with
plugins that expect a gstreamer caps mime type ("video/dv" rather than
"video/avi"). Of course, the same story goes for something like
quicktime and other muxers we might support in the future.

Opinions/thoughts/comments/flames are welcome, I hope we can solve this
asap.

Ronald

-- 
-   .-.
-   /V\    | Ronald Bultje <rbultje at ronald.bitfreak.net>
-  // \\   | Running: Linux 2.4.17-XFS and OpenBSD 3.0
- /(   )\  | http://ronald.bitfreak.net/
-  ^^-^^