[gst-devel] type system... (comment please)
Erik Walthinsen
omega at cse.ogi.edu
Mon Nov 27 06:40:54 CET 2000
On Sun, 26 Nov 2000, Wim Taymans wrote:
> Any comment that can make the system less complicated is
> appreciated.
Heh, you think I'm going to make this *less* complicated??? That sure
is some good crack you got there ;-)
> 2. Type properties
> ------------------
>
> a) identification of the type
>
> Types are identified by one or more MIME types.
There's a potential problem here: the MIME type is not necessarily unique
or sufficient. Uniqueness is violated by MPEG, since video/mpeg could
mean MPEG 1 or 2, elementary or system/program. audio/raw is not
sufficient to describe the data stream, since it ignores the all the
type-specific options:
Format (aLaw/uLaw, little/big endian)
Depth (8 - 32)
Channels (1 - N)
Channel arrangement (interleaved/concatenated)
Video is even worse.... This seems to be the biggest hole in the type
system (either what we have or what you describe). For audio at least,
we'd want to have the source and sink be able to describe whether they can
deal with various forms of audio, or at least have veto over some
attempted operation.
Some kind of properties system on an instance of a type might work, but
gets really messy when you're talking about trying to match arbitrary type
properties.
> b) detection of types
>
> The type of any given GstBuffer can be detected using
>
> - a typefind function: a function that will inspect the bytes
> in the buffer and return a gboolean indicating the
> buffer is of the given type.
>
> - a template for the bytes/bits in the data that must be
> satisfied in order for the GstBuffer to be of the given
> type.
Sound like we'd want to get GNU file(1) to export a library, except
file(1) outputs random strings. If the magic(4) format could be extended
to consistently output some form of MIME type too, that would be the end
of that. But until we (the community) manage to merge mime.types(4) and
magic(4), as well as settle on MIME types and extension, that's kinda
hard.
> - other properties that act more like a hint like:
> the extension of the source filename.
> the URI of the source.
At least mime.types is easy to parse ;-)
> 3. Type registration
> --------------------
>
> The system will keep a directed graph of the types and the plugins
> that operate on them.
>
> example:
>
> video/mpeg
> !
> mpeg1parse
> / \
> video/mpeg1 audio/mp3 -----\
> ! ! \ !
> mpeg_play mpeg2dec mpg123 xing ...
> \ / \ /
> video/raw audio/raw-----\
> ! ! ! !
> videosink SDLsink audiosink alsasink ...
What we have now should work fine, it just keeps a list of the elements
that can handle a given type for input and ouptut. It's just a different
form of the above graph, and I think it's easier to use. Following the
graph around (if you meant actually having a graph data structure) seems
wasteful. The lists themselves sorta form a graph anyway, since the
elements are pointers to elements that have the type ids, from which you
can extract the lists for that type.
> 4. type equivalence
> -------------------
>
> some types can have the same meaning for example:
>
> audio/mp3 and audio/mpeg
>
> or
>
> video/raw and image/raw
>
>
> a plugin that exposes its output type as audio/mpeg and another plugins
> with input type audio/mp3 can be connected. The system has to know about
> the equivalence of both types, even it they have a different mime type.
I think there's a tiny bit of this set up already, in that a given type's
mime string can simply hold any number of MIME types. However, I think it
should be more explicit and more flexible simultaneously. This makes it
much more complex, but the type system at least shouldn't be a performance
issue.
The idea would be to have a distinct type allocated to every unique MIME
type, and have a list for each of which ones are equivalent. This might
even solve the type hierarchy system, since you could have a one-sided
relationship of equivalence between audio/mp3 and audio/mp3-frame.
mp3-frame is equivalent to mp3, but not vice versa. The lists form the
appropriate graph on their own.
> 5. type hierarchy
> -----------------
>
> some plugins can ouput a specific subset of an allready existing type.
>
> example:
>
> mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
> with mime type: audio/mp3-frame
>
> mpg123 only accepts audio/mp3-frame but not audio/mp3.
>
> another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
> audio/mp3-frame)
>
> it must be possible to connect both libmpg123 and mpg123 to the mp3parse
> element.
> it must not be possible to connect mpg123 to an element that outputs only
> audio/mp3 but not audio/mp3-frame.
>
>
> We say that audio/mp3-frame is a more specific subset of type audio/mp3.
>
> we can create a type hierarchy:
>
> audio/mp3
> / \
> audio/mp3-frame audio/mp3-layer12
> / \
> audio/mp3-layer1 audio/mp3-layer2
I'm starting to get a little hesitant to invent too more MIME types. If
we can work out a properties system, so we can describe the MPEG layer
separately, I think I'd be happier. This makes things significantly more
complex, though, since you might have to maintain an equivalency graph
down at the properities level in addition to the MIME level.
> 6. multi-type pads
> ------------------
>
> certain plugins might accept multiple non equivalent types in one of their
> input pads. Consequently a plugin might output non equivalent types in
> its output pad.
>
> It must therefore be possible to specify multiple types for a pad.
>
> example:
>
> mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
> we show the type hierarchy of the video/mpeg as follows:
>
> video/mpeg
> / \
> video/mpeg1 video/mpeg2 ---------------\
> / \ / \ !
> mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
>
>
> the mpegdemux element might specify the type of the input pad as
> one of video/mpeg1-system and video/mpeg2-ts
Not sure whether you're suggesting there be a list or a graph of supported
types. I'd say just a list, possibly with the option of forcing
non-equivalence for any given one. I.e. if you need to specify
audio/mp3-frame without implying audio/mp3 too.
Also, I'd think this would apply to sources pads as well as sinks. This
does make autoplugging harder, but so does having a list of possible sink
types. It just squares the problem, which is tractable.
> 7. definition of types
> ----------------------
>
> A plugin will provide the following information to the type system:
>
> - a mime type string describing the hierarchy and where the types
> they provide are located in that hierarchy.
>
> - typefind functions for some of the types.
>
> - extensions for some of the types
>
> We will propose a syntax to define the type hierarchy
>
> a) equivalent types :
>
> separated with a | sign
>
> ( audio/mp3 | audio/mpeg )
>
> b) type hierarchy :
>
> in braces:
>
> ( audio/mp3 ( audio/mp3-frame))
>
> c) multi-type pads
>
> ( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
>
> the pad will have type mpegdemux which is the parent for
> type video/mpeg1-system or video/mpeg2-ps
>
> mpegdemux
> / \
> video/mpeg1-system video/mpeg2-ps
I'd rather have a programmatic interface instead of a textual format. I'd
hate to have to programatically construct one of these strings within the
plugin. And any property system is going to wreak havoc with a textual
setup.
Erik Walthinsen <omega at cse.ogi.edu> - Staff Programmer @ OGI
Quasar project - http://www.cse.ogi.edu/DISC/projects/quasar/
Video4Linux Two drivers and stuff - http://www.cse.ogi.edu/~omega/v4l2/
__
/ \ SEUL: Simple End-User Linux - http://www.seul.org/
| | M E G A Helping Linux become THE choice
_\ /_ for the home or office user
More information about the gstreamer-devel
mailing list