[gst-devel] type system... (comment please)
Wim Taymans
wim.taymans at chello.be
Sun Nov 26 17:43:31 CET 2000
These are some ramblings about ideas for a new type system.
No attempt for an implementation has been done yet :-)
Any comment that can make the system less complicated is
appreciated.
1. Introduction
---------------
The type system is used to attach meaning to the bytes in a GstBuffer.
A plugin can decide to add metadata to the GstBuffer, this metadata
will carry an associated typeid.
Types are also used by the plugins to expose the type of their pads to
the type system.
Types are essential for autoplugging.
We will explain the inner workings of the type system in this document, as
well as the API used in the plugins.
2. Type properties
------------------
a) identification of the type
Types are identified by one or more MIME types.
b) detection of types
The type of any given GstBuffer can be detected using
- a typefind function: a function that will inspect the bytes
in the buffer and return a gboolean indicating the
buffer is of the given type.
- a template for the bytes/bits in the data that must be
satisfied in order for the GstBuffer to be of the given
type.
- other properties that act more like a hint like:
the extension of the source filename.
the URI of the source.
3. Type registration
--------------------
a) the core libraries will create to types:
video/raw image/raw
audio/raw
b) all other types will be provided by the plugins
before a type can become available to other plugins, the plugin
has to register the type.
The system will keep a directed graph of the types and the plugins
that operate on them.
example:
video/mpeg
!
mpeg1parse
/ \
video/mpeg1 audio/mp3 -----\
! ! \ !
mpeg_play mpeg2dec mpg123 xing ...
\ / \ /
video/raw audio/raw-----\
! ! ! !
videosink SDLsink audiosink alsasink ...
The system can find the needed plugins to convert video/mpeg to
audio/raw using this graph.
4. type equivalence
-------------------
some types can have the same meaning for example:
audio/mp3 and audio/mpeg
or
video/raw and image/raw
a plugin that exposes its output type as audio/mpeg and another plugins
with input type audio/mp3 can be connected. The system has to know about
the equivalence of both types, even it they have a different mime type.
5. type hierarchy
-----------------
some plugins can ouput a specific subset of an allready existing type.
example:
mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
with mime type: audio/mp3-frame
mpg123 only accepts audio/mp3-frame but not audio/mp3.
another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
audio/mp3-frame)
it must be possible to connect both libmpg123 and mpg123 to the mp3parse
element.
it must not be possible to connect mpg123 to an element that outputs only
audio/mp3 but not audio/mp3-frame.
We say that audio/mp3-frame is a more specific subset of type audio/mp3.
we can create a type hierarchy:
audio/mp3
/ \
audio/mp3-frame audio/mp3-layer12
/ \
audio/mp3-layer1 audio/mp3-layer2
6. multi-type pads
------------------
certain plugins might accept multiple non equivalent types in one of their
input pads. Consequently a plugin might output non equivalent types in
its output pad.
It must therefore be possible to specify multiple types for a pad.
example:
mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
we show the type hierarchy of the video/mpeg as follows:
video/mpeg
/ \
video/mpeg1 video/mpeg2 ---------------\
/ \ / \ !
mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
the mpegdemux element might specify the type of the input pad as
one of video/mpeg1-system and video/mpeg2-ts
7. definition of types
----------------------
A plugin will provide the following information to the type system:
- a mime type string describing the hierarchy and where the types
they provide are located in that hierarchy.
- typefind functions for some of the types.
- extensions for some of the types
We will propose a syntax to define the type hierarchy
a) equivalent types :
separated with a | sign
( audio/mp3 | audio/mpeg )
b) type hierarchy :
in braces:
( audio/mp3 ( audio/mp3-frame))
c) multi-type pads
( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
the pad will have type mpegdemux which is the parent for
type video/mpeg1-system or video/mpeg2-ps
mpegdemux
/ \
video/mpeg1-system video/mpeg2-ps
once the type hierarchy has been registered, the typeid of each
element can be obtained with:
guint16 gst_type_find_by_mime (gchar *mime)
extra typefind functions and/or extensions can be added to the
typeid afterwards.
8. type matching
----------------
The more specific typefind functions, the functions associated with
types in the leaf nodes, are tried first.
when a specific type has been found ex. video/mpeg1-system elements
that can handle this type or one of its parents are selected:
mpegdemux: mpeg1parse:
supermpegdecoder:
video/mpeg video/mpeg
video/mpeg
! !
mpegdemux video/mpeg1-system
!
video/mpeg1-system
In the above case, also the super mpeg element is selected because it
stated
that it can handle all sorts of video/mpeg data.
example 2:
suppose the typefind functions finds an mp3 stream, fillowing elments
are selected:
libmpg123 mp3parse:
audio/mp3 audio/mp3
mpg123 is not selected because its pad type is too specific
(audio/mp3-frame):
mpg123
audio/mp3
!
audio/mp3-frame
if the typefind would find a mp3-frame type, all three objects would be
selected.
8. consideration
----------------
It is clear that clear indications have to be given to the type hierarchy,
especially for the top nodes.
The more specific an element is in its mime type specification, the more
flexible
and extendible the plugin system can be.
Wim
--
If pregnancy were a book they would cut the last two chapters.
-- Nora Ephron, "Heartburn"
More information about the gstreamer-devel
mailing list