[gst-devel] Streaming only text

[Magnus Bergman]

On Sun, 4 May 2003 16:49:37 +0200 (DFT)
Benjamin Otte <in7y118 at public.uni-hamburg.de> wrote:

> On Fri, 2 May 2003, Magnus Bergman wrote:
> 
> > I am in need of a plug-in-based framework for streaming only text. I have the intention of writing every plug-in myself, if needed. Is there any major reason why gstreamer may not be suitable for this kind of application? Some potential reasons I could think of:
> >
> Great that someone finally uses it for something else. From the top of my
> head GStreamer is capable of streaming everything, but could suffer from
> problems when someone has not thought about something.
> 
> > 1) I suppose gstreamer is optimized for streaming a few large files. How will if perform with a lot of small files (let's say a few thousand 4k text files)?
> >
> GStreamer's core is not what people call optimized in any direction
> because most time is spent inside the plugins and they are optimized.
> Example: If you decode an mp3, 80% or more of the time required is spent
> in the mp3 decoding plugin.
> 
> > 2) I understand that streaming audio and video requires timing. But how about just stream as fast as possible (which has to be useful for media conversion too)?
> >
> Timing is something a plugin can request. Most of the plugins have no
> interest in timing. Only start-/endpoint plugins synchronize and only
> some of them. A file reading plugin for example doesn't care, and recoding
> mp3 -> ogg works as fast as possible without any timing going on.
> So it is no problem at all.
> 
> > 3) In some cases it would be useful (for me) to use other char-sets than UTF-8 (it would be a waste of time to convert to UTF-8 and then back again). I read something about gstreamer only supporting UTF-8. The question is, can text buffers have a field telling about the encoding?
> >
> Plugins process data given to them inside GstBuffers. The type of data
> that is sent is negotiated between the different plugins via caps (short
> for capabilities). If you were to write all plugins yourself it should be
> no problem to define capabilities yourself and attach a charset property
> to it.
> I don't know where the idea comes from that we would be bound to UTF-8
> only.

That sounds great. I'll go right ahead learning about how to write plug-ins and use gstreames.

> 
> What interests me now is: What do you need this framework for? What do you
> want to do with it?

Basically to use with a search engine. I'm working for a company called Observer (http://www.observergroup.com) and write plug-ins and such for a search engine based on (closed source) software from Verity (http://www.verity.com). But I have noticed a few design flaws in that sowfware (like with most commercial software you get the feeling it was put together in a haste) and I try to find ways to work around those. So I'll try to replace all the plug-ins related to streaming text with just one plug-in based on gstreamer and then just write plugin for gstremer. I'll be glad to answer more specific questions, but I'm not quite sure of all the details myself yet.