[gst-embedded] How to use DSP in gstreamer

[Zhao Liang-E3423C]

On Tue, 2008-03-04 at 10:10 +0800, ext Zhang Yanlong-PBVM47 wrote:
> Yes, I think it is not a simple things if integrate gstreamer with 
> other hardware, e.g dsp.
> Maybe implement a set of API in dsp side and provide it to gstreamer 
> is a good solution.
> 
> BRs
> Zhang
> 
> -----Original Message-----
> From: gstreamer-embedded-bounces at lists.sourceforge.net
> [mailto:gstreamer-embedded-bounces at lists.sourceforge.net] On Behalf Of

> Zhao Liang-E3423C
> Sent: Tuesday, March 04, 2008 10:05 AM
> To: Benoit Fouet
> Cc: gstreamer-embedded at lists.sourceforge.net
> Subject: Re: [gst-embedded] How to use DSP in gstreamer
> 
> 
> Hi,
> 
> Zhao Liang-E3423C wrote:
> > Zhao Liang-E3423C wrote:
> >   
> >> Hi all,
> >>  
> >> On embedded device, DSP is used widely, many DSPs have different 
> >> features, for example:
> >>  
> >> 1. decoder
> >> DSP is just a hardware decoder
> >> 2. decoder + sink
> >> DSP is a decoder plus sink, it can directly accept encoded data, 
> >> and then decode it and render pcm data into audio device directly.
> >>     
> >
> > creating a sink with right caps should be enough for this one...
> >
> > Zhao Liang:  By my experience, it is not simple like just adding a 
> > new
> 
> > caps.
> > 		Just  a example, how to handle preroll? Generally, dsp
> need to do
> > initialization when start to work, so what is the time to do this?
> > 		another issue is seek, how does DSP handle seek?	
> >
> >   
> 
> >this is generally the demuxer/parser's job to handle seek, do you 
> >have
> examples where it is up to the decoder to do it ?
> 
> As it is a sink for DSP although it includes decoder functions, sink 
> would handle preroll, in current preroll mechanism, it only allows 
> sink do once preroll virtual function, all other buffers are pushed 
> into queue, but some hardwares need more buffers to do initialization.
> 
> Another issue is if the DSP is audio decoder + sink, how to handle 
> clock line increasing? Currently, we can use gstaudiosink to do it, 
> gstringbuffer can handle all clock line increasing, but some DSP can 
> not do it if no data is fed into, so the clock calculation is really a

> issue. If no mechanism, rtsp streaming application will not handle 
> buffer delay or lost.
> 
> 
> >> 3. A/V sync
> >> DSP can do A/V sync internally or not.
> >>     
> >
> > then you can ask basesink not to synchronize flows.
> >
> > Zhao Liang: If DSP is just a video decoder and sink, how does DSP 
> > sync
> 
> > with audio sink? maybe current basesink considers more software sink

> > than hardware sink.
> >
> >   
> 
> >I was talking about your hardware handling A/V synchronization, not 
> >the
> opposite.
> >buffers that are handled by sink are timestamped (by demuxers for
> >instance) and are rightly handled by basesink when synchronization 
> >has
> to be done in framework, and not in HW.
> 
> I understood your meaning, A/V sync is really done by basesink, but 
> some DSPs also have internal buffers to store buffers synced. and 
> these synced buffers are encoded buffers, not raw buffers.
> There are  4 combinations:
> 1. DSP videosink  v.s  software audiosink 2. DSP videosink  v.s  DSP 
> audiosink 3. software videosink  v.s  DSP audiosink 4. software 
> videosink  v.s  software audiosink

> When there's a decoder+sink, one approach is to map the hw clock to
the GStreamer clock, that way synchronization is still handled by
GStreamer transparently.

> However, I would like to see some numbers regarding the performance
improvement when doing such thing (decoder+sink), I wonder if it's worth
the trouble.

It is a good ideal to wrap hw clock to gstclock. For decoder+sink,
sometimes it is restricted by HW, I saw nokia N800 using DSP gateway as
audio sink, by my understanding, is it a decoder+sink? If you can share
some experience on it, it will be appreciated.


> Best regards.

--
> Felipe Contreras


------------------------------------------------------------------------
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Gstreamer-embedded mailing list
Gstreamer-embedded at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstreamer-embedded