[gst-embedded] How to use DSP in gstreamer
Zhao Liang-E3423C
E3423C at motorola.com
Tue Mar 4 00:40:47 PST 2008
On Tue, 2008-03-04 at 10:10 +0800, ext Zhang Yanlong-PBVM47 wrote:
> Yes, I think it is not a simple things if integrate gstreamer with
> other hardware, e.g dsp.
> Maybe implement a set of API in dsp side and provide it to gstreamer
> is a good solution.
>
> BRs
> Zhang
>
> -----Original Message-----
> From: gstreamer-embedded-bounces at lists.sourceforge.net
> [mailto:gstreamer-embedded-bounces at lists.sourceforge.net] On Behalf Of
> Zhao Liang-E3423C
> Sent: Tuesday, March 04, 2008 10:05 AM
> To: Benoit Fouet
> Cc: gstreamer-embedded at lists.sourceforge.net
> Subject: Re: [gst-embedded] How to use DSP in gstreamer
>
>
> Hi,
>
> Zhao Liang-E3423C wrote:
> > Zhao Liang-E3423C wrote:
> >
> >> Hi all,
> >>
> >> On embedded device, DSP is used widely, many DSPs have different
> >> features, for example:
> >>
> >> 1. decoder
> >> DSP is just a hardware decoder
> >> 2. decoder + sink
> >> DSP is a decoder plus sink, it can directly accept encoded data,
> >> and then decode it and render pcm data into audio device directly.
> >>
> >
> > creating a sink with right caps should be enough for this one...
> >
> > Zhao Liang: By my experience, it is not simple like just adding a
> > new
>
> > caps.
> > Just a example, how to handle preroll? Generally, dsp
> need to do
> > initialization when start to work, so what is the time to do this?
> > another issue is seek, how does DSP handle seek?
> >
> >
>
> >this is generally the demuxer/parser's job to handle seek, do you
> >have
> examples where it is up to the decoder to do it ?
>
> As it is a sink for DSP although it includes decoder functions, sink
> would handle preroll, in current preroll mechanism, it only allows
> sink do once preroll virtual function, all other buffers are pushed
> into queue, but some hardwares need more buffers to do initialization.
>
> Another issue is if the DSP is audio decoder + sink, how to handle
> clock line increasing? Currently, we can use gstaudiosink to do it,
> gstringbuffer can handle all clock line increasing, but some DSP can
> not do it if no data is fed into, so the clock calculation is really a
> issue. If no mechanism, rtsp streaming application will not handle
> buffer delay or lost.
>
>
> >> 3. A/V sync
> >> DSP can do A/V sync internally or not.
> >>
> >
> > then you can ask basesink not to synchronize flows.
> >
> > Zhao Liang: If DSP is just a video decoder and sink, how does DSP
> > sync
>
> > with audio sink? maybe current basesink considers more software sink
> > than hardware sink.
> >
> >
>
> >I was talking about your hardware handling A/V synchronization, not
> >the
> opposite.
> >buffers that are handled by sink are timestamped (by demuxers for
> >instance) and are rightly handled by basesink when synchronization
> >has
> to be done in framework, and not in HW.
>
> I understood your meaning, A/V sync is really done by basesink, but
> some DSPs also have internal buffers to store buffers synced. and
> these synced buffers are encoded buffers, not raw buffers.
> There are 4 combinations:
> 1. DSP videosink v.s software audiosink 2. DSP videosink v.s DSP
> audiosink 3. software videosink v.s DSP audiosink 4. software
> videosink v.s software audiosink
> When there's a decoder+sink, one approach is to map the hw clock to
the GStreamer clock, that way synchronization is still handled by
GStreamer transparently.
> However, I would like to see some numbers regarding the performance
improvement when doing such thing (decoder+sink), I wonder if it's worth
the trouble.
It is a good ideal to wrap hw clock to gstclock. For decoder+sink,
sometimes it is restricted by HW, I saw nokia N800 using DSP gateway as
audio sink, by my understanding, is it a decoder+sink? If you can share
some experience on it, it will be appreciated.
> Best regards.
--
> Felipe Contreras
------------------------------------------------------------------------
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Gstreamer-embedded mailing list
Gstreamer-embedded at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstreamer-embedded
More information about the Gstreamer-embedded
mailing list