[gst-devel] [gst-embedded] can playbin handle the sink withdecoding functions?

[Donny Viszneki]

On Fri, Feb 20, 2009 at 9:41 PM, Liang Zhao <liangzhao.bit at gmail.com> wrote:
> Yes, I am working on the device with HW acceleration. My concern is how to
> be compatible with playbin or playbin2, if the sinks are not standard as the
> available sinks in community, such as the sink with decoding functions.
>
> I think this may be a trend in embedded devices as these devices don't have
> powerful cpu but strong DSP or GPU, and with HW limits, some data such as
> audio and video can not be picked up after decoding.

Sounds very interesting. What device are you working with?

I understand what you're saying about how the media "cannot be picked
up after decoding." I was originally thinking that the situation could
be solved well with caps. My hope was that for each type of media you
have hw acceleration for, you could create a hw decoder element and a
hw sink element. Caps could be used to make sure that the hw decoder
element's src could only be connected to either the hw sink element,
or to a hw-land-to-sw-land "download" element, which would basically
retrieve the decoded media so that it is available to most other
elements. But your case looks like that much complexity is not
desireable at all, as you seem completely unable to retrieve the
decoded media.

One example situation I would like to compare your situation to is
someone's project with the SNES APU:

http://www.raphnet.net/electronique/snes_apu/snes_apu_en.php

The APU for all intents and purposes may not even *have* a digital
representation of the decoded audio. For all we know, the APU may
contain digitally controlled analog synthesizer elements (on the SNES
I don't believe this is actually the case as it's all DSP, but one can
dream.) This is another situation where you would not want a decoder
nor an "output sink" (hey GStreamer people: what is the appropriate
GStreamer vernacular for the final output sinks that actually allow
you to hear or see the result of the pipeline?)

In this case, my next thoughts have already been posted by someone else:

On Fri, Feb 20, 2009 at 10:37 PM, Dan Taylor <dtaylor at startrac.com> wrote:
> Since the function of playbin is to decide what decoders to use, and
> this
> platform already IS the decoder, why isn't this element simply replacing
> playbin in the pipeline?  I don't understand the logic of using a
> decoder-
> match-and-connect element to front-end a decoder, unless there's an
> option
> of additional decoders, and this hardware is just one of the many.
>
> source->[demux->]queue->hw_decoder->audio_sink

Why do you want to use a playbin at all? One reason occurs to me: to
be compatible with applications which use playbin?

If you do want playbin compatibility, I believe that my suggested
method above can be used, with the exception that there will be no
hw-land-to-sw-land "download" element to pull the decoded media back
into main RAM from the hw decoder. To get it to work with playbin
though, I think you'd still need a "dummy output sink" to satisfy
playbin's preconceptions about what a playbin pipeline is supposed to
look like

I hope all this information is accurate, I would appreciate
corrections from more experienced GStreamer people.

-- 
http://codebad.com/