[Gstreamer-openmax] Re: [Gstreamer-openmax] 回复： [Gstreamer-openmax] Discussion on the hardware accelerator solution in GstOpenMAX project.
wangfei
bjwf2000 at gmail.com
Fri Aug 1 02:21:25 PDT 2008
Hi All,
I'm working on a project based on several different hardware platforms. TI
34xx is one of my option. I'm very interest on topic 2: how to integration
GStreamer with OMX on GPU/DSP hardware solutions. My thoughts are as
follows:
1. Supper source/sink element or tunneling solution is something depends
how OMX was implemented on a specific hardware. OMX IL spec didn't specify
how to partition different processing functions into OMX IL components
(Refer to openmax_il_spec_1_1_1.pdf page 21, Figure 2-1). For example, in TI
OMAP3, a C64x DSP was equiped, a super OMX component was provided by TI for
audio streaming. http://omapzoom.org/gf/project/openmax/wiki The super OMX
component includes audio decode, PP, rendering functions which were
processed by C64x DSP. In such case, seems a supper GstOpenMax sink element
is the only choice? In another OpenMAX IL implementation, decoder, render
provided by separate OMX complents, in such case, maybe separate GstOpenMAX
elements support OMX tunneling is better.
2. I guess GstOpenMAX didn't want to give a recommendation or specification
for OMX IL provider on how to paration there OMX IL components. Mybe I'm not
right?
Thanks & Regards
Wang Fei.
2008/8/1 Dale Tian <daletian at yahoo.cn>
> Victor:
> Your analysis sounds great for playback use case. But for camera use
> case, we need similiar elements, supersrc and viewfinder, and also a
> convertor element for format translation. If omx provides demux and mux
> support, gstreamer layer will be very thin. So I prefer the tunneling
> solution, gsteamer layer will focus on use case organization, omx side will
> focus on efficient data processing and hardware abstraction.
>
> Best regards,
> Dale
>
> ----- 原始邮件 ----
> 发件人： Victor Manuel Jáquez Leal <ceyusa at gmail.com>
> 收件人： gstreamer-openmax at lists.sourceforge.net
> 已发送： 2008/8/1(周五), 上午6:38:27
> 主题： Re: [Gstreamer-openmax] Discussion on the hardware accelerator solution
> in GstOpenMAX project.
>
>
> Even though I haven't work in these things for a while, I used to and
> have thoughts about them.
>
> I haven't read the NXP tunneling implementation in gstopenmax but once
> we implemented something related: when the omx gst element is linked
> with another omx gst element a tunnel is setup, but as no buffers
> traverse the pipeline, because the buffer communication is done
> beneath the omx layer, we had to push ghost buffer (empty buffers with
> calculated metadata), and those ghost buffers simulated the gst a/v
> sync, nevertheless the real a/v sync was done by omx.
>
> That solution is not sound: might work in some cases but we couldn't
> assure it for every case.
>
> We dismissed the super sinks since the beginning of the development,
> because, as you mentioned, is not a flexible solution.
>
> But we have a trade of: the first solution is not concordant with the
> gstreamer philosophy, and the second is not concordant with the omx
> philosophy, because a semantic overlapping, as in the buffer
> communication assumptions, the state management among the components,
> etc.
>
> Nowadays I'm more convinced that a supersink could be the best
> solution to integrate gst and omx in the A/V playback use case:
>
> 1. it's easy to build and setup the omx pipelines given the caps
> 2. it's easy to control the sync
> 3. it's easy add gst interfaces as volume, contrast, et all
> 4. it's easy to manage the state among the omx components
> 5. no dirty hacks as ghostbuffers
> 6. afaik the supersink elements can be autoplugged by playbin2
>
> I admit the supersinks break the flexibility offered by gst, but as
> far as i can foresee, is the straight strategy to obtain the
> performance promised by omx in his interop profile.
>
> Maybe one day the interop profile won't be necessary when the pBuffer
> in the bufferheader leave its readonly property...
>
> Víctor Manuel Jáquez Leal
>
> On Wed, Jul 30, 2008 at 9:00 AM, Ling Shi <shil66 at gmail.com> wrote:
> > Hi, all
> > I'm in a research project to port gstreamer into embedded system. Now, we
> > encounter the issue on how to integrate hardware accelerators (DSP/GPU)
> into
> > gst. After evaluating different solution, we think GstOpenMAX project may
> be
> > the best one for us, because
> >
> > 1) OpenMAX is an industry standard
> > 2) more and more DSP/GPU vendor support OpenMAX
> >
> > But, I still have several questions on this project.
> > 1. Does Nokia N8xx serials use GstOpenMAX?
> > I know Nokia engineers lead this project. I also know N8xx serials use
> > gstreamer as default playback engine, and it uses TI OMAP 2420, which has
> > DSP. Can anyone tell me if N8xx use GstOpenMAX? If no, does N8xx plan to
> use
> > it in the future?
> >
> > 2. What's GstOpenMAX plan to support DSP/GPU in the future?
> > I review several plugins in GstOpenMAX and find current design can only
> > support none-tunnel communication. It's not the best solution in
> hardware,
> > because of bad performance. So, we plan to improve it by adding tunneled
> or
> > proprietary communication. Do you have such plan? If yes, can we involve
> in
> > design?
> >
> > In addition, most accelerators work as a decoder and a render. It means,
> the
> > encoded data sent to it will directly be decoded and rendered, and will
> not
> > be retrieved back again. How the gst or omx organize its pipeline in this
> > situation? We are evaluating two solutions.
> >
> > ===Solution 1===
> > We can design a super omx sink component to cover decoder and render.
> This
> > is the solution is used by N8xx.
> > src ! demux ! sink
> > |
> > super omx sink
> > |
> > +--------------------------------------------+
> > | hardware accelerator |
> > +--------------------------------------------+
> >
> > ===Solution 2===
> > We can separate omx decoder, omx post processer, and omx sink elements.
> We
> > enhance decoder, post processor, and sink plugin in GstOpenMAX. If
> > GstOpenMAX plugin found its neighborhood are GstOpenMAX plugin, it will
> try
> > to establish tunneled communication or proprietary communication firstly.
> It
> > means, although we have 3 OMX plugin in gst pipeline, there is no data in
> > gst pad and omx port. The last two gst/omx plugin only provide control
> > function, but not support process data flow. Of cause, If the connection
> is
> > failed, it will use none-tunnel communication.
> >
> > src ! demux ! decoder ! post processor ! sink
> > | | |
> > omx dec omx pp omx sink
> > | | |
> > +--------------------------------------------+
> > | hardware accelerator |
> > +--------------------------------------------+
> >
> > It seems solution 2 is more flexible. How about your suggestion on the 2
> > solutions? Which one is feasible? Do you have other solutions?
> >
> > Thank you very much.
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> > Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the
> world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Gstreamer-openmax mailing list
> > Gstreamer-openmax at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax
> >
> >
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Gstreamer-openmax mailing list
> Gstreamer-openmax at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax
>
> ------------------------------
> 雅虎邮箱，您的终生邮箱！ <http://cn.mail.yahoo.com/>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Gstreamer-openmax mailing list
> Gstreamer-openmax at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-openmax/attachments/20080801/1fa009ee/attachment.htm>
More information about the Gstreamer-openmax
mailing list