<div dir="ltr">Hi All, I'm working on a project based on several different hardware platforms. TI 34xx is one of my option. I'm very interest on topic 2: how to integration GStreamer with OMX on GPU/DSP hardware solutions. My thoughts are as follows: 1.  Supper source/sink element or tunneling solution is something depends how OMX was implemented on a specific hardware. OMX IL spec didn't specify how to partition different processing functions into OMX IL components (Refer to openmax_il_spec_1_1_1.pdf page 21, Figure 2-1). For example, in TI OMAP3, a C64x DSP was equiped, a super OMX component was provided by TI  for audio streaming. <a href="http://omapzoom.org/gf/project/openmax/wiki/">http://omapzoom.org/gf/project/openmax/wiki</a>   The super OMX component includes audio decode, PP, rendering functions which were processed by C64x DSP. In such case, seems a supper GstOpenMax sink element is the only choice?  In another OpenMAX IL implementation, decoder, render provided by separate OMX complents, in such case, maybe separate GstOpenMAX elements support OMX tunneling is better. 2. I guess GstOpenMAX didn't want to give a recommendation or specification for OMX IL provider on how to paration there OMX IL components. Mybe I'm not right? Thanks & Regards Wang Fei. <div class="gmail_quote">2008/8/1 Dale Tian <<a href="mailto:daletian@yahoo.cn">daletian@yahoo.cn</a>> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"><div>Victor: Your analysis sounds great for playback use case.  But for camera  use case, we need similiar elements, supersrc and viewfinder, and  also a convertor element for format translation.  If omx provides demux and mux support, gstreamer layer will be very thin.  So I prefer the tunneling solution, gsteamer layer will focus on use case organization, omx side will focus on efficient data processing and hardware abstraction. Best regards, Dale </div><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"> <div style="font-family: arial,helvetica,sans-serif; font-size: 10pt;">----- 原始邮件 ---- 发件人： Victor Manuel Jáquez Leal <<a href="mailto:ceyusa@gmail.com" target="_blank">ceyusa@gmail.com</a>> 收件人： <a href="mailto:gstreamer-openmax@lists.sourceforge.net" target="_blank">gstreamer-openmax@lists.sourceforge.net</a> 已发送： 2008/8/1(周五), 上午6:38:27 主题： Re: [Gstreamer-openmax] Discussion on the hardware accelerator solution in GstOpenMAX project.<div> <div></div><div class="Wj3C7c"> Even though I haven't work in these things for a while, I used to and have thoughts about them. I haven't read the NXP tunneling implementation in gstopenmax but once we implemented something related: when the omx gst element is linked with another omx gst element a tunnel is setup, but as no buffers traverse the pipeline, because the buffer communication is done beneath the omx layer, we had to push ghost buffer (empty buffers with calculated metadata), and those ghost buffers simulated the gst a/v sync, nevertheless the real a/v sync was done by omx. That solution is not sound: might work in some cases but we couldn't assure it for every case. We dismissed the super sinks since the beginning of the development, because, as you mentioned, is not a flexible solution. But we have a trade of: the first solution is not concordant with the gstreamer philosophy, and the second is not concordant with the omx philosophy, because a semantic overlapping, as in the buffer communication assumptions, the state management among the components, etc. Nowadays I'm more convinced that a supersink could be the best solution to integrate gst and omx in the A/V playback use case: 1. it's easy to build and setup the omx pipelines given the caps 2. it's easy to control the sync 3. it's easy add gst interfaces as volume, contrast, et all 4. it's easy to manage the state among the omx components 5. no dirty hacks as ghostbuffers 6. afaik the supersink elements can be autoplugged by playbin2 I admit the supersinks break the flexibility offered by gst, but as far as i can foresee, is the straight strategy to obtain the performance promised by omx in his interop profile. Maybe one day the interop profile won't be necessary when the pBuffer in the bufferheader leave its readonly property... Víctor Manuel Jáquez Leal On Wed, Jul 30, 2008 at 9:00 AM, Ling Shi <<a href="mailto:shil66@gmail.com" target="_blank">shil66@gmail.com</a>> wrote: > Hi, all > I'm in a research project to port gstreamer into embedded system. Now, we > encounter the issue on how to integrate hardware accelerators (DSP/GPU) into > gst. After evaluating different solution, we think GstOpenMAX project may be > the best one for us, because > >  1) OpenMAX is an industry standard >  2) more and more DSP/GPU vendor support OpenMAX > > But, I still have several questions on this project. > 1. Does Nokia N8xx serials use GstOpenMAX? > I know Nokia engineers lead this project. I also know N8xx serials use > gstreamer as default playback engine, and it uses TI OMAP 2420, which has > DSP. Can anyone tell me if N8xx use GstOpenMAX? If no, does N8xx plan to use > it in the future? > > 2. What's GstOpenMAX plan to support DSP/GPU in the future? > I review several plugins in GstOpenMAX and find current design can only > support none-tunnel communication.  It's not the best solution in hardware, > because of bad performance. So, we plan to improve it by adding tunneled or > proprietary communication. Do you have such plan? If yes, can we involve in > design? > > In addition, most accelerators work as a decoder and a render. It means, the > encoded data sent to it will directly be decoded and rendered, and will not > be retrieved back again. How the gst or omx organize its pipeline in this > situation? We are evaluating two solutions. > > ===Solution 1=== > We can design a super omx sink component to cover decoder and render. This > is the solution is used by N8xx. > src ! demux ! sink >                          | >                        super omx sink >                          | >                    +--------------------------------------------+ >                    |    hardware accelerator                | >                    +--------------------------------------------+ > > ===Solution 2=== > We can separate omx decoder, omx post processer, and omx sink elements. We > enhance decoder, post processor, and sink plugin in GstOpenMAX. If > GstOpenMAX plugin found its neighborhood are GstOpenMAX plugin, it will try > to establish tunneled communication or proprietary communication firstly. It > means, although we have 3 OMX plugin in gst pipeline, there is no data in > gst pad and omx port. The last two gst/omx plugin only provide control > function, but not support process data flow. Of cause, If the connection is > failed, it will use none-tunnel communication. > > src ! demux ! decoder ! post processor ! sink >                            |                    |                  | >                      omx dec          omx pp      omx sink >                            |                    |                  | >                    +--------------------------------------------+ >                    |    hardware accelerator                | >                    +--------------------------------------------+ > > It seems solution 2 is more flexible. How about your suggestion on the 2 > solutions? Which one is feasible? Do you have other solutions? > > Thank you very much. > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > <a href="http://moblin-contest.org/redirect.php?banner_id=100&url=/" target="_blank">http://moblin-contest.org/redirect.php?banner_id=100&url=/</a> > _______________________________________________ > Gstreamer-openmax mailing list > <a href="mailto:Gstreamer-openmax@lists.sourceforge.net" target="_blank">Gstreamer-openmax@lists.sourceforge.net</a> > <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax</a> > > ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world <a href="http://moblin-contest.org/redirect.php?banner_id=100&url=/" target="_blank">http://moblin-contest.org/redirect.php?banner_id=100&url=/</a> _______________________________________________ Gstreamer-openmax mailing list <a href="mailto:Gstreamer-openmax@lists.sourceforge.net" target="_blank">Gstreamer-openmax@lists.sourceforge.net</a> <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax</a> </div></div></div></div></div> <hr size="1"><a href="http://cn.mail.yahoo.com/" target="_blank"> 雅虎邮箱，您的终生邮箱！</a></div> ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world <a href="http://moblin-contest.org/redirect.php?banner_id=100&url=/" target="_blank">http://moblin-contest.org/redirect.php?banner_id=100&url=/</a> _______________________________________________ Gstreamer-openmax mailing list <a href="mailto:Gstreamer-openmax@lists.sourceforge.net">Gstreamer-openmax@lists.sourceforge.net</a> <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-openmax</a> </blockquote></div> </div>