[Gstreamer-openmax] Discussion on the hardware accelerator solution in GstOpenMAX project.

[Ling Shi]

Felipe and Bruno,
Thank for your reply. We have a very similar idea on how to use OMX in gst.
Now, I got more confidence to use, involve, and contribute into this
project.

Brouno,
Could you tell me where can find your change? Is it possible for me to study
your code before release?

Felipe,
TI OMAP 3xxx is just one of our target platform. I just got an TI OMAP 3xxx
board in test. I will investigate the OMX IL code from omap zoom.

Please check other comments in line.


On Wed, Jul 30, 2008 at 4:39 PM, Felipe Contreras <
felipe.contreras at nokia.com> wrote:

> Hi,
>
> On Wed, 2008-07-30 at 15:00 +0800, ext Ling Shi wrote:
> > Hi, all
> > I'm in a research project to port gstreamer into embedded system. Now,
> > we encounter the issue on how to integrate hardware accelerators
> > (DSP/GPU) into gst. After evaluating different solution, we think
> > GstOpenMAX project may be the best one for us, because
> >
> >   1) OpenMAX is an industry standard
> >   2) more and more DSP/GPU vendor support OpenMAX
> >
> > But, I still have several questions on this project.
> > 1. Does Nokia N8xx serials use GstOpenMAX?
> > I know Nokia engineers lead this project. I also know N8xx serials use
> > gstreamer as default playback engine, and it uses TI OMAP 2420, which
> > has DSP. Can anyone tell me if N8xx use GstOpenMAX? If no, does N8xx
> > plan to use it in the future?
>
> The current Maemo Products don't use OpenMAX IL, they use TI's DSP
> directly through the open source version of the DSP bridge (DSP
> gateway).
>
> The plan is to use OpenMAX IL so we can choose between different
> implementations without much effort.
>
> TI has started to provide their OpenMAX IL source code:
> http://omapzoom.org/gf/project/openmax/wiki/
>
> > 2. What's GstOpenMAX plan to support DSP/GPU in the future?
> > I review several plugins in GstOpenMAX and find current design can
> > only support none-tunnel communication.  It's not the best solution in
> > hardware, because of bad performance. So, we plan to improve it by
> > adding tunneled or proprietary communication. Do you have such plan?
> > If yes, can we involve in design?
>
> Indeed, as Bruno mentions, NXP has contributed code that adds support
> for tunneled communication. It is maintained in a separate branch and
> will soon be merged to the master branch.
>
> > In addition, most accelerators work as a decoder and a render. It
> > means, the encoded data sent to it will directly be decoded and
> > rendered, and will not be retrieved back again. How the gst or omx
> > organize its pipeline in this situation? We are evaluating two
> > solutions.
> >
> > ===Solution 1===
> > We can design a super omx sink component to cover decoder and render.
> > This is the solution is used by N8xx.
> > src ! demux ! sink
> >                          |
> >                        super omx sink
> >                          |
> >                    +--------------------------------------------+
> >                     |    hardware accelerator                 |
> >                    +--------------------------------------------+
>
> The disadvantage of this solution is that it requires the creation of
> many elements to cover all the possible combination of elements. This
> becomes specially a problem when you add for example some filtering,
> like a volume control, etc.


[Shi Ling]
You idea is exaclty same with me. I also think it's not a good solution.



>
>
> > ===Solution 2===
> > We can separate omx decoder, omx post processer, and omx sink
> > elements. We enhance decoder, post processor, and sink plugin in
> > GstOpenMAX. If GstOpenMAX plugin found its neighborhood are GstOpenMAX
> > plugin, it will try to establish tunneled communication or proprietary
> > communication firstly. It means, although we have 3 OMX plugin in gst
> > pipeline, there is no data in gst pad and omx port. The last two
> > gst/omx plugin only provide control function, but not support process
> > data flow. Of cause, If the connection is failed, it will use
> > none-tunnel communication.
> >
> > src ! demux ! decoder ! post processor ! sink
> >                             |                     |                  |
> >                      omx dec          omx pp       omx sink
> >                             |                     |                  |
> >                    +--------------------------------------------+
> >                     |    hardware accelerator                 |
> >                    +--------------------------------------------+
> >
> > It seems solution 2 is more flexible. How about your suggestion on the
> > 2 solutions? Which one is feasible? Do you have other solutions?
>
> This is exactly how NXP implemented it and seems to be working fine.
>
> The problem I see with both solutions is that there will be A/V sync
> issues when using OMX sinks in tunneling mode. The idea is to solve
> these issues by mapping the OMX clock to the GST clock. However, this
> hasn't been implemented yet.
>
[Shi Ling]
Yes, so many things need to be improved in the future.


>
> Best regards.
>
> --
> Felipe Contreras
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-openmax/attachments/20080730/263d5dab/attachment.htm>