architectural question (Android): gstreamer / openmax / libfslvpuwrap / imxvpuapi

[Stephen Webb]

Sebastien,

Thank you very much for the reply.  It was helpful, but of course it raises
more questions. :)

Maybe it would be more clear if I explain what I'm trying to do and someone
could point out problems / make suggestions.

1.  I'm using Android on a Compulab Utilite iMX6 platform.
2.  I'm currently using Gstreamer as a general way to decode video files -
 I want to retain this.
3.  I am adding support for hardware decode of H264 stream - *low latency
is important.*
4.  When I use the iMX6 hardware decode path in Gstreamer I get 8+ frames
of latency:
     decoder  = gst_element_factory_make
("amcviddec-omxfreescalestdvideodecoderavcv3hwbased",  "decoder");
5.  I googled for other people with similar issues and came across
suggestions like this: (effectively "set nReorderEnable=false if your
stream doesn't use B frames"):
   https://community.freescale.com/thread/304322

So I went down the path of changing vpudec.c to change nReorderEnable=false
etc...*I think this may be where my mistake was as I'm realizing that this
may not work on Android*.  Similarly I found the imxvpuapi library as a way
to achieve the same thing, but again I'm now thinking this isn't for
Android.

Basically I want to retain gstreamer for my general video file playback
needs and I also want to come up with a way to decode a live H264 stream
with low latency.  It is OK for the low latency decode to be "tied" to the
iMX6 platform. *I now think that using OpenMax IL directly (with the caveat
you mentioned) is probably my best option*.

Does that seem right?


Steve



On Mon, Oct 6, 2014 at 10:55 AM, Sebastian Dröge <sebastian at centricular.com>
wrote:

> On Mo, 2014-10-06 at 10:47 -0600, Stephen Webb wrote:
> > I'm trying to understand how these things fit together / relate to each
> > other.  I'm afraid that I may not know enough to even ask intelligent
> > questions at this point, so if someone can point me to documentation or
> > other resources tat would help, that would be great.
> >
> > My current questions:
> >
> > 1.  How do gstreamer and openmax relate to each other?  Is it best to
> think
> > of gstreamer as a replacement/alternative to Openmax?  A replacement for
> > the OpenMax AL?  They both seem to do a lot of the same things.
>
> GStreamer is a bit like OpenMAX AL and OpenMAX IL together, but has an
> even broader scope than those too.
>
> GStreamer can use OpenMAX IL via gst-omx (so you can use e.g. hardware
> codecs that have an OpenMAX IL API in GStreamer pipelines). In theory
> you could also implement the OpenMAX IL and AL API on top of GStreamer.
>
> > 2.  I was under the impression that OpenMax was the standard way to get
> > multimedia support on Android - as in it is required in some way when
> doing
> > multimedia work on Android (specifically when any low level hardware
> access
> > comes into play).  Having dug in and looked around a fair amount, I don't
> > see much mention of OpenMax - it doesn't seem so central at all.
>
> OpenMAX IL is not public API on Android but many devices use it (but you
> shouldn't use it directly unless you target exactly a specific device),
> OpenMAX AL is public API on Android but last time I looked only
> supported MPEG TS.
>
> The public API for accessing codecs on Android is the Java
> android.media.MediaCodec API, which is supported by the GStreamer
> androidmedia plugin.
>
> > 2.  I found the imxvpuapi project.  If this works on Android this seems
> > like exactly what I need.
> > a.  Does it work on Android?
> > b.  With the only dependency being libfslvpuwrap / vpu_wrapper.h this
> > appears to interface directly with the freescale vpu driver (no OMX, no
> > gstreamer).  Is this right?  Does this work on Android?  If so, am I
> > confused about how Android locks things down and doesn't allow
> applications
> > to get low level hardware access?
>
> I don't know any answers for this one, but you're constraining yourself
> to iMX based devices then in any case :)
>
> --
> Sebastian Dröge, Centricular Ltd - http://www.centricular.com
> Expertise, Straight from the Source
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20141006/d158746f/attachment.html>