[Bug 758699] Assume unknown OMX color formats are similar to YUV420

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=758699

--- Comment #6 from Mathias Hasselmann (IRC: tbf) <mathias.hasselmann at gmx.de> ---
(In reply to Sebastian Dröge (slomo) from comment #5)
> (In reply to Nicolas Dufresne (stormer) from comment #4)
> > (In reply to Mathias Hasselmann (IRC: tbf) from comment #3)
> > > (In reply to Nicolas Dufresne (stormer) from comment #2)
> > > > That seems a very crash prone idea for the case where it's not.
> > > 
> > > Yes, still we have to find a way to deal with all the undefined color
> > > formats reported by all this real hardware out in the wild. Actually could
> > > not find a single Android device that had a H.264 hardware decoder reporting
> > > any supported color format.
> > 
> > Yes, that way already exist, it's called texture rendering. We are already
> > working on this.
> 
> And that is already finished now in gst-plugins-bad GIT with Matthew's
> latest changes. Only needs some wider testing :)

Is there a branch for this work? For me master still rejects any
OMX_COLOR_FormatAndroidOpaque format (0x7F000789).

I'd also like to point out that master consumes 2.4 the CPU time of pure
Android Media or even QtMultimedia solutions still. So there is room for
improvement.

> > > Well, guess I should add all the well known formats to the switch statement.
> > > Any maybe we can have some sanity checks, that verify if the buffer has
> > > exactly the expected size, as indicated by the chosen color format.
> > 
> > For the mappable memory, the appropriate thing to do is report all the
> > values that are found and map them. There is additional meta-data for most
> > of these I420 and NV12 (I believe it's half NV12 and half I420 unlike what
> > you mention). The size and the stride of I420 and NV12 is exactly the same.
> 
> Yes I also dislike this idea. And if anything we should probably use NV12
> instead of I420, NV12 seems to be slightly more common.
> But in general this is not a good idea, instead we should report an error
> message and let the user report a bug directly. Otherwise users will just
> see that "GStreamer does not work" instead of getting the information that
> it's just Android that is providing completely useless APIs.

I fully agree, that my approach above is too simplistic and broken. Alone
because it claims to certainty about the video format while it was guessing,
potentially causing equivalent, but properly recognized formats to get
rejected. This is plain wrong and needs to be fixed for a proper patch.

I also agree, that within controlled environments like embedded platforms being
strict about formats is the proper approach. Predictable behavior is the key to
such projects and like Sebastian points out, missing features, slow code paths
are found quickly in such environment.

Now let's enter the world of Android, sometimes used as embedded platform, were
all the assumptions about strict error handling apply. Sometimes just a wild
zoo of consumer hardware, with new devices, new chipsets popping up "every
other week". In that scenario it seems more beneficial to me be fault tolerant
and to run a best effort approach to support even the most exotic animals of
the zoo. That mindset is where this bug report comes from.

Now the question is, what Android devices GStreamer targets. Does it aim for
the embedded use case only, or does it also target Play Store applications. In
first case this bug report is just invalid, in the second case we should come
up with a solution for doing a best effort approach to identify unknown
formats.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.