Caps for full-range YUV data?

[Gruenke, Matt]

Thanks for the prompt reply!


On Friday, July 29, 2011 02:02 -0400, David Schleef wrote:

>> Over the years, I've heard a number of people ask "But 255 > 219,
>> isn't full-range YUV more precise?"  The answer is no, because
>> it's almost always compressed (not to mention noisy), in which
>> case the precision is entirely dependent on the codec bitrate.
>> And if you're doing uncompressed and care about precision, you'd
>> use 10-bit video.

While you are correct that this is typically not an issue, in practice,
I can point out two exceptions I've personally encountered during my
career.

First and foremost to my current interests, some analytic algorithms &
processing are sensitive to quantization.  The issue is not so much
about whether the quantization is above the noise floor, but rather its
aggregate structure that can cause problems.

Years ago, I was involved in adding video effects to a popular nonlinear
video editing system (which was based on MJPEG compression).  Users were
quick to point out cases where the "banding" (i.e. contours visible on
smooth gradients) were readily apparent.  It turned out that some of the
effects (especially blurs and certain color effects) made the
quantization introduced by rescaling between full- & 601- range readily
visible.  The effects had originally been written for a system which
used full-range.  In some cases, the effects could function effectively
as-is on 601-range video.  In other cases, the source code of effects
had to be altered for them to work properly.


>> Did you consider this option, or what's the
>> rationale behind 'color-matrix'?

> Yes, I considered several options, including adding fields for
> transfer characteristics, color primaries, etc.

I guess part of my quibble with 'color-matrix' is that if you didn't
explain it to me, I'd still be thinking it determined only the specific
colorspace, using values like sdtv, hdtv, and jpeg as merely shorthand
for the default matrices used by those formats.  For such a broad cap, I
think a name like 'video-signal-type' would be more self-explanatory.


> It turns out that none of these are relevant, because
> nobody uses them for real work.

I don't mean to be glib, but, there's certainly such a thing as a
self-fulfilling prophesy - or, to put it another way: if you don't build
it, it certainly won't get used.

Speaking for myself, we actually do use that stuff (in our old,
proprietary pipeline that we're moving away from), because we are trying
to analyze video content.  Since errors can accumulate, compound, and
change the characteristics of the data, it can pay off to get the
details right in each stage of processing.  Certain folks in video
editing, post-production, and mastering might care about precision,
artifacts, and processing-induced generation-loss.  Videophiles might
notice problems in certain corner cases, or in an A-B comparison with a
product using another video engine.  These are the kinds of folks who
can really help push the technology forward if you can get/keep them on
board.

I say this because it seems a shame to have spent so much time and
energy on making GStreamer so general, abstract, and flexible, but stop
short of accurately capturing media signal semantics.  But that's just
my perspective, and I realize it doesn't count for much, as I'm not
exactly a primary contributor.


I'll conclude with one idea.  It seems like MPEG-4 tried to do something
similar to what you're after.  The fields I mentioned in my previous
message are all optional, including one I didn't mention: video_format.
This can take values such as NTSC, PAL, SECAM, and MAC.  The spec
doesn't indicate how this should be used, but you can imagine they
considered having it switch the defaults in the other fields.

What I'm getting at is that if you want to make it easy for the majority
of users, one idea might be to have some special utility functions (for
developers) & properties in capsfilter & capssetter (for commandline
users) for applying or filtering by certain video signal defaults (e.g.
SDTV/HDTV or NTSC/PAL/HDTV/JPEG).  Perhaps an approach like that might
keep life simple for the majority of users & developers, while enabling
those developers focused on specialized applications or the low level
minutiae of video processing to get the details right.


Thanks, again, for your quickly and precise answers to my questions.  I
also appreciate your focus on ease of use & the core user community.  I
just hope we can find ways to accomplish this without limiting
applicability to problems in high-end, industrial, professional, and
scientific applications.


Regards,
Matt Gruenke