Video standards

[salsaman]

I don't think you are paying enough attention to the main points. Ir is not
simply a case of extending the fourcc values to include more. If I didn't
make it clear enough, the whole fourcc system is obscure, inadequate,
ambiguous. The only reason ever to use it would be when you don't have meta
data and you are forced to encode the format in the first 4 bytes.

Colorimetry is only relevant when displaying on a monitor. In the video
world we just have red, green and blue (plus alpha, y, u and v). These are
just labels for the colour channels, mapping them to bit formats.

The values I mentioned are all necessary if you want to convert from one
colourspace to another. For example if I decode a video frame and the pix
format is YUV420P then to convert it to RGBA to display via openGL, I need
to know the YUV subspace (bt709 or itu601) and whether the values are
clamped or full range. Then I apply the standard conversion factors (Kr =
0.2126, Kb = 0.0722 for bt709). This cannot be derived from the fourcc
(generally). No doubt there is a standard definition of definition of the
R,G,B primaries, but that isnr a concern.  I just feed the values into an
openGL texture buffer, and SDL buffer, a gdkpixbuf, QImage or whatever and
ask for it to be displayed. Now in an application I may optionally offer
the user filters to adjust the white balance, contrast, display gamma etc.
but that is outside of the scope of what I am proposing.

And no, it is not a case of "adding another standard" and confusing things,
there is no standard.

I just had a look at pipewire, there is nothing bad about it per se, they
mention their palette values are based on gstreamer. So fine, we have yet
another library specific set of definitions.

It's like I am trying to invent Esperanto, and all you can say is...."oh
you don't like English, well have you considered speaking German instead ?"

Well that is it, I am done. I was asked how XDG video could be useful. I
explained the shortcomings of what exists currently, and outlined various
ways in which having a standard could be useful.

But if there is no will for this, then I am not going to waste any more of
my time on this. My own standards work very well for my own purposes, and
if I ever wanted to use pipewire for example, I can simply add the
constants to my compatibility header.


Cheers.
G,




On Fri, 5 Apr 2024 at 06:34, Pekka Paalanen <pekka.paalanen at haloniitty.fi>
wrote:

> On Thu, 4 Apr 2024 17:13:40 -0300
> salsaman <salsaman at gmail.com> wrote:
>
> > Hi,
> > the problem with the drm.h header is, it is complicated, still needs
> > interpretation, and it lacks some commonly used formats, (e.g YUVA4444p)
>
> They accept additions, if the additions serve userspace
> interoperability. There is no requirement to actually use the format in
> the kernel.
>
> Pixel formats are complicated, yes. There are too many pixel format
> enumerations, every one differently defined, sure. I wouldn't add yet
> another system of definitions.
>
> > Also it doesn't address the gamma value (linear, sRGB, bt701), or the yuv
> > subspace, (eg Y'CbCr vs bt701), the yuv ramge (16 - 240. 16 - 235 =
> clamped
> > / mpeg. 0 - 255 unclamped, full, jpeg range) or uv sampling position, e.g
> > center, top_left)
>
> My opinion is that that none of that is relevant to a pixel format.
> These are additional information that must be decoupled from the pixel
> format to avoid a combinatorial explosion of the format enumeration,
> which is already massive even without them. A pixel format only
> describes a part of the memory layout: which set of bits forms a raw
> channel value of a pixel, and what are the channel names. Giving any
> further meaning to those raw values is for other metadata.
>
> What about colorimetry? Primaries and white point, dynamic range, plus
> the difference between encoding colorimetry (container color volume)
> and the usable/used colorimetry (target color volume, which is present
> in e.g. HDR static metadata typical for BT.2100/PQ signals in the form
> of the Mastering Display Color Volume).
>
> What about the assumed viewing environment, if we want to go from just
> stimulus towards appearance?
>
> > I can see that having some common definitions would be useful for
> > exchanging data between applications. Eg  my app gets a frame buffer and
> > metadata XDG_VIDEO_PALETTE_RGB24, XDG_VIDEO_GAMMA_LINEAR
> > then I know unambiguously that this is planar RGB 8:8:8 (so forget
> little /
> > big endian) and that the values are encoded with linear (not sRGB) gamma.
>
> > If you want to be more specific with palettes, then you could do so, but
> it
> > might require defining metadata structs,
>
> > I'll try to explain the rationale a bit. In the audio world it is quite
> > common for apps to send audio from one to another. Generally speaking
> they
> > would send or receive via an audio server, e.g pulseaudio, jack.
> > Now imagine the same for video,
>
> This sounds like Pipewire. One would develop Pipewire API to carry the
> necessary metadata. One could choose to follow something massive like
> ITU-T H.274, or maybe follow what we are brewing for Wayland.
>
> To my understanding, Pipewire is already becoming very common among
> desktop environments for routing audio and video streams between
> applications and system components and devices.
>
>
> Thanks,
> pq
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/xdg/attachments/20240405/6cd9aaba/attachment.htm>