Video standards

[Pekka Paalanen]

On Fri, 5 Apr 2024 08:28:27 -0300
salsaman <salsaman at gmail.com> wrote:

> I don't think you are paying enough attention to the main points. Ir is not
> simply a case of extending the fourcc values to include more. If I didn't
> make it clear enough, the whole fourcc system is obscure, inadequate,
> ambiguous. The only reason ever to use it would be when you don't have meta
> data and you are forced to encode the format in the first 4 bytes.

Right. You must be talking about some other fourcc system. There are
many of them, and some combine multiple orthogonal things into a single
enumeration, which then becomes very difficult to extend and work with.

drm_fourcc.h is not one of those.

Metadata is always necessary anyway, either implied or explicit.

> Colorimetry is only relevant when displaying on a monitor. In the video
> world we just have red, green and blue (plus alpha, y, u and v). These are
> just labels for the colour channels, mapping them to bit formats.

That is a very surprising opinion. Have you worked on HDR imagery?
Or wide color gamut?

> The values I mentioned are all necessary if you want to convert from one
> colourspace to another. For example if I decode a video frame and the pix
> format is YUV420P then to convert it to RGBA to display via openGL, I need
> to know the YUV subspace (bt709 or itu601) and whether the values are
> clamped or full range. Then I apply the standard conversion factors (Kr =
> 0.2126, Kb = 0.0722 for bt709). This cannot be derived from the fourcc
> (generally). No doubt there is a standard definition of definition of the
> R,G,B primaries, but that isnr a concern.  I just feed the values into an
> openGL texture buffer, and SDL buffer, a gdkpixbuf, QImage or whatever and
> ask for it to be displayed. Now in an application I may optionally offer
> the user filters to adjust the white balance, contrast, display gamma etc.
> but that is outside of the scope of what I am proposing.

Yes, those are all important properties, and not enough.

> And no, it is not a case of "adding another standard" and confusing things,
> there is no standard.

There are standards. ITU-T H.273, coding-independent code points, for
example. That combines well with drm_fourcc.h. Also ICC combines well
with drm_fourcc.h. This works, because drm_fourcc.h does not attempt to
define anything outside of the memory layout and abstract channels.

> I just had a look at pipewire, there is nothing bad about it per se, they
> mention their palette values are based on gstreamer. So fine, we have yet
> another library specific set of definitions.
> 
> It's like I am trying to invent Esperanto, and all you can say is...."oh
> you don't like English, well have you considered speaking German instead ?"

That does seem like an apt analogue.

> 
> Well that is it, I am done. I was asked how XDG video could be useful. I
> explained the shortcomings of what exists currently, and outlined various
> ways in which having a standard could be useful.

Sorry, but I haven't understood what gap there is that would need to be
filled with yet another pixel format enumeration. Or is it perhaps the
same gap that we are filling in Wayland?

We need to communicate a lot more about images than what pixel formats
ever could. We are building that definition based on existing standards
defining their own aspects: drm_fourcc.h, H.273 / ICC, and then adding
what's missing like the how or if alpha channel is baked into RGB (the
two ways to pre-multiply). Since these are all well-defined and
orthogonal, there is no problem combining them.

Wayland also already provides ways to handle some things, like pixel
aspect ratio, so we don't want to define another conflicting way to
define the same thing. That means the solution for Wayland is probably
not applicable somewhere else, and vice versa.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/xdg/attachments/20240405/afffccf2/attachment.sig>