Video standards

Thu Apr 4 00:51:39 UTC 2024

Regarding my expertise, I was one of the developers most involved in
developing the "livido" standard which was one of the main topics of the
Piksel Festivals held in Bergen, Norway.
In the early days (2004 - 2006) the focus of the annual event was precisely
the formulation of free / open standards, in this case for video effects.
Other contributors included:
 Niels Elburg, Denis "Jaromil" Rojo, Tom Schouten, Andraz Tori, Kentaro
Fukuchi and Carlo Prelz.
I've also been involved with and put forward proposals for common command /
query / reply actions (Open Media Control). To the extent that these
proposals have not gained traction, I don't ascribe this to a failing in
the proposals, but rather to a lack of developer awareness.

Now regarding specific areas, I went back and reviewed some of the
available material at  https://www.freedesktop.org/wiki/Specifications/

free media player specifications
https://www.freedesktop.org/wiki/Specifications/free-media-player-specs/
metadata standards for things like comments and ratings - talks mainly
about audio but describes video files also

I am not a big fan of dbus, but this looks fine, it could be used for video
players. I'd be happier if it were a bit more abstracted and not tied to a
specific implementation (dbus). I could suggest some enhancements but I
guess this is a dbus thing and not an xdg thing.

IMO what would be useful would be to define a common set of constants, most
specifically related to frame pixel fornats
The 2 most common in use are fourCC and avformat

Consider a frame in UYVY fornat

fourCC values:

 #define MK_FOURCC(a, b, c, d) (((uint32_t)a) | (((uint32_t)b) << 8)     \
                               | (((uint32_t)c) << 16) | (((uint32_t)d) <<
24))

MK_FOURCC('U', 'Y', 'V', 'Y')
but also
MK_FOURCC('I', 'U', 'Y', 'B')
the same but with interlacing
MK_FOURCC('H', 'D', 'Y', 'C')
same but bt709 (hdtv) encoding

so this requires interpretation by sender / receiver - a simpler way could
be with constants

- probably the nearest we have are ffmpeg / libav definitions, but this is
the wrong way around, a lib shouldn't define a global standard, the
standard should come first and the lib should align to that.

We have AV_PIX_FMT_UYVY422 which was formerly PIX_FMT_UYVY422
and AVCOL_TRC_BT709, which is actually the gamma transfer function, There
is no equivalent bt709 constant fot bt709 yuv / rgb, instead this exists as
a matrix.

Now consider how much easier it would be to share data if we had the
following constants enumerated:

*XDG_VIDEO_PALETTE_UYVY*
*XDG_VIDEO_INTERLACE_TOP_FIRST*
*XDG_VIDEO_YUV_SUBSPACE_BT709*
*XDG_VIDEO_GAMMA_SRGB*

(this is an invented example, not intended to be a real example).

There is a bit more too it but that should be enough to give a general idea.

G,

On Wed, 3 Apr 2024 at 08:12, Pekka Paalanen <pekka.paalanen at haloniitty.fi>
wrote:

> On Thu, 28 Mar 2024 19:19:33 -0300
> salsaman <salsaman at gmail.com> wrote:
>
> > There are two hardware settings from the monitor that overlap video,
> these
> > are
> > - monitor aspect ratio
> > - monitor pixel aspect ratio
> > These are both useful when rendering video. The first defines how much
> > stretch or letterbocing to apply, the second defines non square pixels,
> > which is goof to know if you want to render fixed size objects (a circle
> > for example). Knowing the monitor size in RGB or Y plane pixels can also
> be
> > useful to define a max or min resize limit (whether it is min or max
> > depends on the desired display quality level)
>
> Thanks. I was trying to ask what kind of video standards you have
> experience and expertise in?
>
> I'm also interested in what kind of standards you see as missing. The
> Wayland extension aims to cover everything display related. I'm sure
> video file format specifications do their job.
>
> What would be left to define?
>
> What goals would there be?
>
> I suppose individual APIs like Pipewire might be lacking something, but
> that's a Pipewire API rather than an XDG standard. Or do we need an XDG
> standard to be used as the design guide and reference for APIs?
>
>
> Thanks,
> pq
>
> > On Thu, 28 Mar 2024 at 19:05, salsaman <salsaman at gmail.com> wrote:
> >
> > > colour management and hdr mostly intersect with three areas of video:
> > > pixel formats, yuv <-> rgb conversions and gamma transfer functions.
> > > For example
> > > xdg_pixformat_yuv121010
> > > xdg_subspace_bt2020
> > > xdg_gamma_bt2020
> > >
> > > just off the top of my head, these arent intended to be actual
> suggestions
> > >
> > >
> > > On Thu, 28 Mar 2024 at 18:57, salsaman <salsaman at gmail.com> wrote:
> > >
> > >> In addition, I am not sure if there are xdg standards for audio, but I
> > >> would suggest video and follow similar hierarchies, and that both
> could be
> > >> classed under a more generic xdg multimedia standard.
> > >>
> > >>
> > >> On Thu, 28 Mar 2024 at 18:48, salsaman <salsaman at gmail.com> wrote:
> > >>
> > >>> Hi, IMO hardware related would be more appropriate under display
> > >>> standards
> > >>> Video standards could be more software related, and provide common
> > >>> definitions, for example , allowing exchange of information between
> > >>> applications which produce or consume video frames or streams of
> frames.
> > >>> Some examples I can think of might be
> > >>>  xdg_colorspace_RGB,
> > >>>  xdg_colorspace_YUV
> > >>>
> > >>> xdg_pixfmt_RGB24
> > >>> xdg_pixfmt_YUV420p
> > >>> etc
> > >>>
> > >>>  xdg_gamma_linear
> > >>>  xdg_gamma_sRGB
> > >>>
> > >>> xdg_video_width
> > >>> xdg_video_height
> > >>>
> > >>> I could provide a more full list, but I think if it goes along this
> > >>> route. the starting point has to be what are we setting out to
> achieve with
> > >>> the standards / definitions, and provide a range of speculative use
> cases.
> > >>>
> > >>> Gabriel (salsaman)
> > >>>
> > >>>
> > >>> On Thu, 28 Mar 2024 at 06:07, Pekka Paalanen <
> > >>> pekka.paalanen at haloniitty.fi> wrote:
> > >>>
> > >>>> On Wed, 27 Mar 2024 11:45:00 -0300
> > >>>> salsaman <salsaman at gmail.com> wrote:
> > >>>>
> > >>>> > ISTR that the xdg video standards were never defined, If you need
> any
> > >>>> > advice or assistance with this, I would be happy to act in an
> > >>>> > advisory capacity if that is called for. I have over 20 years
> > >>>> experience of
> > >>>> > developing Free Software video and have been an active
> participant in
> > >>>> > developing other video / effects standards. I have been a bit our
> of
> > >>>> the
> > >>>> > spotlight recently as I have been busy architecting and
> implementing
> > >>>> the
> > >>>> > core components of the upcoming next gen LiVES 4,0 video
> application
> > >>>> plus
> > >>>> > its accompanying state-of-the-art effects standard)
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> what kind of video standards would that be?
> > >>>>
> > >>>> I'm wondering if it would have anything to do with Wayland color
> > >>>> management and HDR:
> > >>>>
> > >>>> https://gitlab.freedesktop.org/pq/color-and-hdr
> > >>>>
> > >>>>
> https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/183
> > >>>>
> > >>>>
> https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14
> > >>>>
> > >>>> Would there need to be any XDG standards to support color managed
> HDR
> > >>>> desktops, or is the window system support enough?
> > >>>>
> > >>>> I have not much in my mind, but then I've been staring only at the
> > >>>> window system interactions, and haven't seen what else the desktop
> > >>>> ecosystem or applications might need.
> > >>>>
> > >>>> Recommended display calibration and measurement procedures maybe?
> > >>>>
> > >>>> Desktop viewing environment standards?
> > >>>>
> > >>>> Viewing environment measurement?
> > >>>>
> > >>>> They could be as straightforward as referring to a freely available
> > >>>> ITU-R or SMPTE papers or others, if there are suitable ones.
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>> pq
> > >>>>
> > >>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/xdg/attachments/20240403/70381927/attachment.htm>