[gst-devel] Proposed MIME types/caps for high bit-depth and float video

[Josh]

I'm working with cameras that output high bit-depth and sometimes
floating point video. While some of these might seem crazy, (almost)
all of these will automatically be supported by the gst-opencv
elements, so many operations including colorspace conversion will be
possible.

Below are the MIME types and caps that I've come up with, omitting the
common width, height, and framerate fields. I'm taking some liberty
with the formatting of these caps to save space, so this isn't exactly
how they'd appear in actual use.

Raw 16-bit, 32-bit grayscale, signed or unsigned (use existing caps,
but add signed field):
video/x-raw-gray
  bpp: [ 1, 32 ]
  depth: { 8, 16, 32 }
  endianness: BYTE_ORDER
  signed: {TRUE, FALSE}

Raw floating point (half, single, and double) grayscale:
video/x-raw-gray-float
  bpp: { 16, 32, 64 }
  depth: { 16, 32, 64 }
  endianness: BYTE_ORDER

Raw 16-bits per channel RGB(A):
video/x-raw-rgb16
  bpp: { 48, 64 }
  depth: { 48, 64 }
  endianness: BYTE_ORDER
  red_mask: { 255, ... }
  green_mask: { 255, ... }
  blue_mask: { 255, ... }
  alpha_mask: { 255, ... }

Raw floating point RGB(A):
video/x-raw-rgb-float
  bpp: { 96, 128 }
  depth: { 96, 128 }
  endianness: BYTE_ORDER
  red_mask: { ?? }
  green_mask: { ?? }
  blue_mask: { ?? }
  alpha_mask: { ?? }

For the floating point grayscale we could also create separate types
such as video/x-raw-gray-half, video/x-raw-gray-single (or float),
video/x-raw-gray-double.

For RGB(A)16 we can't (and probably shouldn't) re-use the existing
video/x-raw-rgb since the masks are only 32-bits and we need at least
64-bits.

For RGB(A)F it might be crazy to think of adding half and double
formats here. The other big problem is we don't have a 128-bit integer
yet so to have masks we'd need two integers for each channel. Since
there aren't many (any?) using this type now we could just force them
to use RGB order and forget the masks entirely.

I would appreciate any feedback. Thanks!
-Josh