Additional pixel formats for wl_shm

[Pekka Paalanen]

On Wed, 8 May 2013 12:22:10 -0400
Kristian Høgsberg <hoegsberg at gmail.com> wrote:

> On Wed, May 08, 2013 at 10:29:21AM +0300, Pekka Paalanen wrote:
> > On Thu, 2 May 2013 10:23:04 +0300
> > Pekka Paalanen <pekka.paalanen at collabora.co.uk> wrote:
> > 
> > > On Wed, 1 May 2013 21:38:48 -0400
> > > Kristian Høgsberg <hoegsberg at gmail.com> wrote:
> > > 
> > > > On Fri, Apr 26, 2013 at 04:08:01PM +0300, Pekka Paalanen wrote:
> > > > > Hi Kristian,
> > > > > 
> > > > > I'm working on Raspberry Pi, where the VideoCore can deal directly
> > > > > with a large number of pixel formats. However, wl_shm only exposes
> > > > > ARGB and XRGB 32-bit per pixel formats, which on such tiny devices
> > > > > are very wasteful.
> > > > > 
> > > > > Would it be acceptable to add more formats to the wl_shm formats
> > > > > enum? I have happily forgot all the previous discussions about the
> > > > > topic, and a quick search into the mailing list archives didn't
> > > > > turn up much.
> > > > > 
> > > > > We are especially interested in 16-bit per pixel formats to save
> > > > > memory and bandwidth, and if I am reading the GLESv2 specification
> > > > > right, RGB565, RGBA4444 and RGBA5551 should be directly supported,
> > > > > so at least those could be uploaded into textures without manual
> > > > > conversion. Of course, the new rpi-renderer I am working on does
> > > > > not use GL for compositing anymore, but it should be able to use
> > > > > those formats directly, too.
> > > > > 
> > > > > I have also another proposition: an extension to let wl_shm support
> > > > > custom pixel formats outside of the core protocol listed formats.
> > > > > 
> > > > > It would have a new global interface, which when bound, would
> > > > > advertise additional pixel formats as a (string, uint) tuple, where
> > > > > the string describes the pixel format, and the uint is the
> > > > > corresponding format name to pass into wl_shm_pool.create_buffer as
> > > > > the format argument. wl_shm would not advertise these format names
> > > > > itself. We would need to define a WL_SHM_FORMAT_CUSTOM_BASE to
> > > > > reserve some name space for the custom formats.
> > > > > 
> > > > > Or something along those lines.
> > > > > 
> > > > > Of course, this is just working around the pixel format enum wl_shm
> > > > > has anyway, so if you would be willing to take more exotic pixel
> > > > > formats into the wl_shm format enum, this extension would not be
> > > > > needed.
> > > > > 
> > > > > Does any of this sound acceptable for upstream Wayland, or do you
> > > > > think we'd better just write a new wl_buffer factory with
> > > > > everything we need?
> > > > 
> > > > There's no reason not to just copy over the drm formats (which wl_drm
> > > > already uses) to the wl_shm enum.  I don't think we need an elaborate
> > > > scheme for work-in-progress formats, lets just add the full list and
> > > > then just advertise the ones the compositor supports.
> > > > 
> > > > The main reason for not doing that originally was that I didn't want
> > > > to overwhelm clients with a huge list of formats that they would have
> > > > to pick and choose from.  But as long as ARGB32 is always available, I
> > > > don't see a problem in advertising extra formats.
> > > 
> > > Ok, cool. And we can just document for wl_shm, that pre-multiplied ARGB
> > > 32-bit-per-pixel, and XRGB, should always be present, and others are
> > > optional, right?
> 
> Yes.
> 
> > > How do we define pre-multiplied alpha or not? Does it work to say that
> > > everything having an alpha channel is pre-multiplied, including ayuv?
> > > I'm sure there are cases where one would prefer non-pre-multiplied due
> > > to practical considerations, but do we care?
> > > Or should we go with the fourcc definition, which I guess would not
> > > have pre-multiplied?
> 
> We did have non-premultiplied in the past, but it's a really awkward
> way to store transparent pixels and not worth the complexity.  So all
> alpha formats are (continue to be) pre-multiplied.  I don't know about
> AYUV, let's figure out what that means when we hit that case.

Alright.

> > > And how about the conflicts? The current wl_shm formats named
> > > "argb8888" and "xrgb8888" are also used in the wl_drm set but
> > > with different numerical value. And maybe also conflicting on
> > > pre-multiplication?
> 
> We'll just accept both the current 0 and 1 values as well as the
> fourcc values for the formats.  There's no overlap.

Yeah, but there is overlap in the names. Should the protocol definition
of argb8888 and xrgb8888 stay as 0 and 1, and add new names for the
fourcc variants; or should argb8888 and xrgb8888 values in the protocol
be changed into the fourcc values?

And if we do the latter, should we add the 0 and 1 values with new
names like argb8888_wl and xrgb8888_wl, or should we only document,
that compositors should also accept 0 and 1, and map them to the
argb8888 and xrgb8888 formats?


Thanks,
pq