[Intel-gfx] [RFC v2 01/22] drm: RFC for Plane Color Hardware Pipeline

[Pekka Paalanen]

On Tue, 26 Oct 2021 11:36:33 -0400
Harry Wentland <harry.wentland at amd.com> wrote:

> On 2021-10-14 15:44, Shankar, Uma wrote:
> > 

...

> >>>>> +
> >>>>> +* Plane CTM
> >>>>> +	* This is a Property to program the color transformation matrix.  
> >>>>
> >>>> No mode property here? Is there any hardware with something else 
> >>>> than a matrix at this point?  
> >>>
> >>> Not that I am aware of.
> >>>  
> >>>> Should we assume there will be hardware with something else, and 
> >>>> have a CSC mode property with only a single enum value defined so far:
> >>>> "matrix"? Or do we say PLANE_CTM is a matrix and if you have 
> >>>> something else in hardware, then invent a new property for it?  
> >>>
> >>> I think this should be good as we have this for crtc with no one complaining.
> >>> There may be hardware with fixed function operation for the CSC but 
> >>> then its not a matrix and it would be pretty hardware dependent, so 
> >>> we can leave that from a generic UAPI.  
> >>
> >> Ok. And if that eventually turns out to be a mistake, it's easy to 
> >> invent more properties.  
> > 
> > I feel that is what would be required. This is just another instance of what we have at crtc level.
> >   
> 
> Would a userspace client ever want to use both CTM and 3D LUT?

The only reason I can think of is to keep the 3D LUT size (number of
taps) small. I suspect that if one can use a matrix to get close, and
then fix the residual with a 3D LUT, the 3D LUT could be significantly
smaller than if you'd need to bake the matrix into the 3D LUT. But this
is just my own suspicion, I haven't looked for references about this
topic.

IOW, it's a question of precision - the same reason as to why you would
want a 1D LUT before and after a 3D LUT.


> We could add a mode property to CTM to allow for a CTM or 3D LUT or
> we could add new properties for 3D LUT support.
> 
> 3D LUT might come with shaper 1D LUTs that can be programmed in
> conjunction with the 3D LUT. 3D LUTs operating in non-linear
> space require less precision than 3D LUTs operating in linear
> space, hence the 1D LUT can be used to non-linearize the
> pixel data for 3D LUT operation.
> 
> FWIW, AMD HW (depending on generation) can do these operations
> (in this order):
> 
> 1) 1D LUT (fixed or PWL programmable)
> 2) simple multiplier (for scaling SDR content to HDR output)
> 3) CTM matrix
> 4) 1D LUT (shaper LUT to non-linearize for more effective 3D LUT transform)
> 5) 3D LUT
> 6) 1D LUT (for non-linear blending, or to linearize after 3D LUT)
> 7) blending
> 8) CTM matrix
> 9) 1D LUT (shaper LUT like above)
> 10) 3D LUT
> 11) 1D LUT (generally for EOTF^-1 for display EOTF)
> 
> Not all blocks are available on all (current and future) HW.
> 
> I sketched a few diagrams that show how these might be used by
> a compositor if we exposed all of these blocks and should
> really try to add some of them to the color-and-hdr docs
> repo.

Yes, please.

That pipeline looks pretty comprehensive.

Btw. how about YUV<->RGB conversion? Where would that matrix go? It
needs to operate on non-linear values, while a color space conversion
matrix needs to operate on linear color values.

> >>>>> +	* This can be used to perform a color space conversion like
> >>>>> +	* BT2020 to BT709 or BT601 etc.
> >>>>> +	* This block is generally kept after the degamma unit so that  
> >>>>
> >>>> Not "generally". If blocks can change places, then it becomes 
> >>>> intractable for generic userspace to program.  
> >>>
> >>> Sure, will drop this wording here. But one open will still remain 
> >>> for userspace, as to how it gets the pipeline dynamically for a respective hardware.
> >>> Currently we have assumed that this would be the logical fixed order 
> >>> of hardware units.  
> >>
> >> If we cannot model the abstract KMS pipeline as a fixed order of units 
> >> (where each unit may exist or not), we need to take a few steps back 
> >> here and look at what do we actually want to expose. That is a much 
> >> bigger design problem which we are currently not even considering.  
> > 
> > I think most of the hardware vendor platforms have this pipeline, so we can implement the properties which include all the possible hardware blocks. If certain units don't exist, the respective properties should not be exposed which will make things easier for userspace.  
> 
> I think the color pipeline should be modeled in a way that makes
> sense from a color science standpoint and in a way that makes sense
> for compositor implementations. Fortunately HW design generally
> aligns with these intentions but we should be careful to not
> let HW design dictate KMS interfaces.

I'm so happy to hear that!


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/intel-gfx/attachments/20211027/61743586/attachment.sig>