per-plane LUTs and CSCs?

[Laurentiu Palcu]

Hi Pekka,

On Thu, Sep 10, 2020 at 11:50:26AM +0300, Pekka Paalanen wrote:
> On Thu, 10 Sep 2020 09:52:26 +0200
> Daniel Vetter <daniel at ffwll.ch> wrote:
> 
> > On Thu, Sep 10, 2020 at 10:25:43AM +0300, Pekka Paalanen wrote:
> > > On Wed, 9 Sep 2020 13:57:28 +0300
> > > Laurentiu Palcu <laurentiu.palcu at oss.nxp.com> wrote:
> > >   
> > > > Hi all,
> > > > 
> > > > I was wondering whether you could give me an advise on how to proceed further
> > > > with the following issue as I'm preparing to upstream the next set of patches
> > > > for the iMX8MQ display controller(DCSS). The display controller has 3 planes,
> > > > each with 2 CSCs and one degamma LUT. The CSCs are in front and after the LUT
> > > > respectively. Then the output from those 3 pipes is blended together and then
> > > > gamma correction is applied using a linear-to-nonlinear LUT and another CSC, if
> > > > needed.
> 
> Hi,
> 
> hmm, so FB -> CSC -> LUT -> CSC -> blending?
> 
> Is it then
> 	blending -> LUT -> CSC -> encoder
> or
> 	blending -> CSC -> LUT -> encoder?

The DCSS pipeline topology is this:

FB1->CSC_A->LUT->CSC_B-\
FB2->CSC_A->LUT->CSC_B-|-blender->LUT->CSC_O->encoder
FB3->CSC_A->LUT->CSC_B-/

Basically, CSC_A is used to convert to a common colorspace if needed
(YUV->RGB) as well as to perform pixel range conversions: limited->full.
CSC_B is for gamut conversions(like 709->2020). The CSC_O is used to
convert to the colorspace used by the output (like RGB->YUV).

> 
> Are all these LUTs per-channel 1D LUTs or something else?

LUTs are 3D, per pixel component.

Thanks,
laurentiu

> 
> What ever the KMS UAPI for these is going to be looking like, it
> obviously needs to define all this. So I'm guessing we need to define
> the abstract color pipeline of KMS in general that includes everything
> any driver might want to expose. Then each driver picks the blocks in
> the pipeline it wants to expose and the other blocks are assumed to be
> "identity transform".
> 
> Without such general abstract color pipeline defined and documented it
> is very unlikely IMO for generic userspace to make use of the driver
> features.
> 
> All blocks must also default to identity transform. A block
> unimplemented by a driver is the same as a block not used or understood
> by a KMS client.
> 
> Userspace that does not understand all the blocks will need to use the
> "harmless default values". This then ties in with what I've discussed
> with danvet before: when you are VT-switching from one KMS client to
> another, how do you reset the full KMS state (including blocks you
> don't understand) to the same state you had before. The other KMS
> client may have messed them up while you were gone.
> 
> All this default value talk is to ensure that the abstract KMS color
> pipeline can be extended without breaking existing userspace.
> 
> ...
> 
> > > > So, how should I continue with this one? Any pointers?  
> > > 
> > > Hi,
> > > 
> > > I can't help you, but I can say that we are currently in the process of
> > > designing a color management and HDR extension for Wayland, and I'm
> > > sure in the long term I would like to use per-plane color space
> > > transformation features of KMS in Weston eventually.
> > > 
> > > IOW, one more userspace that is going to be taking advantage of such
> > > features as long as they are not too driver-specific.  
> > 
> > Personally I think best to wait for userspace to come up with color
> > manager protocols, that should give us a much clearer idea of what the
> > kernel interface should look like. Since hw is pretty special in this
> > area, I expect we'll have to do a bunch of impendendance mismatching in
> > the kernel anyway.
> 
> Speaking of that, where should we scream if we feel like we are missing
> KMS properties to get the best out of color management and HDR in
> Weston, assuming we're not kernel devs?
> 
> Who to Cc?
> 
> We currently have intel and AMD hardware at hand if that makes any
> difference.
> 
> 
> Thanks,
> pq