[PATCH RFC] dt-bindings: display: document display panel occlusions

[Caleb Connolly]

On 10/10/2023 23:53, Piotr Masłowski wrote:
> On Tue Oct 10, 2023 at 10:36 PM CEST, Caleb Connolly wrote:
> 
>>> So why am I writing all of this? Well, the problem I see is that any
>>> shape-based approach will likely suffer from both accuracy and
>>> complexity issues. Describing curves is hard and processing them is
>>> not something that should be included in e.g. whatever handles VTs.
>>
>> My proposal here doesn't require processing curves or doing any
>> complicated calculations. If you know that the display has a 30px radius
>> on all corners, you can adjust the VT to not use the top 30px of the
>> screen and it will start exactly where the radius stops.
> 
> Just a nitpick, but on a round smartwatch this approach will give you
> a $width × 0px famebuffer ;D

right right, it's a little more complicated, but not more or less with
either approach.

[...]
>>> - gathering the data is very straightforward – just light the relevant
>>>   pixels one-by-one and check if you see them
>>> - pixel-perfect accuracy is the default
>>
>> I think it would be fairly straightforward to do this for curve data
>> too. You just bump the radius up/down until it looks right, or "good enough"
> 
> Well, different people will have different standards and what might be
> good enough for some will annoy others. I'm not saying that we need to
> be perfect at all cost, but if there's a relatively easy way to cut down
> on inconsistency, it might be worth taking.
> 
> Additionally, having a very clear-cut quality indicator could remove a
> whole class of bikeshedding options. (speaking of the devil xD)
> 
> 
> The problem I have with curves is, the more 'correct' you make them, the
> more convoluted they become. Like take for example the corners. Rounded
> corners are circular right? But what if someone makes them 'squircular'?
> Well, they are usually quite small anyway so maybe it will work out.
> 
> But then what about a smartwatch with big, squircle-shaped display? Bam,
> now you need to complicate how corners are handled.

They don't need to be correct, you don't need to complicate it, you just
need a value that plays nice. When it comes down to it you're much more
likely to be constrained by UI layout limitations by not being able to
model the precise corner curves of your device. The difference between a
circular and elliptical arc is negligible in all real world use cases.
> 
> But also: are rounded coners typically circular? Just now I've thrown a
> OnePlus 6 (thank you so much for the great mainline support btw!) onto a
> flatbed scanner. While a circle fits decently well there it's not really
> a perfect fit, so maybe they went with a different curve after all.
> 
> That being said, imperfection isn't my main issue with curves. It's all
> the non-discreteness they bring to the table. As in, you now need to care
> about approximations, rounding, imprecise measurements and so on.

My point is that actually you don't. Other than for animated visual
effects (where an undersized curve would absolutely be acceptable) I
can't conceive of a situation where plain triangles wouldn't be
suitable, again slightly undersized of course.
> 
> 
>> I think the unfortunate truth is that approximating notches and rounded
>> corners exclusively with regular arcs at the cost of pixel accuracy is
>> just such a no-brainer. Pixel masks would be pixel accurate, but there
>> is no benefit compared to a slightly underfit curve.
> 
> Frankly, I don't see any significant cost here. It's very easy to gather
> and rather easy to process.

Unless I'm mistaken, it would mean that for "odd" shapes like rounded
corners, the number of values you need to record would be directly
proportional to the number of rows. You can do some optimisations, but
that worst case is really not great. Especially when the arc
approximation requires a single value and covers all the same usecases
as well or better.
> 
> Let me think about it… The most common operations I see people actually
> doing with this data would be:
> 
> * letterboxing – easy with both curves and masks
> * ensuring padding | margins for icons on the screen – with curves you
>   can use a formula, but won't it be easier to just count pixels anyway?

You'd probably want your status bar icons to be equi-distant from the
top and side, at least that's what my Android phone does. But hey, maybe
you don't, we can't bikeshed UX design all day but I'd think a simple
radius is gonna be easier to deal with than a pixel mask in most cases
(especially considering that most UI frameworks don't work in the pixel
space anyway because of scaling).

> * routing a spline along the border – like if you want to have some
>   periodic pattern drawn there, it's probably a bit easier to do with
>   curves
> * drawing something at a constant distance from the border – with curves
>   you can again use exact formulas. But isn't that an overkill really?
>   I'd think most people will go for something like a morphological
>   dilation instead

Right right
> 
>>
>> Any program which wanted to make use of the curve information would have
>> to run a whole curve fitting algorithm, whereas there simply aren't any
>> programs which need to know about display occlusions to a pixel-accurate
>> level. For padding a status bar you do a single trig equation, for
>> avoiding the notch in fullscreen you would query the DRM subsystem which
>> presumably would export the dimensions of a letterboxed "usable size".
> 
> What could the curve information be needed for though?
> The more I think about it, the less value I see in it really. Like, if
> you're adjusting the screen contents, pixels _are_ the smallest unit you
> need to concern yourself with. The only thing that comes up to my mind
> is if you were to animate the cutout shape somewhere else, zoomed in.
> And thats already a quite contrived scenario.

GNOME doesn't even use real pixels, macOS doesn't use real pixels. When
scaling comes into play the advantage is lost either way.
> 
> Is there actually any use case that instead of ending up with pixels
> either way, would be better served by a (possibly inaccurate) curve?
> (future me here: that spline-along-the-border from earlier I guess)

I feel like the burden of proof still lies with you here. I feel much
more comfortable with a handful of easy to reason about and explain
values (that can be easily reviewed by maintainers) over a blob of data
that grows with the resolution of your display.

Thanks,
> 
> --
> Cheers,
> Piotr Masłowski

-- 
// Caleb (they/them)