Question about the future of Xorg

[Vladimir Dergachev]

On Sat, 14 Jun 2025, Carsten Haitzler wrote:

> On Fri, 13 Jun 2025 20:34:59 -0400 (EDT) Vladimir Dergachev
> <volodya at mindspring.com> said:
>
>> Intel Core i7-8550U
>
> not that old really. my old laptop is that. no dedicated gpu though.
>
>>>> I found that restarting kwin and restarting plasmashell helps, and also
>>>> occasionally I kill firefox and restart it. The latter is a nuisance,
>>>> because while it does try to restore windows and tabs it does not restore
>>>> all of them.
>>>
>>> this smells of some kind of leak? where? ... dunnos.
>>
>> Could be a leak, could be entropy (like fragmentation in memory).
>
> it could be .. though i dont think it'd do what you describe.
>
>>> have normally around... and are they all "maximized" ? as it's a window that
>>> consumes a buffer.
>>
>> Firefox is usually maximized. I think right now I have around 120 Firefox
>> windows.
>
> and there is where i go "wtf? why?". that's nuts. i would say - looking at most
> peoples screens and workflows. you have a totally out-there workflow there.
> it's not common. not with that many windows.

Well, I don't think its unreasonable - let me explain.

One of the things I do which is publicly visible are searches for 
gravitational waves.

So I have my search code and when I am developing some part of it, I might 
need to lookup things like Bessel filters or Hann windows and so on. Most 
of the time I know the math by heart, but the formulas need to be exact so 
I double check with Wikipedia and other places. Might pull up some papers 
- Firefox windows with PDFs use more memory. This is easily 10-20 windows.

After I make a first pass at the code I need to test it. Test simulations 
take at least a few hours on a cluster, could be a couple of days.

So I leave this and do something else, for example work on CUDA 
optimization - another 10-20 windows, another hours or days to test.

My results are distributed using libMVL 
(https://github.com/volodya31415/libMVL), so, of course, I have a few 
windows for that.

If I get a paper for review, I need to read and then to lookup other 
relevant papers. The review takes several days because my rule is to read 
the paper and then leave it be - this allows me to understand it better 
and to think of angles that I could have missed.

If I am a designing a part, I would have FreeCAD open full screen and a 
bunch of pages on material properties or part dimensions.

And then there is a bunch of stuff I can't talk about.

So now, suppose every mini-project I am working on only uses 12 windows.
10 projects makes 120 windows. 10 projects is not much.

>>> be a "poor caching algorithm" :)
>>
>> It's 10-20 times the number of different things I work on. So it adds up.
>
> i mean i might have 1 virtual desktop with maybe 20 terminals - when coding and
> what not. but that's it ... i also don't maximize them - i lay them out so they
> are like tiled side by side and split vertically in a kind of step-like grid
> with some taller, some shorter but all 80 col wide. no overlap so i can see a
> lot of context.
>
> but my point is - even where i have a lot of windows, they dont cover a lot of
> SPACE. i've observed others and they mostly live life with maximized windows
> and switch - but  they often only have maybe 2-5 - maaaybe 10 at most. it's
> rare to have more and then only temporarily.

I think you are thinking from perspective of a single computer and a 
single screen. My desktop has two 4K monitors - super handy. For people 
doing finance it is not uncommon to have 8 screens.

I very much believe in looking at the data with my own eyes before I 
start writing algorithms.

>
> my point is - your usage pattern is "rare". it's also going to be the most
> memory hungry one in a composited world - if so then i'd suggest you upgrade
> with a lot more ram. :)

It might be rare now, but it might be what people would be doing in 5 
years.

It is really about scalability - if the system you are designing can 
barely scrape by with 100 windows on one 4K screen on modern hardware, it 
will have problems with larger screens in the future.

Imagine consumers using organic led wallpaper to cover walls and expecting 
to put windows with pictures, videos and homework there. Suppose you have 
low-resolution cheap OLED wallpaper, 96dpi. 5 meters (16ft) will translate 
to more than 18000 pixels!

The X that we like was designed for scalability. It run perfectly fine 
with dozens of windows on computers with 8 *mega* bytes of RAM total.

X is fantastic now and the only thing slowing it down is proliferation of 
javascript and non-scalable coding practices.

>
> remember compositing is also pretty much a compute vs memory tradeoff. if you
> have a buffer of a window and always have it you can avoid redraws every time
> its shown/exposed. you just use the data you already have and the cost of just
> reading that data is almost always a lot less than the cost of re-computing it.

Right, but even laptops now have 8-16 cores, each of which does billions 
of *vector* operations per second. Why is it a problem to redraw for a 
rare operation of switching between windows?

You could also use scaled down buffer - just compute what the matrix would 
be for the "present windows" effect, and allocate buffers for the tiny 
windows it uses. This way you never allocate more than screen-size buffer.

>
>> 18 terminals are too small a test. Does it work if you open 1000 ?
>
> well i couldn't in my std way - i'd run out of x client fd's - the xserver
> limits x client count to 128... :) i'd have to enable single process mode in my
> terminal to keep it a single client.

Hmmm - scalability issue. We need to fix that in X :)

One can easily test with Tcl/Tk, here is an example:

#!/usr/bin/env wish
#

set N 200

for { set i 0 } { $i < $N } { incr i } {
        toplevel .test$i -width 1024 -height 768
         }

If you want to make test windows more fun you can use "label", "button" 
and "canvas". You can set titles with "wm title". There are manpages for 
Tcl/Tk functions, but they are often in a separate package, not installed 
by default.


>
>>> ??? the default for consumer gpu's is 16g these days. 8g is a low end "cut
>>> price" gpu. the latest gen of gpu's is now more pushing towards 24/32g.
>>
>> They are all "cut price" right now - you cannot buy 24/32gb, at least in
>> stores near me. The companies do this on purpose for market segmentation.
>
> the higher end - nv's rtx5090's are at 32gb now. as i said - pushing there. the
> mid to lower tiers like 5070 are 12gb or go back a gen to 4070 - same thing.
> the rx9070 is mid-to-high with 16gb the lower 9060 is also 16gb unless its the
> absolute lowest end which is 8gb. so 3/4 of the current gen amd cards are 16g.

RTX 5090 might be worth the risk of melting down your computer if you are 
playing games with raytracing or having fun with AI.

Opening plain old 2d windows should work fine on any computer with 
integrated GPU.

>
>> And on a notebook the RAM and bandwidth are even smaller.
>
> but bandwidth here shouldn't be the issue - you wont be rendering 100's of
> windows AT ONCE. you will only see a small selection of them on screen at any
> time and thus only some small subset need access for a re-composite every frame.
>
> this is why i am wondering if there's "something not right?". like is it kde's
> pager or something rendering every window on every desktop every time you
> switch focus - no elimination of obscured windows etc. ? and the pager might
> use the original full buffer and scale it down of course  every time it
> renders... ?

I think what happens is that the windows thumbnails get updated 
periodically and the updates might be triggerred by windows redraws.

This way you can have a cool effect of seeing a video play in a preview.

The problem is that a lot of windows that you might think are static are 
not.

For example, try the following experiment: open a firefox window, and do a 
google search for anything. You will get a bunch of links.

Do this again in, say, 20 tabs, so each has results of a Google search. 
Now look at your CPU usage - you will see firefox consuming 20-80% of a 
single core, depending on your CPU.

So there is some Javascript running, doing who knows what, on a webpage 
that you think is just a static list of links.

>
>> Also, right now Microsoft is very busy alienating a lot of people with
>> computers without TPM that cannot upgrade to new Windows version.
>>
>> Those people are happily installing Linux and we should not impose
>> requirements of more than 8GB video RAM just to open some webpages.
>
> these people are not going to have 100's of windows :) your workflow is unusual
> for sure.

But they will have older computers with integrated GPUs and 8-16 GB of RAM 
total.

A 4K external monitor is a cheap upgrade. Replacing a computer with a new 
one using 16GB VRAM discrete card is not.

>>
>> This problem only arises on Android and IOS because they are designed for
>> closed source apps and for controlling the user.
>>
>> On Linux there is no such problem as long as you use software you can
>> examine.
>
> the problem is 99.9999% of people don't have time to examine it and never will.
> that includes geeks and developers. i certainly have no time to do that. i will
> not be re-auditing the source for every app i use every update. i won't even do
> it once. all those people windows is alienating and try linux certainly won't
> be doing it either.

No need for everyone to do checks.

This is a perfect business case for a distribution like Debian or Ubuntu 
or Redhat, which offers pre-checked packages, secured with hash sums.

>
> and so i assume you never play games then. or well not on your pc. you will
> never get source for these so sure - you can live your life without games but
> most people won't - they want to have some entertainment. they may use a
> dedicated console. they may use a pc. but it'd be good to know the thing i am
> entertaining myself with and once i have completed its quests and storyline is
> never played again... is not able to do bad things because my display system is
> designed to not allow it.

There are plenty of good open source games. For commercial games you 
really should have a separate computer - both because you might want to 
put a better graphics card or audio there, but also for security.

The problem is that while CPUs have some security flaws (just look at the 
buglist in /proc/cpuinfo), the GPUs are just wide open. Any game that has 
almost direct access to GPU can exploit a bug in the driver (or worse - a 
bug in the hardware) to write arbitrary data to arbitrary place in 
physical memory.

Worrying about it snooping on position of your windows is the least of 
your problems.

>
>> On Android you could improve things immeasurably if open source apps were
>> installed with complete user access to app directory (to check which
>> binary actually shipped) and no permission restrictions.
>
> i'm not going to go down this rabbit hole. :) wayland has a security model that
> does not go trusting everything and everyone by design. in this way it is
> right. spot on. a compositor could just not care and allow everything to
> everyone but the model allows for these things to be optional features and they
> may not work or may be denied by policy and apps have to live in a world where
> they cant DEPEND on these existing like they can in x.

What do you think happens when regular people are presented with a 
permission request? They always click yes. There is no way to tell 
whether permission is right or not.

The only true way to decide whether a permission is to be allowed is to 
see which code will be executed. But if you checked that already, why 
bother the user?

More generally there are two major types of security:

   * the ability to deny others access

   * the ability to assure your own access

Somehow modern developers tend to forget the second and focus too much on 
the first (xkcd/538). The users care about the second and always hit yes.

Also, leaking personal data is not really about your computer. It is about 
other companies (like Equifax) carefully collecting the data and then 
being hacked. You don't just leak one account number like when your phone 
gets hacked, you leak all of them.

>
>>>> app that you use once a month and then you have to debug why it does not
>>>> work. Especially sucks if you need to take a quick snapshot with a thermal
>>>> camera or a similar tool.
>>>
>>> and this is the current problem area - how to grant permission AND keep it
>>> granted persistently.
>>
>> It is a very simple problem - you have an xmag/kmag like app. You examine
>> code. You see it does not send screenshots to some random IP or random
>> country. You install it and use with no restrictions.
>
> that is not simple. 99.999% of people couldn't read the first line of code and
> know what it does. you don't design a display system for 0.0001% of people. it
> just so happens historically no one cared and there was little to no ability to
> isolate processes. hell ye olde windows and macos didn't even have memory
> protection between processes - they could stomp all over each other at a drop
> of a hat. things have evolved.

They don't need to - that's why we have distributions of open source code.

If you want secure you use pure open source distribution for your company.

If you want extra secure you hire outside reviewers and/or have your own 
people review for vulnerabilities in your specific use cases.

If you need commercial app you either sign some contract, or just get 
insurance for eventual break-in. You can also put it on an isolated 
system.

Open source software is premium software - it has been carefully written 
to be readable and checkable. It offers verifyable trust.

>
>>> a screenshot is just 1 frame of a video... that is how zoom, teams and every
>>> video conf app works now today. they keep taking screenshots repeatedly and
>>> quickly. that's how they can "share my screen" over that video conference...
>>> they grab these frames then encode them into a video stream - on the fly.
>>> they do that in x11 today...
>>
>> Yes, but ideally you could do it in such a way as to guarantee a frame
>> every 1/N seconds and also guarantee that a frame is fully rendered to
>> avoid tearing. This is something that I think X cannot do right now.
>
> you don't get that guarantee in x either - indeed - yet you seem happy and it
> works.

Right, you don't - I pointed this out as something that could be 
implemented and become an edge over existing software.

>
>>> much you want to scale THAT session may vary from target to target it is
>>> connecting to... and it should remember such scale settings machine by
>>> machine you register/connect to. the compositor has no clue what is inside
>>> that app's window. in wayland or x11. it's the app's business.
>>
>> Not really - I just run x11vnc on the remote and connect to it. I don't
>> start a new session, and I don't change fonts. Very handy, both to help
>> someone else and to use your desktop when you are away.
>
> not the login session it's looking at but the "viewing session". the VIEWER
> should just scale up pixels then when rendering rather than just dumbly drawing
> them 1:1 and nothing else.

But then they don't fit on the screen. I am doing a VNC from a 32" 4K 
notebook screen into the 15" 4K desktop screen. I need to see the entire 
screen so I can click buttons on the panel or see plots in big windows.

But some text is too small and I just use xmag/kmag to read it.

best

Vladimir Dergachev