Question about the future of Xorg

[Carsten Haitzler]

On Fri, 13 Jun 2025 17:37:59 -0400 (EDT) Vladimir Dergachev
<volodya at mindspring.com> said:

> 
> 
> On Fri, 13 Jun 2025, Carsten Haitzler wrote:
> 
> > On Fri, 13 Jun 2025 11:41:54 -0400 (EDT) Vladimir Dergachev
> > <volodya at mindspring.com> said:
> >
> >>
> >> I am using KDE right now, and the kwin compositor has 3 choices for "keep
> >> window thumbnails":  "Never", "Only for shown windows" and "Always". And,
> >> of course, I can switch compositor off with Alt-Shift-F12.
> >
> > this is in x11?
> 
> Yes. Probably old - I am keeping it on Ubuntu 20.04 until the search 
> results get published, this way I have the same compiler and tools I 
> started the search with. My other computers use newer software.
> 
> >
> >> I tried setting "Always" and the system bogs down completely. So I usually
> >> have it with "Only for shown windows". Once every few weeks I am doing
> >> something particularly demanding of it and I just switch the compositor
> >> off.
> >
> > wth do you run and do to be that bad. i have NEVER seen anything like
> > this... not on my laptops with integrated gpu's... not on my desktop with
> > discrete...
> 
> This is a Dell, it has an integrated Intel GPU and and NVidia one. I 
> usually don't use NVidia GPU - too much heat and no visible benefit.
> 
> The NVidia chip is Geforce MX 130, 2 GB RAM.

what intel chip/gpu?

> > maybe it's a kde problem/thing? like i can have 50+ windows with zero issue.
> 
> If you have a fresh reboot it is fine, but if you keep it running for 
> months it gradually gets slower.
> 
> I found that restarting kwin and restarting plasmashell helps, and also 
> occasionally I kill firefox and restart it. The latter is a nuisance, 
> because while it does try to restore windows and tabs it does not restore 
> all of them.

this smells of some kind of leak? where? ... dunnos.

> Btw, if you have the same problem there is a setting in about:config 
> that lets you increase timeout for reaching website. When you restart 
> firefox and it tries to open 300 windows with 10-15 tabs each, the urls 
> will timeout too early in default configuration. Change that setting fixes 
> it (mostly).

i never restore tabs... when i close my browser.. i'm done. :)

> > currently have 27 and nothing is bothered at all. admittedly that's windows
> > spread across multiple virtual desktops... but i know i do NOT make any
> > effort to "undirect" windows to remove buffers for windows on other
> > desktops. i do know that they are completely nuked from the compositing
> > scene graph and get no render or even context change effort as the scene
> > graph skips them... but the buffers exist and use video ram. in today's
> > world that is not that much... unless you really e beginning to run very
> > low on video ram or "regular ram" for integrated gpu's - i don't see why
> > this matters?
> 
> What's your screen resolution? Mine is 3840x2160, so a single full screen 
> buffer should be 33MB. A few hundred buffers and you would be out of RAM 
> on discrete GPU (and way earlier on my NVidia chip).

2560x1440. i do have 16gb ram on the card - but ... how many windows do you
have normally around... and are they all "maximized" ? as it's a window that
consumes a buffer.

> > what's your working style? put 50+ windows on a single desktop and "alt+tab"
> > between them?
> 
> I got 8 virtual screens, and a bunch of windows on each of them. Most are 
> firefox.

and how many ffox windows?

> What happens is that I work on something and it usually involves 10-20 
> windows, but then I have to pause or wait for one reason or another and 
> I switch to something else.

10-20 is not too much. and at your fullscreen 33m per window that's ~300-700m -
so not a problem for memory usage. even if you run out of vram the gpu can
migrate some buffers/textures to system ram and map them over the pcie bus as
render targets. its a bit slower. it might not migrate and just alloc new
buffers there never migrating lesser used ones off. that'd be a "poor caching
algorithm" :)

> It is very convenient that I just leave things as they are and then I just 
> come back and pick up where I left off. Often I minimize the windows 
> because I only got 8 virtual screens - its a compromise for space in KDE 
> panel.
> 
> Looking at xrestop right now, kwin has 69 pixmaps and uses 2.7GB RAM, 
> while firefox has 276 pixmaps and only 300MB RAM. There are a bunch of 
> other windows, mostly konsole. Compositing is on right now. I am pretty 
> sure firefox has a lot more than 69 windows.

you said like 10-20 windows above? 69 windows? or 69 tabs? i just opened 18
maximized terminals. also have this email client (2 windows) plus hexchat +
chromium ... e uses about 330m with 25 pixmaps but reality is pixmaps are
really only for the windows and nothing else - everything else is rendered
inside the compositor with gl (or software) and thus is part of texture
atlases etc. nvtop says e uses 440m of video memory which makes sense
(2 screens, each 2560x1440). it'd be 270m just for the terminal textures mapped
in (shared between x and the client and compositor), as well as wallpaper,
other icons/buffers and backbuffers for rendering (90m for backbuffers for both
screen if triple buffered). so all in all 440m doesn't sound wrong to me.

> > i just opened 30 terminals and maximized them on the same desktop... i can
> > alt-tab between them without anything bogging down. i can open a 31st small
> > terminal on top of these 30 and drag it about smoothly. my gpu is a 6800xt
> > - so a good gpu but from like 5 years ago. it's not some top of the line
> > thing.
> 
> Interesting!
> 
> >
> > all my terminals are the usual solid (not transparent type) so technically a
> > good renderer will actually skip and avoid rendering all but the top most
> > maximized window... maybe this is kde's problem?
> 
> It is an older KDE, my desktop with newer Kubuntu is smoother, but then it 
> has more memory and newer GPU.
> 
> >
> > so either your workflow is something i'm not reproducing, you have one hell
> > of a potato of a gpu or cpu, or ... this is a kde thing?
> >
> >> The only thing I lose is the "present windows" effect, but instead I just
> >> get a list of window titles which is perfectly fine.
> >>
> >> So I am hoping other people use Wayland to get things done, and resource
> >> requirements are not as dire as you describe - otherwise it would be a
> >> complete no-go for me.
> >
> > they are the same really as a composited x11. no real difference at all.
> 
> Naively, if all the windows always have a buffer than 8GB GPU can only 
> afford 242 4K windows. And you don't get more memory in consumer GPUs 
> because they will then compete with AI market devices.

??? the default for consumer gpu's is 16g these days. 8g is a low end "cut
price" gpu. the latest gen of gpu's is now more pushing towards 24/32g.

> I am, of course, hoping that actual Wayland compositors are smarter than 
> this and dismiss buffers for windows that are not being rendered at the 
> moment.

well this does require co-operation between compositor and client. the
compositor can release its handles on the client buffers - but clients will
need to voluntarily also give them up when idle.

> > they can;'t be implemented as standard separate wayland apps. they have to
> > be tightly bound to the compositor. so it's either a compositor feature
> > built-in, or some extension/tool/plugin for your compositor... well if you
> > think of a tool like a live magnifier around the mouse cursor wherever it
> > goes...
> 
> So there has got to be a way to let teleconferencing apps work without 
> being part of the compositor. Lots of people need to share slides.

keep reading.

> > if its a 1-off "screenshot then display a copy of it and just scale that up"
> > then there are wayland protocols for that - but the idea is that
> > screenshotting protocol access will be limited and a compositor may do a
> > very android/ios thing of ask you to grant permission first.
> 
> I hate the permission stuff on android. The worst is that they've taken to 
> removing permissions from apps when you don't use them. So you have some 

so you're happy with rogue games you run screenshotting your browser with
banking details and sending it back to home? :)

that's the point of this. the point is that the display system should stop
being a leak of info/security. it cant force you to sandbox apps... but it can
STOP being the problem that makes sandboxing ineffective.

> app that you use once a month and then you have to debug why it does not 
> work. Especially sucks if you need to take a quick snapshot with a thermal 
> camera or a similar tool.

and this is the current problem area - how to grant permission AND keep it
granted persistently. you'd need some kind of tagging model - maybe smack,
where you then assign a new smack label to a newly installed app and the
compositor grants access for protocol X to smack label Y. at this point there
isn't really an agreed on security model for apps this way that i know of. the
compositor could just always grant permission without a dialog - i assume most
compositors would have a checkbox in settings you can toggle to tell it to go
away and stop bothering you if you don't care. if it doesn't - ask your
compositor authors to add it, or send them a patch, or change compositors.

> A lot of it is pointless anyway - the apps that do shady stuff will find a 
> way anyway, and the users of good apps are just getting annoyed.

and on the flip side if you go to a lot of effort to sandbox an app in a
container or a smack label (read up on them) that then quite effectively limits
that app - the display system is a massive leaking hole you cant plug... and
this is one of the things wayland wants to address and does.

> > well some like neko etc. -= yes. per compositor. accessibilty features would
> > also be "per compositor" and i'd lump xmag in that. teleconferencing (zoom
> > etc.) will be as ai described - limited access "screenshot" protocol -
> > pipewire offers this. there's another wl protocol based one (imho
> > better/right thing to do).
> 
> Screenshot is not perfect - you really want a video, so you can play an 
> animation.

a screenshot is just 1 frame of a video... that is how zoom, teams and every
video conf app works now today. they keep taking screenshots repeatedly and
quickly. that's how they can "share my screen" over that video conference...
they grab these frames then encode them into a video stream - on the fly. they
do that in x11 today...

> Another useful tool is screen recorder like vokoscreenNG - with it you can 
> record your talk to be played back later. Again, you want the ability to 
> record a video of the screen.

guess how those work too... :) see above :)

> >> (PS xmag/kmag are really useful if you are looking at 32" 4K monitor via
> >> VNC from 15" laptop. All the fonts are tiny)
> >
> > i know.. tho you should just bump your fonts up .. though wayland can scale
> > much better than x could due to being bale to individually scale different
> > client windows and they don't need to know (because they don't get absolute
> 
> No, no, I was talking about doing VNC to another computer. The fonts are 
> normal on that computer's monitor, but when I display fullscreen on mine 
> they are too small for some apps. So you don't want to change fonts on the 
> remote computer, but you occasionally want to magnify what you see on the 
> laptop.

then this is certainly something your vnc viewer should support IMHO. as how
much you want to scale THAT session may vary from target to target it is
connecting to... and it should remember such scale settings machine by machine
you register/connect to. the compositor has no clue what is inside that app's
window. in wayland or x11. it's the app's business.

> >>> doing this in optimal ways. kms abstracts this so it can be done - but
> >>> different compositors may do different things.
> >>
> >> Well, the 2d library does exist and it is called X11. You also have higher
> >> level Qt and gtk, and even Tcl/Tk :)
> >
> > well tbh the 2d rendering in x11 is likely now backing through glamor which
> > uses the 3d unit... :) likely... depends. :) but there has been no lower
> > level api/abstraction "direct to device" beyond mesa + drm.
> 
> I see. Maybe it is not needed, just do it on the CPU. The CPU memory 
> bandwidth and vector processing are pretty good nowadays, to the point 
> that you might argue you don't need a GPU for some uses - just maybe CRTC 
> to talk to the monitor. Or just send it over network.

the gpu is better. 2d accel units are often quite limited today - thus the gpu
just can do so much more with a single api (gl/vulkan). software rendering is
doable but still - a gpu does do better. but .. sw likely is enough for older
style rendering/ui's

> If you look at latest Xeons and Epycs they have bandwidth that competes 
> well with consumer GPUs and have TFlops to match. Priced through the roof, 
> of course.
> 
> >
> >> Speaking of which, it would be fun to see what happens if one implements
> >> compositor with Tcl/Tk. If only I had more spare time..
> >
> > in theory you could - assuming they had access to all the libraries and
> > api's. :)
> 
> Right, wrap libwayland with access calls and create something like the 
> canvas widget that paints directly to hardware.

yes - also opengl, libdrm and so on etc. etc.

> best
> 
> Vladimir Dergachev
> 
> >
> > -- 
> > ------------- Codito, ergo sum - "I code, therefore I am" --------------
> > Carsten Haitzler - raster at rasterman.com
> >
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
Carsten Haitzler - raster at rasterman.com