[systemd-devel] [Tracker] How to use cgroups for Tracker?

[Lennart Poettering]

On Thu, 23.10.14 00:52, Philip Van Hoof (philip at codeminded.be) wrote:

> > Don't try to work around limitations of kernel APIs by
> > implementing inherently not scalabale algorithms in userspace. I
> > mean, you implemented something that scales O(n) with n the numbers
> > of dirs. That's what you need to fix, there's no way around that.
> > Just looking for magic wands in cgroups and scheduling facilities
> > to make an algorithm that fundamentally scales badly acceptable is
> > not going to work.
> 
> The problem with allowing unprivileged processes is that a misbehaving
> process will cause the kernel to buffer events for it, forcing the
> kernel to dynamically allocate memory. Not something kernel or inotify
> developers will like a lot.

This is fixable, by enforcing a size limit on the queue. As the limit
is hit the algorithm should coalesce queued events based on
subtrees. For example, if one event for /foo/bar/buzz and one for
/foo/bar/waldo is queued, and the queue hits its limit, both are
replaced by an entry for /foo/bar which is marked with a new flag that
some event was lost below this subtree. For clients this would then
mean that when they receive this they must rescan that specific
subtree again, but not the whole tree.

It's a simple algorithm, the max number of entries stays fixed, but
perfomance doesn't go completely horrible when the limit is reached.

Such a scheme should be implemented in fanotify on the kernel side.

Lennart

-- 
Lennart Poettering, Red Hat