[systemd-devel] [RFC 0/6] A network proxy management daemon, systemd-proxy-discoveryd

[Marcel Holtmann]

Hi Lennart,

>>>>> As it has been discussed in the systemd hackfest during the Linux Conference
>>>>> Europe, one daemon could centralize the management of all network proxy
>>>>> configurations. Idea is something user can do per-application (like in
>>>>> firefox for instance) or broader (per-DM like in Gnome), user could do it
>>>>> once and for all through such daemon and applications would then request it
>>>>> to know whether or not a proxy has to be used and which one.
>>>>> 
>>>>> As a notice, this is nothing new. Such standalone daemon has been already
>>>>> done by the past, pacrunner. systemd-proxy-discoveryd will more or less
>>>>> implement the same ideas with improvements. It will get rid of big JS
>>>>> engines like spidermonkey or v8 which are overkill for the tiny PAC files
>>>>> to be executed on, for instance. From pacrunner experience, APIs will be
>>>>> also improved.
>>>> Hi,
>>>> 
>>>> the idea of having centralized proxy config is certainly nice. But the
>>>> PAC files make me shiver. So the first question: is it really necessary
>>>> to support PAC files? 
>>> 
>>> Yes, it's kinda necessary. PAC is pretty widely used in corporate
>>> setting. Windows does the WPAD stuff (the protocol to discover PAD) in
>>> all its versions since a long time. In fact, it immediately issues the
>>> wpad requests as first thing when you plug in an ethernet cable, in
>>> addition to DHCP.
>>> 
>>>> Are they widely used in corporate setting or something?
>>>> Is there no saner standard?
>>> 
>>> Nope, not really, I fear.
>>> 
>>>> 
>>>> If the PAC files must be interpreted, I think this is the hardest
>>>> part.  FindProxyForURL is started for every request, potentially
>>>> hundreds of times per second and more. This means that starting a
>>>> process per invocation is out of the question, and even starting a
>>>> thread per invocation seems to be too much. But each call fall into an
>>>> infinite loop and hang. So the run time of FindProxyForURL should be
>>>> bounded. FindProxyForURL can also resolve names over the network,
>>>> which would best be done asynchronously.
>>>> 
>>>> Things in systemd are usually implemented through poll loops, which
>>>> makes it easy to support thousands of concurrent "jobs". I'd think
>>>> that this would be the best option here too, with a number of "workers"
>>>> executing FindProxyForURL()s and stopping when name resolution is
>>>> requested and continuing when the name is resolved.
>>> 
>>> Well, it's Java script code. People can just add code like "for (;;);
>>> ", and we must be able to cancel the lookup then safely. I doubt
>>> that's cleanly doable with either thread-based or event loop based
>>> solutions. I am pretty sure a worker process logic is the way to
>>> go. The worker process should get the PAC data when it is forked off,
>>> and then read FindProxyForURL data from a pipe and output the result
>>> on another, or something similar, and easily sandboxable...
>> 
>> are you sure that you are not overthinking this? I think that
>> duktape actually allows just canceling the JS engine execution, no
>> matter what operation it runs. So you could just cancel the JS
>> context.
> 
> How is this implemented in duktype? I mean, cancelling threads is
> fricking awful. POSIX thread cancellation is awful, and I am pretty
> sure NSS calls are incompatible with it anway. Which means you have to
> check flags after each javascript instruction -- which however doesn't
> really work too well either, since NSS calls will block as long as
> they want, hence you couldn't use this to cancel those... 

I am not saying that we cancel the thread. I know that this is painful. I am saying that we just cancel the duktape context and its execution, then the thread would just yield all by itself. So I think the question is if a master thread could just tell the duktape context to quit. That is something we might want to figure out.

> I really would prefer doing this out-of-process. Then we can kill the
> stuff, regardless what it is doing, without interfering with anything
> else. Heck, we can even let the kernel help us with the timeout thing
> with RLIMIT_CPU...

We would need to define some IPC and managing the pool of these processes. Of course this is possible since almost any browser does it this way. It however sounds like a lot of effort and complexity if we can do the same with threads and just cancel the duktape execution instead.

Regards

Marcel