[avahi] Resolving many ( > 200 ) items seems to lead to apparent congestion in the dbus

[Daniel Wynne]

Lennart Poettering schrieb:
> On Fri, 15.05.09 10:17, Daniel Wynne (daniel.wynne at mobotix.com) wrote:
>
>   
>> Hi All!
>>
>> Since we sometimes deal with more than 250 hosts, we often run into
>> timeouts from the Resolvers. For now, we immediately try again until all
>> Hosts are resolved. This can take up to 10 mins...
>>
>> After restarting the application we end up resolving only about 50% of
>> the hosts. A restart of the dbus allows the application to find all
>> hosts again.
>>
>> Is there a knwon issue about the dbus system that "free"-ed items remain
>> for an amount of time? Is there an easy workaround for this? 
>> Do you think it would be good practice to establish an queue system to
>> avoid the resolver-timeouts?
>>     
>
> How are you using this? Are you resolving 250 hosts one after the
> other?
>
> Avahi enforces limits at all places, i.e. on the cache, timeouts,
> queries sent out and so on, to make sure we don't flood the network,
> and that Avahi cannot be used for a DoS attack. 
>
> mDNS is intended to be used in smaller networks, up to 200 hosts or
> something like that. I.e. small offices, homes. If you have larger
> networks than that mDNS is not the protocol you should be using.
>
> Lennart
>
>   
Hi Lennart!

Though its not the standard case, we deal with more than 250 hosts for 
testing purpose. We want to resolve all of them, either without spamming 
the DBus-System or crash into any Avahi-limitation.
First, we tried resolving everything right on the way, but this lead to 
apparent DBus-congestion, which could only be solved by restarting the 
whole service. After that we tried a queued system which allows only a 
few resolvers to be coexistent, but this did not lead to any perceptible 
improvement. The DBus still seems to be congested after a short while. 
Only a restart of the service solves the problem. Without deep insight I 
would assume, that the DBus-objects created are not freed-up cleanly or 
lets say immediately. They seem to remain and so the bus gets filled-up.
To give you a more detailed overview I explain to you what exactly happens:
- We browse the .local domain for HTTP and HTTPS services with two 
coexistent browsers
- For every resolver we create, we create another one to receive 
additional address records
- The first resolver is freed immediately after resolving, the second 
after the last address has been resolved
- This works fins for about 100 Hosts, then it starts to stumble. The 
other resolvers achieve timeouts. If a timeout occurs we trie to resolve 
again.
- After about 1min we receive another circa 50 hosts before the next 
stumbling
- Then about every 1-2 minutes we receive a variable amount of hosts
We also tried a queued algorithm with only a certain amount of 
coexistent resolvers without a noticeable improvement.

As you have a now a litlle more detailed view on what we do, could you 
please give us some hints on how to improve the use of Avahi?
Maybe its not an DBus but an Avahi issue?
What are the exact limits of Avahi concerning cache, timeouts queries, 
etc...?
How does the free-mechanism work exactly, especially on the DBus-side?

Thanx again!

Daniel