infrastructure heads-up

[Florian Effenberger]

Hello,

I wanted to give all of you a quick infrastructure heads-up. You might 
have seen that from time to time, there are certain hiccups, which is 
due to the fact that in the last weeks, the popularity of our sites and 
therefore the use of the infrastructure has *heavily* increased. After 
some thorough checking, we can (at least relatively sure) rule out 
configuration issues as well as attacks, so it's actually good news - 
grown interest and demand. The result, sometimes 503 errors and 
timeouts, is bad of course.

We have therefore decided to change the way we plan our systems 
migration. During the last admin meeting in September, the plan was to 
make a clean start using centralized configuration management. This is a 
logical and senseful step, but unfortunately, this requires huge amounts 
of time - putting the documentation into a configuration language is 
tedious.

In order to eliminate the bottlenecks, we will now first migrate to the 
new, much stronger hardware and newer software, configuring services 
manually, to finally get back to a stable state. In other words: 
Replicating the existing setup with relatively few changes, but using 
stronger hardware. My personal plan is, even if it involves some lenghty 
nights and working weekends, to have the web services migrated this 
month, if all goes well.

Only thereafter, when the bottlenecks are eliminated, we will start with 
centralized configuration and further planning the growing of the 
infrastructure.

We are sorry for the inconveniences the hiccups cause. The admin team is 
doing what they can, but as for many people in the project, time of 
course is a limiting factor, and it's not so easy to overcome the laws 
of physics. ;-) Plus, scaling certain tasks to too many people isn't a 
solution either.

We'll keep you posted. I'd like to take a chance to thank all those 
silent workers in the background, especially Alex, Alin, Andreas, 
Christian, Erich, Friedrich, Robert and Uwe, who already spent many 
hours and endless nights to ensure stable services. Thanks a lot folks, 
you all rock!

Florian

-- 
Florian Effenberger, Chairman of the Board (Vorstandsvorsitzender)
Tel: +49 8341 99660880 | Mobile: +49 151 14424108
The Document Foundation, Zimmerstr. 69, 10117 Berlin, Germany
Gemeinnützige rechtsfähige Stiftung des bürgerlichen Rechts
Legal details: http://www.documentfoundation.org/imprint