AppStream Ideas and Thoughts

[James Rhodes]

On Wed, Feb 16, 2011 at 10:48 PM, Jos Poortvliet <jos at opensuse.org> wrote:
> On 2011-02-16 James wrote:
>> On Wed, Feb 16, 2011 at 8:11 PM, Enrico Zini <enrico at enricozini.org> wrote:
>> > On Wed, Feb 16, 2011 at 10:55:36AM +1100, James Rhodes wrote:
>> >> What is the value of distributions?  The maintainers get to pick and
>> >> choose how they want the end user's system to run (by default).  I
>> >> think that's really the only value in distributions.  Occasionally
>> >> distributions also contain custom software (such as YaST).
>> >
>> > By all means, if you believe there is a better approach to software
>> > distribution, please feel free to work on it.
>> >
>> > Allow me to point out, however, that it would be beneficial for you, as
>> > well as for the level of the discussion in the list, if you tried to
>> > understand, or at least to respect, the vast amount of often thankless
>> > work others are doing.
>> >
>> > Like mail delivery or gas pipe maintenance, it tends to be the kind of
>> > overlooked hard work that nobody ever notices or cares about, until the
>> > moment when it stops.
>>
>> Not trying to be disrespectful in anyway; that's just honestly what I
>> see when I install and use a distribution, compared to say building my
>> own using Linux From Scratch (in addition, the latter does not have an
>> implicit package management supplied).
>>
>> By all means, please enlighten me on the other work that goes into
>> distributions.  If anything it'll give me a clearer view of the
>> situation.
>
> I don't think it is up to Enrico to prove building a linux distribution is
> hard work. I'm sure you can find that out yourself.
>
> As to your responses to what I wrote. I am not a distro developer, but I have
> compiled the odd piece of software in my time, and heck - the only value of a
> distro is in picking and choosing how they want end user systems to run by
> default (& some custom software)? Yes, that is right. And you seem to
> underestimate that task by a HUGE margin. It's not about choosing default KDE
> settings, you know. It is about fighting incompatibilities between
> combinations of packages and versions of packages (which claim to be
> compattible), testing stuff. I've heard horror stories of circular
> dependencies, incompattible libraries, stuff which can only be fixed by
> someone who controls the whole chain. Add a larger (say +10) number of
> different repositories to ANY distro, be it deb or RPM based and see it become
> more and more unstable over time until it breaks (about a year, usually).
> Multiply by 100 and you get the situation you advocate.

The only reason there are incompatibilities is because package
management makes it so.  Current package management systems force you
to pick and choose between one version of a library or another.  Why
is this necessary?  It isn't.  But it's the way that the package
systems have been designed that force you to make that decision and
hence lead to the issues you're describing such as circular
dependencies.

There's really no need for it to be the case.  With the system I'm
suggesting it's actually easier to resolve circular dependencies
because you can force the installation of a package regardless of
dependencies, because unlike other package systems, you don't have to
get rid of the old version or whatever; you can have two versions of
the same library happily existing side-by-side on the same system, and
it all works.

Essentially there's two ways you can get an application to use the
right version of a library.  I believe there was some work done which
listens to the library that an application attempts to open by
monitoring it's requests and then substituting the correct library in
based on what the application requires.  The alternative to this is
that, because each application is contained entirely within it's own
directory, to use uchrooting and an overlay filesystem to show the
correct libraries in /lib, etc.  Obviously the latter is rather messy
and I don't particularly like it either, but if the former doesn't
work out I can't see any other way of doing it (you have to remember
that under AppTools, files in /lib and /usr/lib are symlinks to the
actual libraries, which are stored into their own directories per
package; see GoboLinux for a similar system to how this is done).

> And the above is just *stability*. I'm not talking about security and sharing
> libraries. You say we can keep sharing libraries and security is not an issue.
> I beg you pardon? So you claim that if I have 50 apps from 50 different
> vendors and repositories and in 1 library which they all share a security
> problem is found I can upgrade that library right away because you guarantee
> that all those vendors will have re-compiled their package *right away*? What
> world do you live in? What if there are different branches of that library and
> half the vendors pick one, the other half picks another? Or if there are
> different competing libs and I want one but most vendors pick another? What if
> a vendor stops updating an older version but I don't want to upgrade?

This is answered in my response above; you just keep both versions of
the library while developers make the migration.

> Sorry if I'm harsh. I just have a strong feeling that you know even less than
> I do about the issues distro developers deal with - which means you have a lot
> of figuring-that-out todo before you can solve the problems you are trying to
> solve. I would suggest becoming a Fedora, Arch, Gentoo, openSUSE or Debian
> packager for a few years first.

I've compiled my fair share of software in my days, and I've had
plenty of experiences where the current package management systems
have the same issues you explicitly mentioned before.  Circular
dependencies, incorrect versions; it all happens with the *existing*
systems.  Even in the event that both of those issues were applicable
to AppTools (and they aren't; see above), it's still a better system
than the one we have now.

I should point out that AppTools does not take away your repositories
or your centralized system for officially supported packages, since a
few people seem to be assuming that's the case.  This functionality is
implemented in AppServer, but it works with AppFS packages.
Distribution maintainers can still say "these are the packages we
support and they work together" and offer their users a place to get
officially supported software.  AppTools will use the AppServer
repositories to resolve dependencies if it can, above the fallback
URLs that developers specify in their packages.

I hope that clears up a few things.

Regards, James.