AppStream Ideas and Thoughts

[James Rhodes]

On Tue, Feb 15, 2011 at 8:38 PM, Enrico Zini <enrico at enricozini.org> wrote:
> On Tue, Feb 15, 2011 at 07:54:11PM +1100, James Rhodes wrote:
>
>> I appear to have not written a wiki article on this, but the naming
>> standards apply to AppFS packages only.  For compatibility with
>> existing systems, a central package mapping service called DepResolve
>> will exist that maps the dependencies listed in a package with the
>> actual package that needs to be installed based on the current
>> distribution.
>
> Leaving aside the rest of the discussion, I just need to point out that
> we now have such package mapping service:
>
>  http://www.enricozini.org/2011/debian/distromatch/
>  with an example deployment here: http://dde.debian.net/dde/q/distromatch/match/
>
> The mapping is not perfect (yet) but it works. Improvements will come
> with usage.
>
> I'm still working on having regular data exports for all involved
> distributions, in order to enable anyone to set up a working distromatch
> on their own systems. This should be done by the end of the week.

That sounds like an incredibly useful database you have there.
There's no doubt I'll be using that to populate the initial DepResolve
service (it's an online service using HTTP requests with cached
results in the packages themselves).

On Tue, Feb 15, 2011 at 9:11 PM, FlorianFesti <ffesti at redhat.com> wrote:
> On 02/15/2011 09:54 AM, James Rhodes wrote:
>>
>> I know why package managers work the way they do; having everything in
>> a central repository at first seems to be a great way to ensure that
>> every software that the distribution wants to offer has the
>> dependencies available for it, which for well-known open source
>> software is fine.  There's a high change that users will be able to
>> find the software they want in the repository.
>
> As you kinda admit getting the dependencies right is not trivial (There are
> in fact some nicely NP-complete problems lurking there). A package format
> alone does neither solve this nor does it integrate with the distribution in
> more than adding the duplicates into their database. May be some of this
> difficulties can me solved by leveraging the work already done in the
> distributions but it still is not trivial.
>
> There are a couple of other reasons for why distributions look like the way
> they do, that need to be taking into account (list does not claim
> completeness):
>
> Have someone taking care of every component that got packaged. How can a
> user expect that the vendor is capable of taking care of all issues that may
> emerge in the libraries they have bundled. The distributions are assigning
> some one to every library and they have a separate security response team to
> make sure the maintainers do their job.

You see, I would think that the distribution owners would actually
want to hand that work off to the person building or developing the
software in the first place, since the developer knows when they
release updates, they know what dependencies their software has and
hence they can maintain their packages in a much more timely and
secure manner than having a third-party distributor build and ship
them.

> Is such a packaged world the amount of data need for updating a
> (compromised) library is enormous. This basically shuts down updates for
> everything but the most urgent exploits and even they generate an ugly
> amount fallout - especially as these updates come in one big chunk (think
> about an exploit in zlib).

I understand your point here to be that the original software
developer may not have the bandwidth available to ship updates to all
the users of their library / application, right?  In that case, the
distributor could mirror the the AppTools package on their AppServer,
and the updating system would use that (it obviously prefers using the
distributor's service over an individual's server; and since it's a
mirror, there's no need for each package to be managed by the
distributor, it can be an automatic mirroring).

> The distributions are a trusted third party that makes sure that the
> software they get from upstream is not malicious. Sure vendors with a strong
> brand don't need a third party (e.g. the adobe repositories). But the target
> audience of such package formats typically don't have such a brand.

In order to ensure that updates you receive come from source that
originally produced the initial package you installed, you use
signing.  It's really that simple.

> The know how of good packaging and package maintenance does not scale down
> very well. There is a serious amount of general knowledge and continuous
> work needed. This is significantly easier within a big projects dedicated to
> this task than on your own. No matter how good your tools are they are still
> putting an pretty big burden onto the third party vendors (have a look at
> the rpms they build).

I really don't understand what you're trying to say here.  Do you mean
that it's a lot of work for the software developer to package their
software in a binary format?  Or do you mean it's a lot of work for a
maintainer to package the software in a binary format?  In the former
case, you use appcompile to ensure a standard build process takes
place when packaging the software (and it deals with handling the path
differences between distributions too).  For the latter, you'd clone
the package from the developer's server, or alternatively use
appcompile to produce the package just like they would (one of the
planned features of AppServer is the ability to automatically retrieve
source code from source control systems and build it with appcompile).

> I think the overall approach is flawed. If I were interested in this topic
> I'd use the SUSE build system tools or something similar and offer a service
> to create packages for all distros. May be charge a fee for closed source
> applications or offer a build system as an appliance or cloud image. Then
> setup an repository or a repository list that makes it easy for users to
> subscribe.

I agree that the SUSE build system has come along way to making the
process easier, but in my opinion, it's still a workaround to a broken
system (and there's no such build option for third-party, closed
source software at this point).

Regards, James.