[packagekit] 1-click; Third party vendors; etc.

[James Westby]

On Mon, 2008-06-02 at 13:12 +0100, Benji Weber wrote:
> Greetings All,
> 
> I have been watching with interest the discussion of one-click and
> third party repositories resulting from fosscamp on this list. I have
> been too busy with exams to risk commenting and embroiling myself in a
> discussion until now. I would like to comment on some of the points
> raised throughout this thread. I'll start a new thread as the other
> was already getting rather off topic and there's no single message to
> which I want to reply.

Hi Benji,

Firstly, thanks for your detailed and considered message on the subject.


> The first thing that was asked was what the use cases for OCI are. The
> traditional open package manager -> search -> install works well for
> finding and installing software in the distribution's official
> repository. However, there are two limitations:
> 
> a) Only software published by the system vendor is available (unless
> other repositories are added)
> b) Software is only locatable based on metadata supplied by the
> packagers - name, summary, provides...

Did you read the discussion on extending apturl to work through
packagekit to provide some solution to the second limitation here?
Are there ways in which the OCI solution is better than that
solution on this aspect?

> OCI allows a single install link to contain instructions for multiple
> distributions. The handler should choose the instructions appropriate
> for the distribution it is running under.

You say "allows", so this isn't necessary if the same instructions
work on many distros?

> Through package & metadata
> signing users are already able to establish the identity of the
> packager, but there is not usally a good way for users to make a
> meaningful decision as to whether to trust this identity. I think this
> is something that can be done better, the packager's identity can be
> related to how much the community trusts him/her. There is some
> interesting work being done on such a system for the openSUSE build
> service[14]. If the installer always looked up and displayed to the
> user both the packager's trust rating and the software's own rating,
> then the user could make a more informed choice as to whether to
> install the software or not.
> 

This is an interesting idea, but I think it would take an awful lot
of work to have something here that would actually make users be
able to make sensible decisions about enabling a specific repository.

This isn't to discourage the idea, I just think that you have pointed
to a solution that is a long way off (even if the build service
brings it closer for openSUSE we are talking about doing this 
cross-distro), and so the concerns about relying on users to judge
the risk of enabling a specific repository still stands.

> b) Difficulty upgrading distribution between versions caused by third
> party packages installed on the system not having an upgrade path.
> 
> This is indeed a problem, and has often caused upgrade issues for suse
> users. The upgrade can usually be performed without problems by
> disabling all old repositories, adding the new repositories, and
> upgrading. Unfortunately this method often means that removal of old
> third party software that is not available in the distribution is be
> required, as its dependencies will not be satisfied on the new system.
> Although this can be suggested and performed automatically the user
> does not really want to lose all his/her third party software.
> 

This is actually the approach used currently by Ubuntu. It's obviously
not ideal, and I know some people will be unhappy about it, but it
at least increases the number of upgrades that don't break (if you
don't take some users' definition of break, that includes removing
an application they use that isn't available in the repositories).

It would be possible to improve on this with projects working more 
closely with distributions to ensure that the package fits in with the
distro, and that there is a upgrade path when it comes to release time.

Is there a concern that making OCI more widespread would lead to
projects working less closely with distributions as they don't have
to worry so much about the state of packages in the distro? I imagine
that it generally wouldn't happen, and the good upstreams certainly
wouldn't do this. However, it may make some upstreams another step
removed from the distro.

> 
> So, in my opinion, most of the problems that can occur when dealing
> with third party repositories are not problems with the concept of
> third party repositories,

I think they are. I think they stem from the nature of third party
repositories. I could create a repository that fit exactly in with
the policies of a specific distro so that I would not cause any
trouble for users of that distro that chose to use my repository.
However, there's no way I could do it for all at once.

Also, some repositories won't be that careful about ensuring that
there are no problems caused for their users.

>  but rather failings in how the system deals
> with those repositories.

I agree that we could do better at dealing with them, whether we agree
they are naturally problematic or not.

> Telling vendors they have publish their software under a specific
> licence, or that they must work under the distributor's own policies,
> procedures, and timescales to reach users seems like a good way of
> discouraging vendors from targeting your distribution.
> 

I agree on the license point. However, I don't think that requiring
that a package conforms to Debian/Ubuntu policy before the distro
makes it easy for a user to install that package is a bad thing.

> b) Release Schedules
> 
> Many distributions operate on a stable release every $TIMEPERIOD
> system, that involves freezing the versions of applications bundled
> with each version of the distribution for the lifetime of that
> distribution. While this is great for ensuring that that software all
> works well together, where does it leave the person who needs to
> follow the latest version of $APPLICATION for his/her work? e.g. A
> java developer might require the latest version of eclipse/netbeans
> and other java productivity software. Should he/she follow the
> unstable branch of the distribution just so that he can have the
> latest software? Or should he install the software manually on the
> stable version and lose all the benefit of the package management
> system? In openSUSE it is common to use third party repositories to
> allow special interest groups to follow current versions of specific
> software on the stable version of the distribution. There are
> repositories for latest java tools, latest KDE, etc. The users can
> keep their stable supported system and install the newer versions of
> the software that they require.
> 

Ubuntu has backport repositories to try and fulfil this need. It is not
broken down in to type of software (java/KDE/etc.), but it does
have distro developers trying to ensure that no harm will come to users
who use those repositories.

Would your proposal have each upstream maintain a backport of the 
development for each release of each supported distro? That seems like
a lot of work for them. Could we encourage them to pass of some of the
work to the distros through efforts like backports.org?

> The other question was that even if we accept the concept of third
> party repositories, does OCI make it too easy for users to add them,
> without understanding the consequences? Is it, like Sebastian said
> "Opening the gates of hell"[17]? It is true that OCI makes adding
> third party repositories easy, that is the whole point. However,
> consider these points:
> 
> a) You control the process.

Good point.

> b) It is not the easiest attack vector.

I'm not so concerned about the malicious attack vector parts of this,
perhaps others are.

I would add another point here,

d) many users will do it anyway.

If there is something we can do that helps them do it right, then I think
we should consider it.


Over the last could of weeks on this list we have talked about some 
things that are relevant here, if not specifically about OCI. Can I
outline a proposal that aims to do a similar thing to OCI, but would
perhaps be more palatable to some people? This proposal is not meant
to be rigorous, and it is a discussion point, so please point out
where it is bad, and those who support OCI point out where that solution
is superior.

This proposal basically revolves around the extended apturl. There would
be handlers installed wherever necessary to support the links and
do what it is saying. apturl was created to solve the simple "install
this package from the repositories" case, so we have that covered:
   
    install:packagekit

(there still stands the problem of mapping this to package names, 
perhaps following OCI here and using a file to describe what to do
is better)

Then, for the third party repo case we can have a way to enable a
repository before installing the package:

    install:foo?repo=http://wherever/

Richard has proposed shipping in packagekit a list of keys, so for
big repositories we can omit the key checking dialog for the user.

To try and stop users from having a repository enabled that is 
incompatible with their system (think Debian/Ubuntu), i proposed
extending this to have a list of distributions that the repo is
known to be compatible with.

Then, to try and help the user understand the dangers of enabling
that repository we could list a risk factor for each distribution.
So, if I was working with an upstream, and knew that they put
a lot of info in to making their repository work well with the
whole distro I could rate them a "1". For a commercial vendor that
was usually ok, but didn't talk to us that much I could rate them a
"2", and for a repo that was pretty good at breaking upgrades I could
rate them a "3". This would then change the dialog that the user
is presented with.

This takes the effort of judging the repository away from the users,
and to the developers of the distro, who are best placed to judge it.

For unknown repositories you would get a bunch of scary dialogs when 
adding the repository.

Now, one obvious problem with this is it creates a huge burden on
the distros to rate the repositories, and to get this right. I don't
have any immediate thoughts on this, but if it reduces the number of
users who enable known-bad repositories then they may like it.

Hopefully it also creates an incentive for repository providers to
do good work, and to engage with the distros.

Obviously, OCI could be mutated to do something similar to this.

So, please tear the idea apart and lets see what we learn.

Thanks,

James