[Freedesktop-sdk] License blacklisting [Was: license-checking script for BuildStream projects]

[Chandan Singh]

Hi Tristan,

First off, apologies for the delayed response. But, better late than never :)

A couple of comments inline.

On Fri, Aug 28, 2020 at 10:12 AM Tristan Van Berkom
<tristan.vanberkom at codethink.co.uk> wrote:
>
> Hi Chandan,
>
> On Thu, 2020-08-27 at 22:54 +0100, Chandan Singh wrote:
> > Hi Tristan,
> >
> > I don't have enough background on the FreeDesktop side of things, but
> > a couple of comments inline regarding the BuildStream part of the
> > proposal.
> >
> > <snip>
> >
> > > For a vast portion of open source / free software available in the
> > > wild, this conscious interpretation and decision needs to be made by a
> > > human being.
> > >
> > > I would see this implemented in BuildStream in the following way:
> > >
> > >   * Declare a new "licenses" public data format in the bst public data
> > >     domain[3]
> > >
> > >     This is a place where BuildStream project maintainers can record
> > >     the decided license for the module being built, similar to yocto's
> > >     LICENSE variable[1].
> >
> > Personally, I think BuildStream itself shouldn't care about licenses.
> > It should probably not even know that licenses are a thing. As I see
> > it, BuildStream is a build tool and managing licenses shouldn't be its
> > responsibility. Auditing of sources seems like a separate concern to
> > me.
> >
> > At least the way I use BuildStream, by the time I decide to use a
> > source in a BuildStream element, I have already made a decision to use
> > it and hopefully have done my due diligence and auditing. But, maybe
> > others have different usage patterns. In any case, I feel like
> > BuildStream core shouldn't have an opinion about licenses or how to
> > manage them. But, having a plugin do this job is definitely fair game.
>
> It is especially relevant because BuildStream projects are a shared
> good, this is what junctions essentially solve.
>
> When building something in the BuildStream ecosystem, you are probably
> interacting with upstream junctions which they themselves provide a
> hand full of modules under various licenses.
>
> These upstreams are probably a mix of FOSS upstreams and ISVs.
>
> In the end you may very well trust that these upstreams have done their
> due diligence in attributing the correct licenses to the modules they
> provide you (and these upstreams may even be liable for such depending
> on the contract you have with your upstreams).
>
> At the end of the day, you will probably junction a hand full of
> upstreams, add your own highlevel application on top of it all, and
> ship it.
>
> At this time, it becomes highly valueable that you can simply run an
> assertion element which check for license compatibility of all of the
> modules which ended up in your dependency chain.

Thanks for the summary. I do understand what we are trying to do here,
and I do think BuildStream itself should allow for such an assertion
to happen, This can be done via allowing sufficient API for a plugin
to exist that makes such assertions. I just don't think BuildStream
core itself should have to do anything relating to licenses. Would you
agree with this?

> > >     For compatibility across tooling, and consideration of possible
> > >     further automation (see further below), we should probably assert
> > >     that these license annotations be valid SPDX license
> > >     identifiers[4].
> > >
> > >   * We would add a new Element plugin in BuildStream, and call it
> > >     something like `assertlicense`
> > >
> > >     In this element's `config`, it would allow the user to declare
> > >     a blacklist.
> > >
> > >     This element could output a manifest of licenses in the artifact,
> > >     or produce no output at all, the important part is that this
> > >     element can be added to the pipeline, depend on some elements,
> > >     and halt the build with an error in the case that invalid
> > >     licenses are detected.
> >
> > I don't have anything against such a plugin. But, I'd be much happier
> > if the public data specification was defined by such a plugin rather
> > than BuildStream itself. Maybe a family of plugins decide to follow a
> > shared format, and maybe some other family could decide to do things
> > differently if their needs aren't met. Either way, BuildStream core
> > won't need to change or care.
>
> This raises an interesting point about naming public data which belongs
> to various plugins.
>
> We could theoretically split up the `bst` domain into separate
> categories, keeping only keys which are understood by plugins that are
> wired into the code BuildStream plugins.
>
> I think it makes better sense to have the `bst` domain cover all
> plugins which are maintained under the BuildStream umbrella.
>
> In the end, this is only a question about how to use the `bst`
> namespace.

Indeed, this is mostly a question around the ownership of the `bst`
namespace. My suggestion is that it should be reserved by BuildStream
core itself. That is, if the presence of that plugin data concerns
BuildStream itself in any way.

The current builtin public data fields are: integration-commands,
split-rules and overlap-whitelist. All of these change BuildStream's
behavior in one way or the other. For example, integration-commands is
so much part of the core interface, that it gets its own flag in `bst
artifact checkout --integrate/--no-integrate`.

As such, I think we should keep the `bst` namespace for things
concerning the BuildStream core.

If we agree on this separation of concerns, I'm not sure what would be
the best alternative, i.e. how should we group things together, and
how fine-grained should that be. If we can't come up with a better
scheme, something like `core-plugins` or `bst-plugins` won't be any
worse than `bst` in my opinion.

Cheers,
Chandan