Rebasing yocto to new freedesktop platform?

[Alexander Larsson]

On Wed, 2017-03-08 at 12:57 +0000, Robert McQueen wrote:
> On 06/03/17 08:28, Alexander Larsson wrote:
> > The current freedesktop platform is based on yocto 2.0, which is
> > now
> > "Community supported"[1], and i've heard some complaints about the
> > age
> > of some components. Maybe its time to do a freedesktop 1.6 based on
> > the
> > latest stable yocto (2.2).
> 
> I think if we're claiming to maintain the runtimes centrally for
> reasons 
> including timeliness of incorporating security updates, we should
> meet 
> that expectation and do so eagerly... :)

Yeah. I wish maintainance of the runtime was more than just me
though...

Anyway, a freedesktop 1.6 with new yocto and some cleanups is comming
soon. In practice though, the yocto base is pretty small, so the amount
of changes are not terribly exciting. The main attractions is gcc 6.2
and python 3.5.

> > I'm going to do some experimenting with building it today. If it
> > works
> > out we should probably switch to it in the gnome nightly builds and
> > target it for the gnome 3.24 release.
> 
> If you are in an experimenting mood, I wonder if there are any yocto 
> reproducible build overlays/whatever that could usefully be turned
> on? 
> Even if not perfect / full coverage, anything that increases our
> chance 
> of not spitting out new binaries when nothing else has changed,
> reduces 
> download churn for end users.

I haven't really done any deep research on flatpak-builder
reproducibility, but the few times i tried its "pretty good", but it
clearly would be interesting to do better. For the yocto base I think
we're pretty safe in that most of the time the new build only changes a
few packages, and we're reusing old package builds. This means the base
will generally not change much.

> Generally speaking, I'm a little nervous in general about the size
> of 
> the enforced re-download when we rebuild these base runtimes.
> Hopefully 
> something like BuildStream can help us reach a "fully reproducible" 
> flying car future, but in the meantime I'd be interested in seeing
> what 
> Yocto has to offer.

Any help here would be appreciated!

> The effect is compounded with "derived" runtimes like GNOME, etc -
> if 
> the user doesn't update them at the same time, you would also
> duplicate 
> many likely source-identical but different-binary objects from the
> old 
> and new fd.o runtimes on the system, both on disk and in memory when 
> apps are running.

Yeah. Another issue i have is the on-network deduplication not working
when using deltas.

For instance, consider an freedesktop.org update that changes 1 ostree
object, coupled with a gnome update that inherits the fd.o update and
changes 1 object itself.

When we generate the delta for the gnome runtime both the changed
objects will be put in its delta, so if you update first the fd.o one
and then the gnome one we will download the new fd.o twice.

ostree does check before downloading a delta part if all the objects in
it are available and then skips it, but as long as one object is not
there it has to download it.

Ostree could do better here if we could tell it about the layering
somehow. For instance it could put all the objects from the base in a
separate delta part. Of course, then we have to tell ostree about this
layering somehow. Walters, do you have any ideas here?

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl at redhat.com            alexander.larsson at gmail.com 
He's a maverick Republican gangster gone bad. She's a mistrustful 
cat-loving museum curator with a song in her heart and a spring in her 
step. They fight crime!