Using flat-manager for private app repo

[Alexander Larsson]

On Fri, May 3, 2019 at 2:15 PM Robert McQueen <rob at endlessm.com> wrote:
>
> On Thu, 2019-05-02 at 11:13 -0500, Dan Nicholson wrote:
> > On Tue, Apr 30, 2019 at 11:11 AM Alexander Larsson <alexl at redhat.com>
> > wrote:
> > > But yeah, I see what you mean. Its basically what happens now
> > > except
> > > the current limit is on a per-repo basis, and it would need to be
> > > changed to per-commit or per-ref or something more finegrained. It
> > > means each single download would need to hit the master server
> > > though,
> > > which seems like it could be a performance problem. I think we
> > > should
> > > try our very best to make it at least possible to move the
> > > authentication to the edge.
> >
> > This is definitely the weakest part of this scheme. We're highly
> > dependent on our CDN at Endless for making access fast. Requiring
> > connections through to the origin would slow things down.
>
> Sure it would slow them down... for the metadata (commits, delta
> superblocks), but we can still access these via the CDN and mark the
> caching as dependent on the presented token. All of the big stuff such
> as objects will be cached as normal through the CDN.

Its not only about network slowness, it also means more load on the
main repo, as well as less robustness to long-range network issues.
But I think we all agree that we should be able to handle both.

> If you /do/ have a super fancy CDN with arbitrary edge logic (some like
> Cloudflare have like JS lambdas - you could teach it to maintain a ref-
> >commit mapping etc) you could eventually disperse this determination
> too. But I didn't consider it to be a substantial bottleneck enough to
> design flat-manager out from the picture from the start - more
> something to design/optimise later.

We never have to maintain such a mapping for the edge. All we need is
for flat-manager to generate this mapping for the requested url and
return it via a header and for the edge to cache this with the data
for that url. There is never any need for super complex edge work. All
you need is to look at a header and use that to driver the required
token.

> > One thing that could be useful is an API endpoint that takes a ref
> > and
> > returns a list of commit and delta superblock URL paths. Then the
> > authorization server could generate the proper signed URLs in the
> > format the CDN needed after determining that the user was allowed to
> > access a specific ref. I don't know, I haven't quite been able to
> > figure out how this would go.
>
> Potentially we could do the two discoveries in one step - user
> entitlement (you can get these refs, yes or no) and ref -> commit
> mapping (those refs are currently these commits, or these URLs that you
> should be able to get).

I don't understand this. ostree knows how to go from ref to commit id,
and the client will use that not some url it got from somewhere else.
What use do we have from knowing these urls ahead of time? We would
just reimplement ostree pull.

> So as the thing that can resolve refs to commits, flat-manager could
> hand out those temporary tokens via it's API - or a separate service
> that is ostree aware. Either way it would need an API to check the user
> entitlement somehow. (A cachable "user x wants ref y, yes or no?"
> backend. Or maybe just "get me the current permitted refs for user x".)

I do think we need some Entitlement API, but I don't think
flat-manager should implement that. That would be some other server
with an api like:

Basically, here is how I imagine the entire flow:

Client:
1) flatpak install $remote $ref
2) Check if $remote has an entilement server configured (and it has)
3) call entitlement server GetTokenForRequest($ref,
data_about_me_like_user_password_etc))
4) call ostree_repo_pull_with_options, passing the token in http-headers

CDN:
1) Gets a request for repo/../*.commit (or superblock)
2) Do the upstream request, or look in local cache
3) Get $ref from the resulting header x-flatpak-requested-ref
4) Assert that $ref is in the token passed with the request, or 403

flat-manager:
1) Gets a request for repo/../*.commit (or superblock)
2) Reads file, looks up ref binding for the commit
3) Adds ref to x-flatpak-requested-ref response header
4) Optionally (non-CDN case), assert that $ref is in the token passed
with the request, or 403

In this system the entitlement server is not actually flat-manager,
but can be whatever customer tracking system your organization uses.
For example, it  would do some kind of steam store lookup if valve
wanted to use this.