Using flat-manager for private app repo

[Dan Nicholson]

On Tue, Apr 30, 2019 at 11:11 AM Alexander Larsson <alexl at redhat.com> wrote:
>
> On Tue, Apr 30, 2019 at 5:30 PM Dan Nicholson <nicholson at endlessm.com> wrote:
> >
> > This is the same as what I was envisioning except maybe the last part.
> > The way I was thinking about it, the authorization server would have a
> > long lived token that had read permissions for everything in a
> > particular set of repos. After authenticating the user and determining
> > what apps they had access to, it would call the flat-manager
> > /token_subset API with a subset of repos and apps the user could
> > access. Then the authorization server doesn't need a secret key at
> > all. flat-manager hands back a more limited token signed by it's
> > secret and the authorization server gives this to the user. Then the
> > user uses this token to authenticate to flat-manager and it can
> > validate the token. I believe this is exactly how the
> > flat-manager-client works right now.
>
> Do you mean one token with everything the user has access to? Seems
> like that could end up very large. And anyway, to avoid risk of
> leaking such a token we need to time limit it anyway, so generating a
> token per download seems to make more sense.

Yeah, that would probably be a little nicer at the expense of
requiring tokens more often.

> But yeah, I see what you mean. Its basically what happens now except
> the current limit is on a per-repo basis, and it would need to be
> changed to per-commit or per-ref or something more finegrained. It
> means each single download would need to hit the master server though,
> which seems like it could be a performance problem. I think we should
> try our very best to make it at least possible to move the
> authentication to the edge.

This is definitely the weakest part of this scheme. We're highly
dependent on our CDN at Endless for making access fast. Requiring
connections through to the origin would slow things down.

> > > Then the repo server (like flat-manager) can know that certain (or
> > > all) refs require authentication and what signatures are needed to
> > > download those files.
> >
> > I'm not sure what you mean here. How does the repo server know what
> > refs require authentication and what the allowed signature are?
>
> All the CDN sees is a download of e.g. 04/232121244.commit. Say the
> token says the user is allowed to download "org.cool.Game", and that
> is part of the token. How does the CDN match the url being downloaded
> to the required token? Well, we could have the upstream flatp-manager
> set some http header in the reply (for all apps that require a token),
> specifying which tokens are needed. Then the CDN can do the upstream
> request once and cache this header, letting future requests be cache
> only.
>
> So, something like:
>
> user asks central server for a token for org.cool.Game
> user passes token to CDN request
> CDN does upstream request, gets back result + header that says "need
> org.cool.Game token"
> CDN verifies passed token based on header + current time
> CDN returns results
>
> On a second call the CDN can use a cached value for the token and
> avoid an upstream request.

That would be really nice, but I fear it would be highly CDN specific.
It would basically require lifting the application logic into the CDN.
That looks somewhat possible with Fastly which uses Varnish and allows
you to provide customized hooks. And now that I look again even our
CDN provides a way to use URL tokens.

So, probably there is a way to make the CDN edge server handle
authorization, although it would be specific to the CDN provider. That
said, I think it should be possible to deploy a private app store
without also requiring on a CDN. In other words, I think it should be
possible to handle the authorization at both the origin and edge
server.

One thing that could be useful is an API endpoint that takes a ref and
returns a list of commit and delta superblock URL paths. Then the
authorization server could generate the proper signed URLs in the
format the CDN needed after determining that the user was allowed to
access a specific ref. I don't know, I haven't quite been able to
figure out how this would go.

--
Dan