[systemd-devel] [PATCH 0/5] systemd-importd - support for pulling from V2 Dkr registries

[Pavel Odvody]

Hi,

the attached series of patches add support for pulling from V2 docker
registries, so let me break down first what happened to the format since
V1
 - Image is now defined by a JSON manifest
  - contains fields like name, tag, schemaVersion ...
  - and fsLayers - which is an array of sha256 references to a
    *content-addressable FS layers*
  - the manifest is now also signed using JWS/JWT (ECDSA p-256 mostly)
 - Authentication/Authorization now bearer token only
 - To access the V2 registry we need to send a special User-Agent
   docker/1.6.0
 - The whole manifest can be hashed using sha256 to obtain a 
   "digest", which provides an immutable global identifier of the image,
   and can be used instead of a tag when pulling the image (the REST
   API endpoints are the same).

So far so good, now what's in the patches, besides the V2 workflow
 - lightweight JSON parser, written around json_tokenize
 - I've renamed 'tag' to 'reference' to accommodate for the digest
   semantics
 - all layers are saved in a directory .dkr-$imageid - image id is
   resolved from the v1 compatibility section of the manifest
  - since the layers are now CAS, we can't assume that the order, or
    mere presence of certain layers will be preserved throughout
    multitude of images/manifests, and therefore due to the
    incremental nature of BTRFS snapshots we need to throw any
    intermediary snapshots away.
 - small bugfix for the JSON tokenizer (it'd choke after reading 
   any digit)

This is the bare minimum to pull&run V2 images, since the signature is
now embedded in the manifest, it could now support --verify=signature. 
However, I've got one open question - how do we support V1/V2
concurrently (this patch makes V2 the default and only)? Docker first
pings the V2 endpoint and then falls back to V1, but I think that this is 
sub optimal, since --verify=signature makes sense only with V2, so I think 
something like
  
   --dkr-pull-strategy=v1|v2

as an argument would be the best?

Thanks,

Pavel

Pavel Odvody (5):
  shared/import-util: tag renamed to reference to     support v2 pull by
    digest
  shared/json: JSON parser + number tokenizer bugfix
  test/test-json: Tests for the JSON parser and the     tokenizer bugfix
  import/pull: Tag replaced with reference
  import/pull-dkr: V2 Image specification + manifest     support

 src/import/pull-dkr.c    | 531 +++++++++++++++++++++++++++++++++++++++++------
 src/import/pull-dkr.h    |  48 ++++-
 src/import/pull.c        |  28 ++-
 src/shared/import-util.c |  19 ++
 src/shared/import-util.h |   1 +
 src/shared/json.c        | 437 +++++++++++++++++++++++++++++++++++++-
 src/shared/json.h        |  36 ++++
 src/test/test-json.c     |  16 ++
 8 files changed, 1034 insertions(+), 82 deletions(-)

-- 
2.1.0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20150507/8cd5c41e/attachment.sig>