ANNOUNCE: p11-kit 0.19.1

[Stef Walter]

On 28.05.2013 11:34, Nikos Mavrogiannopoulos wrote:
> On Mon, May 27, 2013 at 10:44 AM, Stef Walter <stefw at redhat.com> wrote:
>> This is an unstable development release. It introduces a some new
>> refactored API, and unfortunately deprecates some commonly used functions.
>> As discussed earlier on the mailing list, this adds a dependency on
>> libffi. It is optional, but just barely. It is only optional because
>> certain embedded platforms may have difficulty with it. It is not
>> recommended that any distro make this a build option or build p11-kit
>> without libffi.
>> The reason for the API change, is that we need multiple callers of
>> p11-kit to keep their own state and PKCS#11 module pointers, rather than
>> relying on a global list.
> 
> Hello Stef,
>  It seems I have missed this discussion (I'm unfortunately too busy
> right now and my brain selectively fades things into background). I
> don't quite know what libffi do, 

libffi is a very small library that allows building functions calls and
closures in memory on the fly.

> but is it really required to keep a
> context per user?

Yes per caller.

I did not take adding the libtasn1 and libffi dependencies to p11-kit
lightly. And I've worked hard to make them be optional, albeit with
significant functional loss when not compiled in.

This is about a deficiency in PKCS#11 v2.x and one that is very relevant
to gnutls.

If multiple callers of a PKCS#11 module both call functions like
C_Initialize, C_Finalize, C_CloseAllSessions or several other functions,
they will stomp on each other. There must be a way to coordinate these
calls.

I'm working in the PKCS#11 OASIS TC to see how we can this in future
versions of the specification. Unfortunately solving this means
significant changes, which will incompatible with current PKCS#11.

So for the time being p11-kit needs to coordinate between callers.

In order to do this it must subclass some of the the function calls to
the modules, add its own implementation, and then call the base module
function.

However, unlike vtables used elsewhere (ie: C++, or other specs like
CSSM/CDSA), the CK_FUNCTION_LIST function pointers does not pass a
CK_FUNCTION_LIST vtable to the functions themselves. This is a big flaw.

Therefore in order to subclass these functions, a simple closure is
required (one that contains a single pointer, the vtable). Such as the
closures that libffi provides.

libffi is used by all sorts of projects, and has become the defacto way
to build function calls and closures.

 * It has no dependencies other than C.
 * It has been ported to an amazing amount of architectures, far more
   than either p11-kit or gnutls.
 * It's widely used, by far more projects than listed on its page.
 * It's license is about as liberal as you can get.

See: http://sourceware.org/libffi/

It is also embedded in many projects (Mozilla does it, cpython does it).
If linking to an outside library is a show stopper for you, I guess we
could embed it as well. For obvious reasons embedding is sub-optimal.
But if necessary, it's an option I'm willing to consider.

> I really liked that p11-kit was a thin layer over PKCS #11 that offers
> quite some goodies in addition (such as url parsing). Such a
> dependency makes it no longer thin,

I don't see why it's less 'thin'.

First of all libffi is very 'thin' as described above. It doesn't
even allocate its own memory.

Secondly, p11-kit already subclassed C_Initialize and C_Finalize, albeit
poorly. We did so by saying "don't call them those functions, call us
instead". However this does not scale.

In order to have OpenSSL and NSS be able to use p11-kit's coordination,
we need to be able to have them call the normal PKCS#11 function
pointers, and still provide the functionality desired.

Note that we do not subclass all the functions. That's one benefit of
using the libffi approach. Most of the PKCS#11 function pointers just
continue to call through to the PKCS#11 module.

Lastly, for the places where this really matters, it is possible to
build p11-kit without any of this coordination. In addition it is also
possible to build gnutls without PKCS#11 support.

> and I don't quite understand the
> advantage of it. Were there any practical issues with the global
> state?

So in short:

In order to have multiple callers (both multiple callers of gnutls *and*
callers of mulitple libraries, such as gnutls, NSS, etc.) within the
same process use PKCS#11 v2.x reliably, coordination is necessary.

To provide such coordination, we must be able to subclass the PKCS#11
functions. To do that libffi is the obvious choice.

But please do push back if all of this continues to be unacceptable. I
want p11-kit to be accepted, and your feedback (now that you understand
more about libffi and the issue at hand) is super welcome.

In addition if having the 'trust' stuff and its libtasn1 optional
dependency in the main p11-kit project provides an obstacle to you, that
is an obvious candidate for separation. This is far easier than getting
away from the above libffi solution. Obviously splitting that out
creates more work for me, but if presented with valid reasons I'm
willing to make the trade-off.

> The (bad) way PKCS #11 is designed makes global pointers quite
> unavoidable, so I think anyone using pkcs11 should be familiar with
> that already.

Right, but PKCS#11 modules (and to some extent certificate/crypto usage
in general) hasn't really been used outside of browsers. This is about
making more wide spread usage possible.

When you have a browser just using NSS, which coordinates the PKCS#11
modules then it coordinates the calls. The instant you have gnutls also
trying to use PKCS#11 modules (perhaps because of a dependency) then
things need coordination, which is what p11-kit is supposed to provide.

This is not a theoretical case. See Evolution, the mail client.

Cheers,

Stef