[PATCH v2 1/4] doc: document the enum and bitfield attributes

[Auke Booij]

On 22 October 2015 at 02:46, Bill Spitzak <spitzak at gmail.com> wrote:
> Sorry if this is a duplicate, I am having trouble with gmail.
>
> On 10/20/2015 12:38 AM, Bryce Harrington wrote:
>>
>> On Tue, Oct 20, 2015 at 12:01:14AM -0700, Bryce Harrington wrote:
>>>
>>> On Mon, Oct 19, 2015 at 11:21:23PM +0100, Auke Booij wrote:
>>>>
>>>> Introduce the enum and bitfield attributes, which allow you to refer to
>>>> the enum
>>>> you are expecting in an argument, and specify which enums are to be
>>>> thought of
>>>> as bitfields.
>>>>
>>>> +      Additionally, the protocol can specify <type>enum</type>s.  These
>>>> are used
>>>> +      to list options for <type>int</type> and <type>uint</type> type
>>>> arguments.
>>>> +      Arguments can refer to the specific enumeration that is
>>>> semantically
>>>> +      implied.  Only in the case that the argument is of type
>>>> <type>uint</type>,
>>>> +      it can be specified that the primary interface to its numeric
>>>> value deals
>>>> +      with bitwise operations, for example when arbitrarily many
>>>> choices of the
>>>> +      enum can be ORed together.
>>>> +    </para>
>>>> +    <para>
>>>> +      The purpose of the <type>enum</type> and <type>bitfield</type>
>>>> attributes
>>>> +      is to document what arguments refer to which enums, and to
>>>> document which
>>>> +      numeric enum values are primarily accessed using bitwise
>>>> operations.
>>>> +      Additionally, the enum and bitfield attributes may be used by
>>>> other code,
>>>> +      such as bindings to other languages, for example to enhance type
>>>> safety of
>>>> +      code.  However, such usage is only supported if the following
>>>> property is
>>>> +      satisfied: code written prior to the specification of these
>>>> attributes
>>>> +      still works after their specification.  In other words,
>>>> specifying an
>>>> +      attribute for an argument, that previously did not have an enum
>>>> or
>>>> +      bitfield attribute, should not break API.  Code that does not
>>>> satisfy this
>>>> +      rule is not guaranteed to obey backwards compatibility.
>>>
>>> This next chunk gets a bit too jarringly technical too quickly.  I think
>>> your second paragraph gives a better intro to these attributes, but it
>>> doesn't work to simply swap them.  Let me take a shot at copyediting
>>> this a bit:
>>>
>>> I think this is clearer, and hopefully hasn't lost any meaning.  I'm not
>>> sure it's improved the technicality of this prose...  perhaps this
>>> section would be better promoted to its own section, with maybe just a
>>> reference sentence included here?  Not sure.
>>
>> I'm noticing now that I've misunderstood what the bitfield attribute is;
>> so the above text is incorrect.  Let me try again.
>>           Additionally, the protocol can specify <type>enum</type>s which
>>         associate specific numeric enumeration values.  These are
>>         primarily just description in nature: at the wire format level
>>         enums are just integers.  But they also serve a secondary purpose
>>         to enhance type safety or otherwise add context for use in
>>         language bindings or other such code.  This latter usage is only
>>         supported so long as code written before these attributes were
>>         introduced still works after; in other words, adding an enum
>>         should not break API, otherwise it puts backwards compatibility
>>         at risk.
>>
>>         <type>enum</type>s can be defined as bitfields or just a set of
>>         integers.  This is specified via the <type>bitfield</type>
>>         boolean attribute in the <type>enum</type> definition.  If this
>>         attribute is true, the enum is intended to be accessed primarily
>>         using bitwise operations, for example when arbitrarily many
>>         choices of the enum can be ORed together; if it is false, or the
>>         attribute is omitted, then the enum arguments are a just a
>>         sequence of numerical values.
>>
>>         The <type>enum</type> attribute can be used on either
>>         <type>uint</type> or <type>int</type> arguments, however if the
>>         <type>enum</type> is defined as a <type>bitfield</type>, it can
>>         only be used on <type>uint</type> args.
>
>
> This version definitely better, though you might be able to skip the "This
> is specified via..." sentence since it is describing low-level xml syntax
> that it seems the rest is skipping (for instance there is no description of
> the syntax to actually assign name-value pairs).
>
> I'm not really certain about the back-compatiblity section. Does this mean
> that if previously somebody was able to pass 42 unchanged to a message, that
> it still has to work? Or are we allowed to assume that previously people
> used the enumeration names and not raw numbers? I think it might be best to
> delete this. Treatment of back-compatibility is probably something that the
> language binding writers can decide.

The underlying reason for this part of the specification is, on the
one hand, that it protects protocol developers: this allows them to
freely specify enum attributes that weren't previously specified
(which is a good thing because we want protocol developers to specify
these things and help us), and introduce enum values that weren't
previously present (which is an absolute necessity for e.g.
xdg_surface::state, see
http://cgit.freedesktop.org/wayland/weston/tree/protocol/xdg-shell.xml#n321
). On the other hand, it forces bindings writers to write bindings
that don't break when a change is introduced in the protocol files
that *should* have been backwards compatible, such as specifying an
attribute that was previously was not specified (the enum="" attribute
in particular), or switching on the bitfield attribute somewhere.

Of course, bindings writers are still free to do so, but in that case
this specification clarifies that it is their problem if their code
doesn't compile anymore after a wayland/weston upgrade.

See also the discussion with Pekka Paalanen around the 2nd of October
on this list, and Daniel Stone, 8th of October.

> Bitfields also allow certain values to be tested somehow, ie there is a
> test(v, enum_constant) function of some type that returns true/false. It is
> ok if this only works for enum_constants whose value is a power of 2. Not
> sure how to word this...
>
> "sequence" might also be misleading. The values do not have to be
> consecutive. In particular there may be more than one name for the same
> value. Maybe use "set" instead.
>
> May also want to indicate that despite the support for int, I think negative
> enum values are not allowed.

Where does this introduce breakage? (Maybe this is worth a new thread.)

> My attempt:
>
>        Additionally, the protocol can specify <type>enum</type>s which
>        associate a set of names with numeric values and indicate that
>        only those names should be used for this argument. These are
>        primarily just description in nature: at the wire format level
>        enums are just integers. But a language binding can use
>        them to enhance type safety and provide descriptive context.
>
>        <type>enum</type>s can be defined as bitfields or just a set of
>        integers. A bitfield enum must support the ability to 'or' together
>        multiple named values to produce a value that can be put in a
> message,

This suggests that the "bitwise or" of any two values in a bitfield
enum makes a new legal value, which the C guys are not willing to
accept. And what about the bitwise or of an empty set of values (ie no
bits set)? This reboots the discussion that we had a long time ago.

>        and the ability to test a value from an event against any value that
>        is a power of two.

Does that include 2^5269762435879 ? Does it include 1 << 31? What are
the boundaries here? What does "test" mean here?

>        A non-bitfield enum does not
>        have to support this and it may be desirable to prohibit it.
>
>        In a few cases there will be the need to pass arbitrary numbers
>        for an enum value. A language binding will need to provide some
>        way to convert an arbitrary number to the enum value, and to compare
>        an enum value to an arbitrary number.

I don't like the phrasing of this: a language binding doesn't "need"
to do anything. Can this be phrased more positively? Is this worth
specifying? At first sight it looks biased towards strongly typed
languages.

>         A <type>uint</type> argument should be used (this is provided for
>        back-compatibility with parsers that predate the enum attribute).
>        For historic reasons, non-bitfield attributes can use a
>        <type>int</type> instead of <type>uint</type> but this should
>        not be done for any new protocol.

I would not be against this (although it's not exactly elegant).
However, (how) can we check against this in the scanner?

Apologies for disseminating your suggestions like this. Your response
is much appreciated, if only for reminding me why I ended up here.