[PATCH wayland] Add "enum" attribute to "arg" elements

[Auke Booij]

On 27 April 2015 at 14:49, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Mon, 27 Apr 2015 13:26:39 +0200
> Auke Booij <auke at tulcod.com> wrote:
>
>> Apologies for my lack of responses, I have been abroad for a few days.
>>
>> On 23 April 2015 at 10:38, Pekka Paalanen <ppaalanen at gmail.com> wrote:
>> >> This is a sort of sanity condition on being a bitfield: it does not
>> >> require all combinations are valid, but it also distinguishes it from
>> >> a regular enum.
>> >
>> > Is that an important distinction to make? That a bitfield with "too
>> > many" restrictions must not be a bitfield?
>>
>> Yes, because what the bitfield flag indicates is that it is useful to
>> think of the values as flags. It means that the values can somehow be
>> combined. If there is no way to validly combine them, then it is not
>> useful to think of them like flags. And if the right way to combine
>> them is not OR'ing, then what you are talking about is not a bitfield.
>> I am not saying it is a kind of value that is unwelcome in the wayland
>> API, merely that it is not in the same category as what is usually
>> considered a bitfield.
>>
>> And indeed some things (e.g. wl_shell_surface.resize and
>> xdg_surface.resize_edge) kinda look like a bitfield, but it is up for
>> discussion whether they actually are, since it also looks like all of
>> the options are explicitly listed, so in that sense we do not need to
>> combine values by OR'ing, and it is hence not a bitfield.
>>
>> Bill's 23rd of April (pseudo)code is very accurate, and highlights the
>> issues I am trying to address in my bindings. Bill, thanks for that.
>>
>> Pekka, you keep suggesting "docenum" only be used for documentation.
>> However, types in richly typed languages *are* documentation (*). They
>> indicate how you should use certain values. The fact that values are
>> packaged as a certain enum type *documents* that you should probably
>> use them in this or that way, without completely blocking use of the
>> raw data (unless Jeroen's plan goes through).
>
> Ok, let me clarify: changing documentation alone cannot cause
> failures. That is what I mean by documentation-only here.
>
> Sure, you can use language constructs in a documentative way, but the
> main point here is can this cause build or runtime failures if it
> suddenly changes.
>
>> (*) In fact, in the case of Haskell, more often than not, you can find
>> functions in an API purely by searching for its type: the type
>> information of functions documents enough to know their behaviour. In
>> some languages, the difference between code and documentation is not
>> so clear-cut.
>>
>> The entire reason I am asking for this "enum" attribute is so that the
>> code I generate is more self-documenting. It does not enable the
>> bindings to compute things it otherwise wouldn't be able to: packaged
>> (u)ints are still (u)ints. It's to help the user understand the API,
>> and unless they know better, suggest how they should interact with
>> wayland.
>
> Then I didn't understand your intent before. I thought you wanted to use
> the enum types in the API you generate, which in strictly typed
> languages means you get build failures if you try to use a value not in
> the enum.
>
> If it cannot cause build or runtime failures, then it's fine.
>
>> Jeroen's request, I think, stands separately from this: Jeroen is
>> asking for strict guarantees on what values are and are not allowed in
>> enums. Perhaps this should be moved into a separate thread?
>
> I (mistakenly?) took that using the types in the API means that you get
> these strict value guarantees Jeroen is asking for.
>
> Can you explain how you will use the enum types in the API without
> hitting the type checks of a strictly typed language? The answer is
> probably language-specific, but that is ok. We only need to answer the
> "can it break?" question, and define the attribute semantics so that it
> cannot break.
>
> In fact, that is the definition for the "docenum" attribute: changing,
> adding, or removing it from the XML file cannot cause any new build or
> runtime failures. Otherwise use it any way in a generator you want. It
> is similar to the "summary" attribute and <description> tag.
>
>> So in summary:
>>
>> - Currently, there is no systematic way to match (u)ints that are
>> passed around with enums that are defined by the protocol. This makes
>> the <enum>s, from a technical point of view, somewhat disconnected
>> from the (u)int arguments.
>>
>> - Currently, there is no systematic way to tell if bitwise
>> computations on enum values are supposed to be possible (as per Bill's
>> example code).
>>
>> - Currently, there is no guarantee that only certain (u)int values
>> will be exchanged under a given version of a protocol.
>>
>> - It would be much welcomed by bindings and documentation generators
>> if there would be a *semantic* correspondence between (u)ints and
>> <enum>s. This correspondence would merely indicate how certain (u)ints
>> are to be interpreted by matching them with an <enum>. This feature
>> alone should not generate an API that is in any sense stronger or
>> weaker than without such information (iow, access to the underlying
>> (u)ints should still be available), but it may look different (e.g.
>> (u)ints packaged in a type that refers to the right <enum>).
>
> This is the part I don't get. Either you use the strict types or you
> don't? What is the "maybe" option? Generating parallel APIs, and then
> the user must know which one is actually the right one to use? I see in
> Bill's example there are "forced casts" from arbitrary values to the
> type, but this requires the programmer to write those casts.
>
> Note, that "can it break?" also applies to cases where someone is using
> an old XML file without these enum attributes (won't know that he
> should be using forced casts, because the implementations of the casts
> aren't even available) and then switches to a new one with the
> attributes. If you can guarantee such a case cannot break, then it is
> all fine.
>
>> - In strongly typed languages, hints on generating the right API shape
>> for bitfield-style arguments would be welcomed (as per Bill's example
>> code). However, details of when something is a "bitfield" need to be
>> discussed.
>>
>> - In strongly typed languages, guarantees on which enum values are
>> allowed in the protocol would be welcomed. However, worries about
>> compatibility (both between languages and between versions) need to be
>> worked out in detail. Also, how do such guarantees combine with the
>> "bitfield" flag?
>>
>> - Whereas interface names are global, enums are named locally in
>> interfaces. However, there might be a need to refer to another
>> interface's <enum>. However, this is a problem that can be solved
>> later as it comes up (for now the enum attribute would use local
>> names). As far as I am aware, there is no such situation in any common
>> protocol right now.
>
> In the end, all I care is about the definition of the new attributes
> related to enum. If you really want to write a generator that can break
> the build of other people, you can.
>
> I would just want the definition of the attribute be such, that
> following it literally, nothing can break even if the attribute gets
> added much later than the interface it is used in. Either this, or
> adding/removing the attribute is allowed to break the generated API,
> which means we cannot add it retroactively to stable interfaces. That
> is the difference between documentation-only and not.
>
> We have added an attribute before: allow-null for object type <arg>,
> defaulting to "false". It cannot cause failures in cases that weren't
> already failing somehow anyway.
>
>
> Thanks,
> pq

Okay, I think we have hit the essence of this part of the debate.

The way I imagine my code generator in the presence of an "enum"
attribute *would break API* when this attribute is added to an XML
file.

And yes, the only real way to resolve this breakage is to have
parallel APIs: one based on raw (u)ints, and one with packaged
(u)ints. One cannot interchange them without proper casting. And no,
this is indeed not a road I would like to take.

However, and I cannot stress this enough, generally speaking, breaking
builds to due added information is preferred to not having the right
type information in the first place. Adding a few casts here and there
when needed is not much work, and the safety it adds has a lot of
value in the context of strongly typed languages. In fact, I am still
a bit confused why an API breakage in these early years of wayland is
considered such a big deal, compared with a daily struggle of not
having sufficient typing information.

I guess this stems from a mentality difference between languages.

On 27 April 2015 at 15:13, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Mon, 27 Apr 2015 15:49:27 +0300
> Pekka Paalanen <ppaalanen at gmail.com> wrote:
>
>> In the end, all I care is about the definition of the new attributes
>> related to enum. If you really want to write a generator that can break
>> the build of other people, you can.
>>
>> I would just want the definition of the attribute be such, that
>> following it literally, nothing can break even if the attribute gets
>> added much later than the interface it is used in. Either this, or
>> adding/removing the attribute is allowed to break the generated API,
>> which means we cannot add it retroactively to stable interfaces. That
>> is the difference between documentation-only and not.
>
> How about we just define a new attribute for <arg> that points to an
> <enum> for documentary purposes, and warns if the type is not int/uint
> or uint for bitfields; and add an attribute to <enum> to say it's a
> bitfield instead, without any further enforced restrictions.
>
> Then, if you want to generate API bindings that have extra
> restrictions not warranted by the above, you can. You also get to keep
> all the pieces if it breaks someone.
>
> This would be the "documentation-only, but we can't stop you from
> abusing it" solution.
>
> The chances are that it would work just fine for 99% of your
> favourite language's users. We can add the attributes retroactively,
> and you can write software that requires the version of libwayland or
> whatever that has these attributes in the XML.
>
> The point is, it will be clear on what protocol writers can and cannot
> do, and with maximum benefits.
>
>
> Thanks,
> pq

I am very much in favor of this solution. You will find me in the abuse camp.