libweston backend configuration API candidates

Tue Feb 16 14:29:06 UTC 2016

Hello Pekka,

Thanks for your review. I agree with you analyses but I have few additions.

On 02/16/16 11:45, Pekka Paalanen wrote:
> Hi all,
> 
> now we have three API candidates with patches (and my own idea[1]
> without patches so it doesn't count). Here is what I have gathered, let
> me know if I got something wrong.
> 
> 
> Giulio's proposal:
> https://patchwork.freedesktop.org/patch/67547/
> 
> It uses transparent structs that get passed through a generic function
> in libweston to the loaded backend. Configuration format is part of
> libweston ABI.
> 
> My old opinion stays: I am ok with the approach, though we probably
> want to version the structs. Something like:
> 
> ------------------------------- src/compositor.h -------
>  
>  /* Configuration struct for a backend.
>   *
>   * This struct carries the configuration for a backend, and it's
>   * passed to the backend's init entry point. The backend will
>   * likely want to subclass this in order to handle backend specific
>   * data.
>   */
>  struct weston_backend_config {
> +	/** Major version for the backend-specific config struct
> +	 *
> +	 * This version must match exactly what the backend expects, otherwise
> +	 * the struct is incompatible.
> +	 */
> +	uint32_t major_version;
> +
> +	/** Minor version of the backend-specific config struct
> +	 *
> +	 * This must be set to sizeof(struct backend-specific config).
> +	 * If the value here is smaller than what the backend expects, the
> +	 * extra config members will assume their default values.
> +	 *
> +	 * A value greater than what the backend expects is incompatible.
> +	 */
> +	size_t minor_version;
>  };
> 
> ABI churn is likely, but I am not particularly concerned about it for
> now.
> 
> With such struct versioning, it is possible to add new options without
> breaking the ABI, as long as all additions are made to the end of a
> struct and nothing existing is ever modified or removed. If existing
> things need to be changed, major_version is bumped, which essentially
> corresponds to an ABI break. Even then, if wanted, a backend could
> choose to support several major versions. Whether all this flexibility
> is actually useful, I am not sure. We could probably be fine with only
> minor_version, and use library version bumps for major.
> 
> 
> Benoit's proposal:
> https://patchwork.freedesktop.org/patch/73206/
> 
> This is based on Giulio's proposal, except the config structs are now
> opaque. The config structs are created and filled with function calls.
> These functions are exported in libweston.so, which means it exports
> many backend-specific functions. However, these functions do not
> require any backend dependencies to be linked in, so no needless
> libraries are pulled in.

The benefit of those functions call is that those functions can report
errors, and the compositor can handle them in his preferred manner. In
#1 you should have something like a function validate_configuration to
handle configuration. It also follow the well-unknown object paradigm
where data are not accessible directly.

> 
> Config versioning is tied to the library versioning. This makes it
> harder to support multiple libweston versions where the only difference
> is adding some configuration options. The compositor must use dlsym()
> for any functions it can live without but wants to use if available at
> runtime.

A possible approach to decouple libweston version and backend
configuration version is to create severals libbackends, where each of
them can maintain there own versioning and can be built outside
libweston. In that case we delegate the job to gcc/ld.

> 
> As a detail, Giulio's proposal has a callback for configuring
> (hotplugged) outputs, while Benoit passes the known output
> configurations to the backend on start-up. The benefit of the callback
> is that output default settings are controlled by the compositor, not
> libweston or the backend. That is why I would prefer to have the
> callback. The defaults may vary arbitrarily per output, and later
> layout could perhaps be part of the configuration.

A side notes that doesn't relate directly to the configuration API.
Personally, I don't like callbacks if there is no strong reason to use
them. For severals reasons:
 - they must be well designed to support any thing that can happen
during the callback. In particular the developer may call any function
in the API that may be mutually exclusive to the current callback or it
also may call a function that will produce an infinite recursive call to
that callback. Those cases can be difficult to fix.
 - they can be abused or miss used easily, for example: using the order
of callback call to detect which libweston version is in use then change
the compositor behavior accordingly, instead of using the proper
get_version. This example is far-fetched but we can find many trick or
workaround in computer science.

This argument also applies latter.

> 
> 
> Quentin's proposal:
> https://patchwork.freedesktop.org/patch/73039/
> https://patchwork.freedesktop.org/patch/73035/
> https://patchwork.freedesktop.org/patch/73037/
> https://patchwork.freedesktop.org/patch/73036/
> https://patchwork.freedesktop.org/patch/73038/
> 
> This proposal first sets out to build a convenience library
> libweston.la to create more structure in the source tree. It seems we
> sorely needed that, because the assumptions here are the opposite from
> what has already landed in upstream. Upstream has compositor.c and
> struct weston_compositor as libwayland items, from which we slowly
> extract the things that do not belong in libweston. Quentin's proposal
> assumes compositor.c to be initally outside of libweston and slowly
> moving all applicable bits into libweston under the new lib/ directory.
> This confusion has to be taken into account when looking at the pathces.
> 
> The essential idea of this API is, that the compositor will register
> configuration entry getter functions with libweston. These functions
> are per config item type: a getter for ints, a getter for strings, etc.
> Backends will then call these getters to retrieve configuration values
> one by one.
> 
> The configuration options are identified explicitly by { section, key }
> name tuples rather than a key name alone in the API. The getters get
> also passed in a default value picked by the backend, in case the
> compositor does not recognize what it is being asked for.
> 
> The benefits of this approach include that the library ABI is very
> stable, as it includes only per-type getter functions. The compositor
> implementation is also free to pass unrecognized options through:
> adding a new option to a backend does not necessarily require updating
> the compositor to understand it, as a user can make the setting in a
> configuration file and it will be passed as "data" through to the
> backend.
> 
> Configuring dynamically added outputs is no different: a backend will
> just query some more options.
> 
> What I see as the downsides here are the (arguable) complexity of
> setting up the getters, and not having an explicit way of knowing
> whether all your options were actually used - you'd have to track that
> yourself in your compositor. Of course, structs have the same problem,
> so at least here you could check it if you wanted. Detecting e.g.
> mispelled configuration items is fairly hard, because the decision of
> what is a valid key or not is hidden inside libweston.
> 
> This is also lacking any explicit notion of a transaction. With structs
> it's easy: you pass a pointer to a function, and once the function
> returns, the configuration is in. Here we need to document which
> function calls query which configurations at what time. This can be
> awkward with output hotplugging, and the compositor may not even know
> when libweston/backend has finished querying the options.

Here I will just add that this solution imply that callbacks must be
invariant for a given section/key or the backend must call section/key
only once.

I also add here the already given issues of callbacks.

> 
> I also see the unrecognized option pass-through to the backend as a
> double-edged sword. It fits perfectly, if users are expecting to be
> configuring libweston. However, I believe users are primarily using the
> compositor, not libweston. The compositor may make assumptions on how
> the backend is configured, and if a user can override those
> assumptions, it's an opportunity for the user to shoot himself in the
> foot by blindly copying instructions from shady websites. Of course, a
> compositor can prevent this, but is there enough reason to support
> pass-through in the first place?
> 
> In summary, Quentin's proposal seems the most flexible, which naturally
> makes it the hardest to program for, while also promising a very stable
> library ABI.
> 
> 
> I can't see an obvious winner in any of the above, they all can be made
> to work, and do not seem to have particularly huge disadvantages at the
> moment. We need to decide which properties we value the most.
> 
> In that light, I think the person who will be implementing the config
> API and converting everything over should make the call. It's not like
> we can't change it later, it would just be more work.

I will try to implements #2 for others backends in independent patches,
your comments on the code are welcome.

> 
> 
> Thanks,
> pq
> 
> [1] https://lists.freedesktop.org/archives/wayland-devel/2015-December/026100.html
> 

Once again, Thanks for your review,

Best regards.