[PATCH wayland 2/2] protocol: Add DnD actions

[Jonas Ådahl]

On Thu, Jan 14, 2016 at 11:50:29AM +0100, Michal Suchanek wrote:
> On 14 January 2016 at 01:54, Carlos Garnacho <carlosg at gnome.org> wrote:
> > Hi Michal,
> >
> > On Mon, Jan 11, 2016 at 11:34 AM, Michal Suchanek <hramrach at gmail.com> wrote:
> >> On 24 December 2015 at 01:58, Carlos Garnacho <carlosg at gnome.org> wrote:
> >>
> >>> @@ -757,6 +883,40 @@
> >>>        <arg name="id" type="new_id" interface="wl_data_device"/>
> >>>        <arg name="seat" type="object" interface="wl_seat"/>
> >>>      </request>
> >>> +
> >>> +    <!-- Version 3 additions -->
> >>> +
> >>> +    <enum name="dnd_action" bitfield="true" since="3">
> >>> +      <description summary="drag and drop actions">
> >>> +        This is a bitmask of the available/preferred actions in a
> >>> +        drag-and-drop operation.
> >>> +
> >>> +        In the compositor, the selected action comes out as a result of
> >>> +        matching the actions offered by the source and destination sides.
> >>> +        "action" events with a "none" action will be sent to both source
> >>> +        and destination if there is no match. All further checks will
> >>> +        effectively happen on (source actions ∩ destination actions).
> >>> +
> >>> +        In addition, compositors may also pick different actions in
> >>> +        reaction to key modifiers being pressed, one common ground that
> >>> +        has been present in major toolkits (and the behavior recommended
> >>> +        for compositors) is:
> >>> +
> >>> +        - If no modifiers are pressed, the first match (in bit order)
> >>> +          will be used.
> >>> +        - Pressing Shift selects "move", if enabled in the mask.
> >>> +        - Pressing Control selects "copy", if enabled in the mask.
> >>> +
> >>> +        Behavior beyond that is considered implementation-dependent.
> >>> +        Compositors may for example bind other modifiers (like Alt/Meta)
> >>> +        or drags initiated with other buttons than BTN_LEFT to specific
> >>> +        actions (e.g. "ask").
> >>> +      </description>
> >>> +      <entry name="none" value="0"/>
> >>> +      <entry name="copy" value="1"/>
> >>> +      <entry name="move" value="2"/>
> >>> +      <entry name="ask" value="4"/>
> >>> +    </enum>
> >>>    </interface>
> >>>
> >>
> >> Hello,
> >>
> >> how do you go about implementing those implementation-specific actions?
> >
> > One detail I think you've missed: it's compositors who decide the
> > action. Clients on both sides will only receive wl_data_source events
> > on the source side and wl_data_device/wl_data_offer events on the
> > destination side.
> 
> That's exactly the problem. The specification suggests that clients
> can implement other application-specific behaviour but there is no way
> to do that.

There are no application specific action/behaivour in the spec. The most
application specific thing there is are application specific mime types.

> 
> >
> >>
> >> Let's say I have a client which id DnD source in an operation and
> >> wants to paste text without formatting when ALT is pressed. It can
> >> technically change the offered mime-types or perform a different
> >> object conversion when sending the data if it learns the key is down.
> >> Will it learn about the key state?
> >
> > It can't technically do that, wl_data_source.offer is cummulative,
> > there's no wl_data_source.reset nor somesuch. There's also no way then
> > to have the drag destination know when/whether the mimetypes list is
> > definitive. And besides, it's the destination which chooses the most
> > suitable mimetype through wl_data_offer.accept. This sounds like the
> > opposite than the current protocol in git offers.
> >
> > Also, why should the drag source choose whether plain/formatted text
> > is transferred?
> 
> Because pressing that modifier does that for local pastes and it wants
> to do that for remote pastes as well?

The source doesn't have an idea of where it is to be dropped, why would
it be able to choose what mime type is suitable? It makes no sense. At
most the destination has some idea, but in the current protocol
additions we don't introduce modifier events.

> 
> > it'd be the drag destination which can actually tell
> > if it can manage either. Your usecase actually doesn't stand if you
> > s/without/with/, what should weston-terminal do if the source enforces
> > formatted text?
> 
> Reject it. However, if it was possible to retract an offered type then
> the source client could offer the text as any of HTML/RTF/plain or
> plain only.
> 
> >
> > Third, this patch is about DnD actions, mimetypes are just tangential,
> > besides you having to .accept one for the transfer to succeed.
> > Mimetypes are exclusively about the destination choosing the most
> > suitable/lossless/etc format.
> 
> They can be used by the source to designate what kind of information
> it is willing to transfer as well.

In the wl_data_device DND protocol, it is the destination side that
chooses the mime type (i.e. "what kindof information"). Are you
proposing to change that to the opposite? Why would you even want to
base such a protocol on wl_data_device?

> 
> >
> >>
> >> Another example would be a client that is DnD destination and wants to
> >> paste only text style when the user holds the 10th mouse button or 5th
> >> touchscreen softbutton when the object is dropped. Will it get to know
> >> that the event happened? It will probably want to reject move action
> >> in this case since the object is not fully transferred. There is no
> >> mime type for text style there is no way to transfer it other than
> >> transferring whole formatted text clip and then trashing the text (and
> >> it's going to be hairy if the text is not formatted uniformly).
> >
> > What is "text style"?
> 
> Everything that is lost when the piece of text is converted to text/plain.

If a source has the abilitity to offer rich text format it should do so
by offering a mime type which supports rich text. If it has the ability
to strip out the rich text-ness, it should offer the plain/text mime
type. It has no place to choose what the destination should request.

> 
> > Why must be "move" rejected?
> 
> It *should* be rejected as a hint to the source that the piece of text
> was not transferred.
> 
> Sure it is the source responsible for destroying (or not) the source
> data on its side once it has been transferred with a move action. So
> it is the responsibility of the destination to reject move action when
> it knows it is going to trash substantial portion of the transferred
> data.
> 
> > Why must this mode
> > be chosen during DnD instead of through a different mimetype on
> > .start_drag, or destination-side UI through an "ask" action?
> 
> It cannot be done by the source by different mime type because it does
> not know about this modifier binding in the destination. The
> destination cannot convey the intent using mime type either. The
> intent is that it receives formatted text (eg text/html or text/rtf)
> from a source possibly oblivious to its internal data conversion and
> converts it to style applied to a piece of text (probably current
> selection but whatever).
> 
> Besides, you just rejected using mime types for hinting what kind of
> content is going to be transferred saying mime types are only
> tangential to this specification. So what is it?
> 
> >
> > How do you know if the user has a 10th mouse button or a softbutton?
> 
> Because the user set it in application preferences.
> 
> > How do you fallback if he doesn't?
> 
> Use a different modifier for ask action. Which is again not part of
> the spec and we are back to the same problem - do you do that
> client-side? You cannot expect the user to plan for it before starting
> the drag and set an ask modifier-lock in the destination beforehand.
> 
> > as for the softbutton case the
> > compositor seems again in a better situation to know so (as it is the
> > case with a11y considerations).
> >
> > I'm really not sure which model you're pushing for, do you want the
> > drag source or destination to handle buttons/modifiers? both?
> 
> I am pushing for the DnD to be practically usable for implementing DnD
> in applications that have existing conventions or settings for
> modifiers. Saying two modifiers ought to be enough for everyone is not
> going to cut it. Unless the clients can do something about the
> modifiers this is going to suck.
> 
> > have you
> > considered compatibility with Xdnd?
> 
> As compatibility with X is always best effort so long as something can
> be transferred it's probably ok.

No, "best effort" is not Ok. Compatibility with XDND means compatibility
with the rest of the whole world for probably quite some time. Part of
the reason why the current proposal looks like it does is for making it
possible to interoperate with XDND via Xwayland.

You seem to mostly care about use cases where the applications use
special modifier or button combinations to change the mime-type that is
transfered. This something the current additions doesn't support, since
it is the compositor that takes the input grab during the grab and we
don't introduce any new modifier events anywhere. This, however, doesn't
mean it is impossible to introduce modifier events in some way in the
future.

> 
> > This all was already discussed in
> > ealier threads with no clear gains on either other option. And being
> > at v9 of this patch, I'm not personally keen on modifying such
> > fundamental aspect.
> >
> 
> 
> Without that I doubt many people will be keen on using it.

On the contrary, these improvements already make it possible for the
majority of DnD clients to start working properly. It's a step in the
right direction, but to start supporting every thinkable use case from
the beginning, before we actually have real world examples of how it
should actually work is risky business, and that is not the intention of
the discussed patches.


Jonas

> 
> 
> Thanks
> 
> Michal
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel