<div dir="ltr"><div><div><div><div>Andrew, </div>I've had ideas kicking around in the back of my head for a while now about how to do just this. However, my ideas have taken a substantially different route. I don't have any real problems with what you suggested (although I didn't read it in careful detail). What follows below is more of a counter-proposal. Some time ago I proposed to the list a protocol for a fullscreen shell that only runs one client at a time. The proposal can be found here: <a href="http://lists.freedesktop.org/archives/wayland-devel/2013-August/010720.html">http://lists.freedesktop.org/archives/wayland-devel/2013-August/010720.html</a>. (As a disclaimer, I have since renamed it to wl_fullscreen_shell because that's a bit more accurate and because I do not intend for actual system compositors.) One of the purposes for this protocol is to allow you to nest compositors and use the Wayland protocol itself as an input/output abstraction. This way compositors (such as GNOME shell or Weston) can run on another system without having to have built-in back-end support for it. All that is needed is for someone to write a fairly simple compositor that provides wl_fullscreen_shell. This simple compositor could be, for instance, a VNC or RDP server. In particular this would allow any other compositor to run on top of Weston's backends because I have already implemented the fullscreen shell in Weston. </div>You may be thinking, "But I don't want to start the compositor inside my VNC server, I want it to mirror what's already on the screen." It wouldn't have be restricted to headless operation. Weston (or any other compositor) could spawn the VNC server and then immediately connect to it as a wl_fullscreen_shell client. (For security purposes, the connection could be established by passing an already opened file descriptor via an environment variable.) Instead of using that as the primary backend, Weston (or any other compositor) would simply mirror outputs to the VNC server. The assignment of seats would then be a compositor-configured thing (although I would vote for just adding the VNC seats). </div>The advantage of this method is that it requires very little repetition of protocol. If we run the VNC server as a client, we have to repeat most of the Wayland protocol only in the opposite direction. If we let the VNC server run as a server and the compositor as a client, we get all of the input handling, buffer passing, etc. for free. Let me know what you think, </div>--Jason Ekstrand </div><div class="gmail_extra"> <div class="gmail_quote">On Fri, Oct 18, 2013 at 9:50 AM, Andrew Wedgbury <<a href="mailto:andrew.wedgbury@realvnc.com" target="_blank">andrew.wedgbury@realvnc.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi, I've been working on a weston module to support an interface for remote access systems such as VNC. This is now at a point where it's usable (and works rather well with a suitably modified version of our VNC server) so I wanted to present it here with a view to including it in weston at some point. The module is named remote-access - which is intended to be a suitably generic name since it could be used by VNC or RDP (or others). It exposes a new global interface called remote-access, which can be used to create instances of remote-capture and remote-seat interfaces as required. The remote-capture interface allows a particular output to be captured. It is created by specifying a shared memory region of the required size, and the output to capture. The interface allows a single refresh to be requested, in which case an update event will signal when the update is ready to be read from the buffer. It can also operate in continuous mode, when update events are sent whenever the output changes and the buffer has been updated. The update event contains the coordinates of the rectangle that changed (in coordinates relative to the output). This part is loosely based on weston's screenshooter code. The remote-seat interface allows input events to be injected via a weston seat. The interface allows multiple remote-seat objects to be created, each of which can use an existing seat, or create a new seat. That way the remote access system has the flexibility of creating a separate seat for each connected user, or simply control an a single seat, as required. There are (at least!) two outstanding issues that I am aware of: * Access to these interfaces should be restricted somehow. This could be achieved in a way similar to screenshooter, where weston could launch the VNC/RDP server. Or weston could prompt the user somehow (although this would not be effective when trying to access a remote, unattended machine). * We need to be able to capture the pointer sprite, and detect when this changes. This would need to be part of the remote-seat interface since each seat can have a pointer. So far I've not found a method to reliably detect when the pointer changes, so I would appreciate some help on this one. I've also written a test client, but this is very much work-in-progress at this point. Any feedback would be greatly appreciated, diff follows below: --- clients/.gitignore | 3 + clients/Makefile.am | 13 + clients/remote-test.c | 437 +++++++++++++++++++++++++++++++ <a href="http://configure.ac" target="_blank">configure.ac</a> | 8 + protocol/Makefile.am | 3 +- protocol/remote-access.xml | 160 ++++++++++++ src/.gitignore | 2 + src/Makefile.am | 14 + src/remote-access.c | 612 ++++++++++++++++++++++++++++++++++++++++++++ 9 files changed, 1251 insertions(+), 1 deletion(-) diff --git a/clients/.gitignore b/clients/.gitignore index 23959cc..3e98fcf 100644 --- a/clients/.gitignore +++ b/clients/.gitignore @@ -45,3 +45,6 @@ weston-multi-resource workspaces-client-protocol.h workspaces-protocol.c weston-simple-im +remote-access-protocol.c +remote-access-client-protocol.h +weston-remote-test diff --git a/clients/Makefile.am b/clients/Makefile.am index 4f9dc48..61c5b7a 100644 --- a/clients/Makefile.am +++ b/clients/Makefile.am @@ -19,6 +19,7 @@ libexec_PROGRAMS = \ $(desktop_shell) \ $(tablet_shell) \ $(screenshooter) \ + $(remote_test) \ $(screensaver) \ $(keyboard) \ weston-simple-im @@ -90,6 +91,10 @@ endif screenshooter = weston-screenshooter +if ENABLE_REMOTE_ACCESS +remote_test = weston-remote-test +endif + noinst_LTLIBRARIES = <a href="http://libtoytoolkit.la" target="_blank">libtoytoolkit.la</a> libtoytoolkit_la_SOURCES = \ @@ -118,6 +123,12 @@ weston_screenshooter_SOURCES = \ ../shared/os-compatibility.h weston_screenshooter_LDADD = $(CLIENT_LIBS) +weston_remote_test_SOURCES = \ + remote-test.c \ + remote-access-protocol.c \ + remote-access-client-protocol.h +weston_remote_test_LDADD = <a href="http://libtoytoolkit.la" target="_blank">libtoytoolkit.la</a> + weston_terminal_SOURCES = terminal.c weston_terminal_LDADD = <a href="http://libtoytoolkit.la" target="_blank">libtoytoolkit.la</a> -lutil @@ -217,6 +228,8 @@ weston_tablet_shell_LDADD = <a href="http://libtoytoolkit.la" target="_blank">libtoytoolkit.la</a> BUILT_SOURCES = \ screenshooter-client-protocol.h \ screenshooter-protocol.c \ + remote-access-client-protocol.h \ + remote-access-protocol.c \ text-cursor-position-client-protocol.h \ text-cursor-position-protocol.c \ text-protocol.c \ diff --git a/clients/remote-test.c b/clients/remote-test.c new file mode 100644 index 0000000..13594e4 --- /dev/null +++ b/clients/remote-test.c @@ -0,0 +1,437 @@ +/* + * Copyright © 2013 RealVNC Limited + * + * Permission to use, copy, modify, distribute, and sell this software and its + * documentation for any purpose is hereby granted without fee, provided that + * the above copyright notice appear in all copies and that both that copyright + * notice and this permission notice appear in supporting documentation, and + * that the name of the copyright holders not be used in advertising or + * publicity pertaining to distribution of the software without specific, + * written prior permission. The copyright holders make no representations + * about the suitability of this software for any purpose. It is provided "as + * is" without express or implied warranty. + * + * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, + * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO + * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR + * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, + * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER + * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE + * OF THIS SOFTWARE. + */ + +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <limits.h> +#include <sys/param.h> +#include <sys/mman.h> +#include <sys/select.h> +#include <sys/time.h> +#include <sys/types.h> +#include <cairo.h> +#include <termios.h> +#include <linux/input.h> + +#include <wayland-client.h> +#include "remote-access-client-protocol.h" +#include "../shared/os-compatibility.h" + +#define US_PER_SECOND 1000000 + +static struct wl_shm *shm; +static struct remote_access *remote_access; +struct remote_seat *remote_seat; +int ptr_x, ptr_y; +static struct wl_list output_list; +int min_x, min_y, max_x, max_y; +int updates; +long int max_update_time = 1 * US_PER_SECOND; + +struct remote_access_output { + struct wl_output *output; + struct wl_buffer *buffer; + int width, height, offset_x, offset_y; + void *data; + struct wl_list link; +}; + +// wl_output + +static void +display_handle_geometry(void *data, + struct wl_output *wl_output, + int x, + int y, + int physical_width, + int physical_height, + int subpixel, + const char *make, + const char *model, + int transform) +{ + struct remote_access_output *output; + + output = wl_output_get_user_data(wl_output); + + if (wl_output == output->output) { + output->offset_x = x; + output->offset_y = y; + } +} + +static void +display_handle_mode(void *data, + struct wl_output *wl_output, + uint32_t flags, + int width, + int height, + int refresh) +{ + struct remote_access_output *output; + + output = wl_output_get_user_data(wl_output); + + if (wl_output == output->output && (flags & WL_OUTPUT_MODE_CURRENT)) { + output->width = width; + output->height = height; + } +} + +static const struct wl_output_listener output_listener = { + display_handle_geometry, + display_handle_mode +}; + + +// remote-access + +static void +remote_capture_update(void *data, struct remote_capture *remote_capture, + int32_t x, int32_t y, int32_t width, int32_t height) +{ + fprintf(stdout, "remote_capture_update: %d,%d %dx%d\n", + x, y, width, height); + ++updates; +} + +static const struct remote_capture_listener remote_capture_listener = { + remote_capture_update +}; + + +// registry + +static void +handle_global(void *data, struct wl_registry *registry, + uint32_t name, const char *interface, uint32_t version) +{ + static struct remote_access_output *output; + + if (strcmp(interface, "wl_output") == 0) { + output = malloc(sizeof *output); + output->output = wl_registry_bind(registry, name, + &wl_output_interface, 1); + wl_list_insert(&output_list, &output->link); + wl_output_add_listener(output->output, &output_listener, output); + + } else if (strcmp(interface, "wl_shm") == 0) { + shm = wl_registry_bind(registry, name, &wl_shm_interface, 1); + + } else if (strcmp(interface, "remote_access") == 0) { + remote_access = wl_registry_bind(registry, name, + &remote_access_interface, 1); + } +} + +static void +handle_global_remove(void *data, struct wl_registry *registry, uint32_t name) +{ + /* XXX: unimplemented */ +} + +static const struct wl_registry_listener registry_listener = { + handle_global, + handle_global_remove +}; + + +static struct wl_buffer * +create_shm_buffer(int width, int height, void **data_out) +{ + struct wl_shm_pool *pool; + struct wl_buffer *buffer; + int fd, size, stride; + void *data; + + stride = width * 4; + size = stride * height; + + fd = os_create_anonymous_file(size); + if (fd < 0) { + fprintf(stderr, "creating a buffer file for %d B failed: %m\n", + size); + return NULL; + } + + data = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + if (data == MAP_FAILED) { + fprintf(stderr, "mmap failed: %m\n"); + close(fd); + return NULL; + } + + pool = wl_shm_create_pool(shm, fd, size); + close(fd); + buffer = wl_shm_pool_create_buffer(pool, 0, width, height, stride, + WL_SHM_FORMAT_XRGB8888); + wl_shm_pool_destroy(pool); + + *data_out = data; + + return buffer; +} + +static void +write_png(const char* filename, int width, int height) +{ + int output_stride, buffer_stride, i; + cairo_surface_t *surface; + void *data, *d, *s; + struct remote_access_output *output, *next; + + buffer_stride = width * 4; + + data = malloc(buffer_stride * height); + if (!data) + return; + + wl_list_for_each_safe(output, next, &output_list, link) { + output_stride = output->width * 4; + s = output->data; + d = data + (output->offset_y - min_y) * buffer_stride + + (output->offset_x - min_x) * 4; + + for (i = 0; i < output->height; i++) { + memcpy(d, s, output_stride); + d += buffer_stride; + s += output_stride; + } + } + + surface = cairo_image_surface_create_for_data(data, + CAIRO_FORMAT_ARGB32, + width, height, + buffer_stride); + cairo_surface_write_to_png(surface, filename); + cairo_surface_destroy(surface); + free(data); +} + +static int +set_buffer_size(int *width, int *height) +{ + struct remote_access_output *output; + min_x = min_y = INT_MAX; + max_x = max_y = INT_MIN; + int position = 0; + + wl_list_for_each_reverse(output, &output_list, link) { + output->offset_x = position; + position += output->width; + } + + wl_list_for_each(output, &output_list, link) { + min_x = MIN(min_x, output->offset_x); + min_y = MIN(min_y, output->offset_y); + max_x = MAX(max_x, output->offset_x + output->width); + max_y = MAX(max_y, output->offset_y + output->height); + } + + if (max_x <= min_x || max_y <= min_y) + return -1; + + *width = max_x - min_x; + *height = max_y - min_y; + + return 0; +} + +static int +handle_input(int fd) +{ + char buffer[16]; + int n = read(fd, buffer, 16); + if (n < 0) return 0; + + if (buffer[0] != 0x1b) { + remote_seat_send_key(remote_seat, buffer[0], 1); + remote_seat_send_key(remote_seat, buffer[0], 0); + return 0; + } + + if (n == 1) return -1; // Escape key + if (n < 3) return 0; + switch (buffer[2]) { + case 0x41: + ptr_y -= 10; + remote_seat_move_pointer(remote_seat, + wl_fixed_from_int(ptr_x), + wl_fixed_from_int(ptr_y), 0); + break; + case 0x42: + ptr_y += 10; + remote_seat_move_pointer(remote_seat, + wl_fixed_from_int(ptr_x), + wl_fixed_from_int(ptr_y), 0); + break; + case 0x43: + ptr_x += 10; + remote_seat_move_pointer(remote_seat, + wl_fixed_from_int(ptr_x), + wl_fixed_from_int(ptr_y), 0); + break; + case 0x44: + ptr_x -= 10; + remote_seat_move_pointer(remote_seat, + wl_fixed_from_int(ptr_x), + wl_fixed_from_int(ptr_y), 0); + break; + case 0x32: + remote_seat_pointer_button(remote_seat, BTN_LEFT, 1); + remote_seat_pointer_button(remote_seat, BTN_LEFT, 0); + break; + case 0x31: + remote_seat_pointer_button(remote_seat, BTN_RIGHT, 1); + remote_seat_pointer_button(remote_seat, BTN_RIGHT, 0); + break; + case 0x33: + remote_seat_pointer_button(remote_seat, BTN_MIDDLE, 1); + remote_seat_pointer_button(remote_seat, BTN_MIDDLE, 0); + break; + case 0x35: + remote_seat_pointer_axis(remote_seat, + WL_POINTER_AXIS_VERTICAL_SCROLL, + wl_fixed_from_int(-10)); + break; + case 0x36: + remote_seat_pointer_axis(remote_seat, + WL_POINTER_AXIS_VERTICAL_SCROLL, + wl_fixed_from_int(10)); + break; + } + return 0; +} + +int +main(int argc, char *argv[]) +{ + struct wl_display *display; + struct wl_registry *registry; + struct remote_access_output *output; + struct timeval update_time; + struct termios old_termios, new_termios; + int fd_wl; + int fd_in = 0; + int width, height; + int seq = 0; + + display = wl_display_connect(NULL); + if (display == NULL) { + fprintf(stderr, "failed to create display: %m\n"); + return -1; + } + fd_wl = wl_display_get_fd(display); + + wl_list_init(&output_list); + registry = wl_display_get_registry(display); + wl_registry_add_listener(registry, &registry_listener, NULL); + wl_display_dispatch(display); + wl_display_roundtrip(display); + if (remote_access == NULL) { + fprintf(stderr, "display doesn't support remote_access\n"); + return -1; + } + + remote_seat = remote_access_create_seat(remote_access, "remote-test"); + ptr_x = 100; ptr_y=100; + + tcgetattr(fd_in, &old_termios); + new_termios = old_termios; + new_termios.c_lflag &= (~ICANON); + new_termios.c_lflag &= (~ECHO); + new_termios.c_cc[VTIME] = 0; + new_termios.c_cc[VMIN] = 1; + tcsetattr(fd_in, TCSANOW, &new_termios); + + if (set_buffer_size(&width, &height)) + return -1; + + wl_list_for_each(output, &output_list, link) { + output->buffer = create_shm_buffer(output->width, + output->height, + &output->data); + + struct remote_capture* capture = + remote_access_create_capture(remote_access, + output->output, + output->buffer); + remote_capture_add_listener(capture, &remote_capture_listener, + capture); + remote_capture_start(capture); + remote_capture_refresh(capture); + } + + gettimeofday(&update_time, NULL); + updates = 0; + + wl_display_roundtrip(display); + + while (1) { + fd_set fds_read; + struct timeval timeout; + struct timeval current_time; + long int delta; + char filename[PATH_MAX]; + + FD_ZERO(&fds_read); + FD_SET(fd_wl, &fds_read); + FD_SET(fd_in, &fds_read); + + timeout.tv_usec = 100; timeout.tv_sec = 0; + select(fd_wl+1, &fds_read, NULL, NULL, &timeout); + + if (FD_ISSET(fd_in, &fds_read)) { + if (handle_input(fd_in) < 0) + break; + } + + wl_display_roundtrip(display); + + gettimeofday(&current_time, NULL); + delta = (current_time.tv_usec - update_time.tv_usec) + + (current_time.tv_sec - update_time.tv_sec) * US_PER_SECOND; + if (updates && delta >= max_update_time) { + sprintf(filename, "remote-%04d.png", ++seq); + fprintf(stdout, "Writing %s [%d updates / %f s]\n", + filename, updates, (float)delta/US_PER_SECOND); + + write_png(filename, width, height); + + update_time.tv_sec = current_time.tv_sec; + update_time.tv_usec = current_time.tv_usec; + updates = 0; + } + } + + tcsetattr(fd_in, TCSANOW, &old_termios); + + //TODO: destroy remote_captures + remote_seat_destroy(remote_seat); + remote_access_destroy(remote_access); + + return 0; +} diff --git a/<a href="http://configure.ac" target="_blank">configure.ac</a> b/<a href="http://configure.ac" target="_blank">configure.ac</a> index 950086d..c069885 100644 --- a/<a href="http://configure.ac" target="_blank">configure.ac</a> +++ b/<a href="http://configure.ac" target="_blank">configure.ac</a> @@ -379,6 +379,13 @@ if test "x$enable_colord" != "xno"; then fi AM_CONDITIONAL(ENABLE_COLORD, test "x$enable_colord" = "xyes") +# Remote access module +AC_ARG_ENABLE(remote-access, + AS_HELP_STRING([--disable-remote-access], + [do not build remote access support]),, + enable_remote_access=auto) +AM_CONDITIONAL(ENABLE_REMOTE_ACCESS, test "x$enable_remote_access" = "xyes") + AC_ARG_ENABLE(wcap-tools, [ --disable-wcap-tools],, enable_wcap_tools=yes) AM_CONDITIONAL(BUILD_WCAP_TOOLS, test x$enable_wcap_tools = xyes) if test x$enable_wcap_tools = xyes; then @@ -497,4 +504,5 @@ AC_MSG_RESULT([ libwebp Support ${have_webp} libunwind Support ${have_libunwind} VA H.264 encoding Support ${have_libva} + Remote access Support ${enable_remote_access} ]) diff --git a/protocol/Makefile.am b/protocol/Makefile.am index 924e48f..b889ce4 100644 --- a/protocol/Makefile.am +++ b/protocol/Makefile.am @@ -8,4 +8,5 @@ EXTRA_DIST = \ workspaces.xml \ subsurface.xml \ text-cursor-position.xml \ - wayland-test.xml + wayland-test.xml \ + remote-access.xml diff --git a/protocol/remote-access.xml b/protocol/remote-access.xml new file mode 100644 index 0000000..a66d526 --- /dev/null +++ b/protocol/remote-access.xml @@ -0,0 +1,160 @@ +<protocol name="remote_access"> + + <copyright> + Copyright © 2013 RealVNC Limited + + Permission to use, copy, modify, distribute, and sell this + software and its documentation for any purpose is hereby granted + without fee, provided that the above copyright notice appear in + all copies and that both that copyright notice and this permission + notice appear in supporting documentation, and that the name of + the copyright holders not be used in advertising or publicity + pertaining to distribution of the software without specific, + written prior permission. The copyright holders make no + representations about the suitability of this software for any + purpose. It is provided "as is" without express or implied + warranty. + + THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS + SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND + FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY + SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN + AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, + ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF + THIS SOFTWARE. + </copyright> + + <interface name="remote_capture" version="1"> + <description summary="Output capture for remote access"> + Allows a particular output to be captured, and provides notifications + whenever updates occur. + </description> + + <request name="destroy" type="destructor"> + <description summary="Destroy output capture"> + Destroy the output capture object. + </description> + </request> + + <request name="refresh"> + <description summary="Request a single capture"> + Request a single capture of the output. + An update event is sent when the capture data is ready in the buffer. + </description> + </request> + + <request name="start"> + <description summary="Start continuous capture"> + Request that the output is captured continuously. + An update event is sent whenever the output changes. + </description> + </request> + + <request name="stop"> + <description summary="Stop continuous capture"> + Stop an ongoing continuous capture. No more updates will be sent. + </description> + </request> + + <event name="update"> + <description summary="Notification of an update"> + Notification that an area of the output has changed. + The arguments x,y,width,height describe the rectangle that changed. + </description> + <arg name="x" type="int"/> + <arg name="y" type="int"/> + <arg name="width" type="int"/> + <arg name="height" type="int"/> + </event> + </interface> + + <interface name="remote_seat" version="1"> + <description summary="Handles input events for remote access"> + Allows remote access clients to inject input events. The remote access + system may choose to create separate remote seats for each of its clients, + or have them all use a single seat. + </description> + + <request name="destroy" type="destructor"> + <description summary="Destroy remote seat"> + Destroy the remote seat object. + </description> + </request> + + <request name="move_pointer"> + <description summary="Send pointer movement event"> + Inject a pointer movement event from the remote system. + </description> + <arg name="x" type="fixed"/> + <arg name="y" type="fixed"/> + <arg name="relative" type="uint"/> + </request> + + <request name="pointer_button"> + <description summary="Send pointer button event"> + Inject a pointer button event from the remote system. + </description> + <arg name="button" type="uint"/> + <arg name="state" type="uint"/> + </request> + + <request name="pointer_axis"> + <description summary="Send pointer axis event"> + Inject a pointer axis event from the remote system. + </description> + <arg name="axis" type="uint"/> + <arg name="value" type="fixed"/> + </request> + + <request name="send_key"> + <description summary="Send keyboard event"> + Inject a keyboard event from the remote system. + </description> + <arg name="key" type="uint"/> + <arg name="state" type="uint"/> + </request> + + </interface> + + <interface name="remote_access" version="1"> + <description summary="Interface for remote access systems"> + This global interface is intended to support remote access systems. + It allows screen outputs to be captured, and mouse and keyboard events + to be injected from the remote system. + </description> + + <request name="create_capture"> + <description summary="Create an output capture object"> + Creates an object that can be used to capture the specified output. + A buffer must be supplied, into which the image from the output is + copied. This must be in XRGB8888 format and large enough to contain + the image. + </description> + <arg name="capture" type="new_id" interface="remote_capture"/> + <arg name="output" type="object" interface="wl_output"/> + <arg name="buffer" type="object" interface="wl_buffer"/> + </request> + + <request name="create_seat"> + <description summary="Create a new remote seat"> + Creates an object that can be used to send input events into the system + from a new remote seat. + </description> + <arg name="seat" type="new_id" interface="remote_seat"/> + <arg name="name" type="string"/> + </request> + + <request name="create_existing_seat"> + <description summary="Create a remote seat for an existing local seat"> + Creates an object that can be used to send input events into the system + from an existing seat. + The name argument must correspond to the name of an existing wl_seat. + </description> + <arg name="seat" type="new_id" interface="remote_seat"/> + <arg name="name" type="string"/> + </request> + + </interface> + +</protocol> diff --git a/src/.gitignore b/src/.gitignore index 539150d..cae3316 100644 --- a/src/.gitignore +++ b/src/.gitignore @@ -21,4 +21,6 @@ input-method-protocol.c input-method-server-protocol.h subsurface-server-protocol.h subsurface-protocol.c +remote-access-protocol.c +remote-access-server-protocol.h diff --git a/src/Makefile.am b/src/Makefile.am index b0eae7c..def50b9 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -97,6 +97,7 @@ module_LTLIBRARIES = \ $(cms_static) \ $(cms_colord) \ $(gl_renderer) \ + $(remote_access) \ $(x11_backend) \ $(drm_backend) \ $(wayland_backend) \ @@ -302,6 +303,17 @@ cms_colord_la_SOURCES = \ endif endif +if ENABLE_REMOTE_ACCESS +remote_access = <a href="http://remote-access.la" target="_blank">remote-access.la</a> +remote_access_la_LDFLAGS = -module -avoid-version +remote_access_la_LIBADD = $(COMPOSITOR_LIBS) $(REMOTE_ACCESS_LIBS) +remote_access_la_CFLAGS = $(GCC_CFLAGS) $(COMPOSITOR_CFLAGS) $(REMOTE_ACCESS_CFLAGS) +remote_access_la_SOURCES = \ + remote-access.c \ + remote-access-protocol.c \ + remote-access-server-protocol.h +endif + noinst_PROGRAMS = spring-tool spring_tool_CFLAGS = $(GCC_CFLAGS) $(COMPOSITOR_CFLAGS) @@ -330,6 +342,8 @@ BUILT_SOURCES = \ workspaces-protocol.c \ subsurface-server-protocol.h \ subsurface-protocol.c \ + remote-access-protocol.c \ + remote-access-server-protocol.h \ git-version.h CLEANFILES = $(BUILT_SOURCES) diff --git a/src/remote-access.c b/src/remote-access.c new file mode 100644 index 0000000..68087cc --- /dev/null +++ b/src/remote-access.c @@ -0,0 +1,612 @@ +/* + * Copyright © 2013 RealVNC Limited + * + * Permission to use, copy, modify, distribute, and sell this software and + * its documentation for any purpose is hereby granted without fee, provided + * that the above copyright notice appear in all copies and that both that + * copyright notice and this permission notice appear in supporting + * documentation, and that the name of the copyright holders not be used in + * advertising or publicity pertaining to distribution of the software + * without specific, written prior permission. The copyright holders make + * no representations about the suitability of this software for any + * purpose. It is provided "as is" without express or implied warranty. + * + * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS + * SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND + * FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY + * SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER + * RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF + * CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN + * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include "config.h" + +#include <unistd.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stdbool.h> +#include <linux/input.h> + +#include "compositor.h" +#include "remote-access-server-protocol.h" + + +struct remote_capture { + struct wl_listener listener; + struct weston_buffer *buffer; + struct wl_resource *resource; + struct weston_output *output; + uint32_t flags; +}; + +enum remote_capture_flags { + CAPTURE_REFRESHING = (1 << 0), + CAPTURE_CONTINUOUS = (1 << 1) +}; + +struct remote_seat { + struct weston_seat *seat; + struct wl_resource *resource; + uint32_t flags; +}; + +enum remote_seat_flags { + SEAT_EXISTING = (1 << 0) +}; + +struct remote_access { + struct weston_compositor *ec; + struct wl_global *global; + struct wl_listener destroy_listener; +}; + + +/* Utils */ + +static void +copy_bgra_yflip(uint8_t *dst, uint8_t *src, int height, int stride) +{ + uint8_t *end; + + end = dst + height * stride; + while (dst < end) { + memcpy(dst, src, stride); + dst += stride; + src -= stride; + } +} + +static void +copy_bgra(uint8_t *dst, uint8_t *src, int height, int stride) +{ + /* TODO: optimize this out */ + memcpy(dst, src, height * stride); +} + +static void +copy_row_swap_RB(void *vdst, void *vsrc, int bytes) +{ + uint32_t *dst = vdst; + uint32_t *src = vsrc; + uint32_t *end = dst + bytes / 4; + + while (dst < end) { + uint32_t v = *src++; + /* A R G B */ + uint32_t tmp = v & 0xff00ff00; + tmp |= (v >> 16) & 0x000000ff; + tmp |= (v << 16) & 0x00ff0000; + *dst++ = tmp; + } +} + +static void +copy_rgba_yflip(uint8_t *dst, uint8_t *src, int height, int stride) +{ + uint8_t *end; + + end = dst + height * stride; + while (dst < end) { + copy_row_swap_RB(dst, src, stride); + dst += stride; + src -= stride; + } +} + +static void +copy_rgba(uint8_t *dst, uint8_t *src, int height, int stride) +{ + uint8_t *end; + + end = dst + height * stride; + while (dst < end) { + copy_row_swap_RB(dst, src, stride); + dst += stride; + src += stride; + } +} + + +/* --- remote-capture --- */ + +static void +remote_capture_frame_notify(struct wl_listener *listener, void *data); + +static void +remote_capture_setup_listener(struct remote_capture *capture) +{ + if (capture->listener.notify == NULL && + capture->flags != 0) { + /* Listener is required but not currently set */ + capture->listener.notify = remote_capture_frame_notify; + wl_signal_add(&capture->output->frame_signal, + &capture->listener); + } else if (capture->listener.notify && + capture->flags == 0) { + /* Listener isn't required but is currently set */ + wl_list_remove(&capture->listener.link); + capture->listener.notify = NULL; + } +} + +static void +remote_capture_frame_notify(struct wl_listener *listener, void *data) +{ + struct remote_capture *capture = + container_of(listener, struct remote_capture, listener); + struct weston_output *output = data; + struct weston_compositor *compositor = output->compositor; + + weston_log("remote-capture: frame_notify\n"); + + pixman_box32_t *r; + pixman_box32_t tr; + pixman_region32_t damage; + int i, n, stride; + uint8_t *pixels, *d, *s; + + stride = capture->buffer->width * + (PIXMAN_FORMAT_BPP(compositor->read_format) / 8); + pixels = malloc(stride * capture->buffer->height); + + compositor->renderer->read_pixels(output, compositor->read_format, + pixels, + 0, 0, + output->current_mode->width, + output->current_mode->height); + + d = wl_shm_buffer_get_data(capture->buffer->shm_buffer); + s = pixels + stride * (capture->buffer->height - 1); + + switch (compositor->read_format) { + case PIXMAN_a8r8g8b8: + case PIXMAN_x8r8g8b8: + if (compositor->capabilities & WESTON_CAP_CAPTURE_YFLIP) + copy_bgra_yflip(d, s, output->current_mode->height, + stride); + else + copy_bgra(d, pixels, output->current_mode->height, + stride); + break; + case PIXMAN_x8b8g8r8: + case PIXMAN_a8b8g8r8: + if (compositor->capabilities & WESTON_CAP_CAPTURE_YFLIP) + copy_rgba_yflip(d, s, output->current_mode->height, + stride); + else + copy_rgba(d, pixels, output->current_mode->height, + stride); + break; + default: + break; + } + free(pixels); + + pixman_region32_init(&damage); + pixman_region32_intersect(&damage, &output->region, + &output->previous_damage); + + r = pixman_region32_rectangles(&damage, &n); + if (n >= 0) { + for (i = 0; i < n; i++) { + tr = weston_transformed_rect(output->width, + output->height, + output->transform, + output->current_scale, + r[i]); + remote_capture_send_update(capture->resource, + tr.x1, + tr.y1, + tr.x2-tr.x1, + tr.y2-tr.y1); + } + } + pixman_region32_fini(&damage); + + capture->flags &= ~CAPTURE_REFRESHING; + remote_capture_setup_listener(capture); +} + +static void +remote_capture_destroy_handler(struct wl_resource *resource) +{ + struct remote_capture *capture = + wl_resource_get_user_data(resource); + capture->flags = 0; + remote_capture_setup_listener(capture); + free(capture); +} + +static void +remote_capture_destroy(struct wl_client *client, + struct wl_resource *resource) +{ + wl_resource_destroy(resource); +} + +static void +remote_capture_refresh(struct wl_client *client, + struct wl_resource *resource) +{ + struct remote_capture *capture = + wl_resource_get_user_data(resource); + struct weston_output *output = capture->output; + + capture->flags |= CAPTURE_REFRESHING; + remote_capture_setup_listener(capture); + weston_output_damage(output); +} + +static void +remote_capture_start(struct wl_client *client, + struct wl_resource *resource) +{ + struct remote_capture *capture = + wl_resource_get_user_data(resource); + + capture->flags |= CAPTURE_CONTINUOUS; + remote_capture_setup_listener(capture); +} + +static void +remote_capture_stop(struct wl_client *client, + struct wl_resource *resource) +{ + struct remote_capture *capture = + wl_resource_get_user_data(resource); + + capture->flags &= ~CAPTURE_CONTINUOUS; + remote_capture_setup_listener(capture); +} + +struct remote_capture_interface remote_capture_implementation = { + remote_capture_destroy, + remote_capture_refresh, + remote_capture_start, + remote_capture_stop +}; + + +/* --- remote-seat --- */ + +static void +remote_seat_destroy_handler(struct wl_resource *resource) +{ + weston_log("remote-seat: destroy handler\n"); + struct remote_seat *rseat = + wl_resource_get_user_data(resource); + if (!(rseat->flags & SEAT_EXISTING)) { + /* Release the seat if we created it */ + weston_seat_release(rseat->seat); + free(rseat->seat); + } + free(rseat); +} + +static void +remote_seat_destroy(struct wl_client *client, + struct wl_resource *resource) +{ + weston_log("remote-seat: destroy\n"); + wl_resource_destroy(resource); +} + +static void +remote_seat_move_pointer(struct wl_client *client, + struct wl_resource *resource, + wl_fixed_t x, wl_fixed_t y, + uint32_t relative) +{ + struct remote_seat *rseat = wl_resource_get_user_data(resource); + uint32_t time = weston_compositor_get_time(); + + if (relative) { + notify_motion(rseat->seat, time, x, y); + } else { + notify_motion_absolute(rseat->seat, time, x, y); + } +} + +static void +remote_seat_pointer_button(struct wl_client *client, + struct wl_resource *resource, + uint32_t button, uint32_t state) +{ + struct remote_seat *rseat = wl_resource_get_user_data(resource); + uint32_t time = weston_compositor_get_time(); + + weston_log("remote-seat[%p]: pointer_button( button=%u, state=%u )\n", + rseat, button, state); + + notify_button(rseat->seat, time, button, + state ? WL_POINTER_BUTTON_STATE_PRESSED : + WL_POINTER_BUTTON_STATE_RELEASED); +} + +static void +remote_seat_pointer_axis(struct wl_client *client, + struct wl_resource *resource, + uint32_t axis, wl_fixed_t value) +{ + struct remote_seat *rseat = wl_resource_get_user_data(resource); + uint32_t time = weston_compositor_get_time(); + + weston_log("remote-seat[%p]: pointer_axis( axis=%u, value=%g )\n", + rseat, axis, wl_fixed_to_double(value)); + + notify_axis(rseat->seat, time, axis, value); +} + +static void +remote_seat_send_key(struct wl_client *client, + struct wl_resource *resource, + uint32_t key, uint32_t state) +{ + struct remote_seat *rseat = wl_resource_get_user_data(resource); + + weston_log("remote-seat[%p]: send_key( key=%u, state=%u )\n", + rseat, key, state); + + notify_key(rseat->seat, weston_compositor_get_time(), key, + state ? WL_KEYBOARD_KEY_STATE_PRESSED : + WL_KEYBOARD_KEY_STATE_RELEASED, + STATE_UPDATE_AUTOMATIC); +} + +struct remote_seat_interface remote_seat_implementation = { + remote_seat_destroy, + remote_seat_move_pointer, + remote_seat_pointer_button, + remote_seat_pointer_axis, + remote_seat_send_key +}; + + +/* --- remote-access --- */ + +static void +remote_buffer_destroy_handler(struct wl_listener *listener, void* data) +{ + struct weston_buffer *buffer = + container_of(listener, struct weston_buffer, destroy_listener); + + wl_signal_emit(&buffer->destroy_signal, buffer); + free(buffer); +} + +static struct weston_buffer* +remote_buffer_from_resource(struct wl_resource *resource) +{ + struct weston_buffer *buffer; + struct wl_listener *listener; + + listener = wl_resource_get_destroy_listener(resource, + remote_buffer_destroy_handler); + + if (listener) + return container_of(listener, struct weston_buffer, + destroy_listener); + + buffer = zalloc(sizeof *buffer); + if (buffer == NULL) + return NULL; + + buffer->resource = resource; + wl_signal_init(&buffer->destroy_signal); + buffer->destroy_listener.notify = remote_buffer_destroy_handler; + wl_resource_add_destroy_listener(resource, + &buffer->destroy_listener); + return buffer; +} + +static void +remote_access_create_capture(struct wl_client *client, + struct wl_resource *resource, + uint32_t id, + struct wl_resource *output_resource, + struct wl_resource *buffer_resource) +{ + struct weston_output *output = + wl_resource_get_user_data(output_resource); + struct weston_buffer *buffer = + remote_buffer_from_resource(buffer_resource); + struct remote_capture* capture = zalloc(sizeof *capture); + + weston_log("remote-access: create_capture\n"); + + if (capture == NULL) { + wl_resource_post_no_memory(resource); + return; + } + + capture->resource = wl_resource_create(client, + &remote_capture_interface, + 1, id); + wl_resource_set_implementation(capture->resource, + &remote_capture_implementation, + capture, + remote_capture_destroy_handler); + + buffer->shm_buffer = wl_shm_buffer_get(buffer->resource); + if (!buffer->shm_buffer) + return; + + buffer->width = wl_shm_buffer_get_width(buffer->shm_buffer); + buffer->height = wl_shm_buffer_get_height(buffer->shm_buffer); + if (buffer->width < output->current_mode->width || + buffer->height < output->current_mode->height) + return; + + capture->buffer = buffer; + capture->output = output; +} + +static void +remote_access_create_seat(struct wl_client *client, + struct wl_resource *resource, + uint32_t id, + const char* name) +{ + struct remote_access *remote = + wl_resource_get_user_data(resource); + + struct remote_seat *rseat = zalloc(sizeof *rseat); + if (rseat == NULL) { + wl_resource_post_no_memory(resource); + return; + } + + rseat->seat = zalloc(sizeof *rseat->seat); + if (rseat->seat == NULL) { + wl_resource_post_no_memory(resource); + return; + } + + rseat->resource = wl_resource_create(client, + &remote_seat_interface, + 1, id); + wl_resource_set_implementation(rseat->resource, + &remote_seat_implementation, + rseat, + remote_seat_destroy_handler); + + weston_seat_init(rseat->seat, remote->ec, name); + weston_seat_init_pointer(rseat->seat); + weston_seat_init_keyboard(rseat->seat, NULL); +} + +static void +remote_access_create_existing_seat(struct wl_client *client, + struct wl_resource *resource, + uint32_t id, + const char* name) +{ + struct remote_access *remote = + wl_resource_get_user_data(resource); + struct weston_seat* seat; + struct remote_seat *rseat = zalloc(sizeof *rseat); + if (rseat == NULL) { + wl_resource_post_no_memory(resource); + return; + } + + wl_list_for_each(seat, &remote->ec->seat_list, link) { + if (strcmp(seat->seat_name, name) == 0) { + rseat->seat = seat; + break; + } + } + + if (!rseat->seat) + return; + + rseat->resource = wl_resource_create(client, + &remote_seat_interface, + 1, id); + wl_resource_set_implementation(rseat->resource, + &remote_seat_implementation, + rseat, + remote_seat_destroy_handler); + + rseat->flags |= SEAT_EXISTING; +} + +struct remote_access_interface remote_access_implementation = { + remote_access_create_capture, + remote_access_create_seat, + remote_access_create_existing_seat +}; + +static void +unbind_remote_access(struct wl_resource *resource) +{ + weston_log("remote-access: unbind\n"); +} + +static int +remote_client_check_permission(struct wl_client *client) +{ + pid_t pid; uid_t uid; gid_t gid; + wl_client_get_credentials(client, &pid, &uid, &gid); + /* TODO: Check if the client is allowed, somehow. + Always allow for now */ + return 1; +} + +static void +bind_remote_access(struct wl_client *client, + void *data, uint32_t version, uint32_t id) +{ + struct remote_access *remote = data; + struct wl_resource *resource = + wl_resource_create(client, &remote_access_interface, 1, id); + + if (!remote_client_check_permission(client)) { + wl_resource_post_error(resource, + WL_DISPLAY_ERROR_INVALID_OBJECT, + "remote-access: permission denied"); + wl_resource_destroy(resource); + return; + } + + weston_log("remote-access: bind\n"); + + wl_resource_set_implementation(resource, &remote_access_implementation, + remote, unbind_remote_access); +} + +static void +remote_module_destroy(struct wl_listener *listener, void *data) +{ + struct remote_access *remote = + container_of(listener, struct remote_access, destroy_listener); + weston_log("remote-access: destroy module\n"); + wl_global_destroy(remote->global); + free(remote); +} + +WL_EXPORT int +module_init(struct weston_compositor *ec, + int *argc, char *argv[]) +{ + struct remote_access *remote; + weston_log("remote-access: init module\n"); + + /* create local state object */ + remote = zalloc(sizeof *remote); + if (remote == NULL) + return -1; + remote->ec = ec; + + /* destroy listener */ + remote->destroy_listener.notify = remote_module_destroy; + wl_signal_add(&ec->destroy_signal, &remote->destroy_listener); + + /* Our global object */ + remote->global = wl_global_create(ec->wl_display, + &remote_access_interface, 1, + remote, bind_remote_access); + return 0; +} -- 1.7.10.4 _______________________________________________ wayland-devel mailing list <a href="mailto:wayland-devel@lists.freedesktop.org">wayland-devel@lists.freedesktop.org</a> <a href="http://lists.freedesktop.org/mailman/listinfo/wayland-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/wayland-devel</a> </blockquote></div> </div>