libwayland and Java

Mon Dec 17 10:17:48 PST 2012

I'll send out an e-mail later about API ideas once I have more time. I've
got an idea of what I'd like to see but I haven't had a chance to write it
down and formalize it yet.

Concerning use cases and why I'm doing it in Java: I foresee two major use
cases (that should possibly be separate apps). The first and primary case
is, as Pekka said, is for chrooted environments. As far as I'm aware, the
state-of-the-art for those environments is either VNC or X forwarding (not
sure how well the latter works). A wayland implementation would provide a
native-speed fully accelerated link to the android GUI. In this use case it
would be more like the proposed system compositor of session compositor in
that another compositor (GNOME, KDE, etc.) would run on top of it and
provide a full linux system turning devices like the ASUS transformer into
a full Linux machine.

The second use case is to be able to run single statically linked apps via
wayland rather than having to implement the GUI directly with Android.
While this is a bad long-term solution, It could prove useful for getting
ports of applications up and running or for testing purposes. There's also
the possibility to provide a WaylandView widget that allows you to embed a
wayland client into an Android app. There's a lot of possibilities here.

When it comes to what language things are written in, I basically have 3
options: use Android's NativeActivity class and do it all in C/C++, write
it all in Java use JNI to bind to libwayland, or write most of it in C/C++
and write my own Java/JNI bindings to the Android system internals. The
obvious solution seems to be Android's NativeActivity class. It already
provides all of the bindings to the Android GUI so that you can write
everything in C/C++ and it's fairly easy to do. The problem is that the
Android system reserves the right to kill your process at any time and the
app is expected to be able to recover seamlessly. In the case of
NativeActivity, this is implemented by allowing you to save a binary buffer
before the program is killed and then load your data back out of the buffer
and restart the GUI. If the entire wayland server is running in this
environment then clients are liable to get disconnected for no good reason
and this would be very bad.

Android's solution to this problem is something called Services. To make a
long story short, with a service you can have a part of your app that stays
running and will (almost) never get killed. I say almost because if the
phone is running extremely low on memory it may kill services to keep the
active app and more core parts of the OS alive. However, if a service is a
foreground service, it will almost never get killed unless something
strange is happening on the device. Having a service allows me to keep the
compositor alive and connected to the clients while still
creating/destroying the GUI as per Android's wishes. The problem with
services is that there is no native service glue. I could write my own but
that gets complicated. Specifically, this keeps me from using
NativeActivity as it provides no good way to break out of the
Android-provided main loop and no good way to make JNI calls.
Theoretically, I could do this with JNI but that is more trouble than it's
worth.

In order to use a Service, I basically have to write my GUI in Java.
However, this still leaves the possibility of doing most of it in C/C++ and
using JNI to bind to the Android stuff. I have a proof-of-concept that I
implemented in mostly C++ (you can see it here: http://youtu.be/8HxEk71vhY8).
However, tying into the Android system (particularly as it concerns touch
events), while not terribly complicated by itself becomes complicated when
you start doing it through JNI. Also, I have to do some threading magic to
keep the Android system, EGL, and my compositor service all happy. While
this isn't too bad up front, it gets complicated when things are happening
on both sides of the JNI line. For this reason, I think it's simpler to
write the entire compositor in Java. I realize that this is going to make
my job a little bit harder from a maintenance perspective; however, since
the primary use case is to run another compositor on top of mine, mine
doesn't have to be as complicated so I think this is acceptable. Also, the
Android system is fairly similar to wayland under the hood so there are a
lot of concepts (drag and drop for example) that I may be able to translate
fairly directly. These translations are going to be much easier if I can do
it in Java.

I'm sorry if that got a bit long-winded but I hope you all can understand
what I'm trying to do a bit better. I appreciate any input you can offer.
--Jason Ekstrand

On Mon, Dec 17, 2012 at 4:15 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:

> On Sun, 16 Dec 2012 20:11:51 -0600
> Jason Ekstrand <jason at jlekstrand.net> wrote:
>
> > Hello All,
> > My name is Jason Ekstrand and I'm currently working on writing a Wayland
> > server for Android. This is not to be confused with what Pekka Paalanen
> is
> > working on; rather, my objective is to make a Wayland server that runs as
> > an actual Android app. I have a basic proof-of-concept working without
> any
> > input capabilities.
>
> Hi Jason,
>
> nice! What is your use case? Allowing a chroot on an Android device to
> use the display and input?
>
> > In order to tie into the Android services and events systems, a lot of
> > things have to be implemented in Java. It will be simpler if I can write
> > the entire server in Java. To that end, I'm writing Java bindings to
> > libwayland and have come across a few snags. Specifically, I really wish
> I
> > could do my own marshaling. Right now, I'm having to generate native
> > methods for every even just so that I can call the variadic
> > wl_resource_post_event. It would be much easier if there were a version
> > that could take an array of arguments so I could do everything at
> runtime.
> > I also have a function that translates c arguments to java for requests.
> > Again, I have to generate a native function for every request so that I
> can
> > pass in valid function pointers. It would be great if libwayland could
> call
> > my marshaling function directly and let me handle calling the request
> > callback. Would it be practical to build this kind of support into
> > libwayland?
>
> Wayland bindings for different languages have come up several times in
> the past, and we don't have really good answers, I think. Could you
> write a sketch (as in code) on what kind of alternate API you would like
> to have in libwayland, and how does it interact with the current API?
>
> Even though you might not have the requirement on Android as an app, we
> also need to support sharing libwayland-{client,server}.so between the
> app and any libraries. It will break, if there are several instances or
> implementations of the .so in one process, because we e.g. compare
> pointers to static data.
>
> I cannot decide whether any new libwayland API is acceptable or not, but
> it would be interesting to see, and maybe we could get comments from
> other languages' developers, too.
>
> Kristian said he is on vacation for two weeks now, and I will be on
> vacation after this week, too. I don't know about others, but it might
> be fairly quiet for the rest of the year.
>
>
> Thanks,
> pq
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20121217/4ce63faa/attachment-0001.html>