Signal handling in Java

[Stephan Bergmann]

On 06/22/2017 09:11 PM, Michael Stahl wrote:
>>> The signal-chaining facility also allows an application to link and
>>> load a shared library libjsig.so before libc/libthread/libpthread.
>>> This library ensures that calls such as signal(), sigset(),
>>> andsigaction() are intercepted so that they do not actually replace
>>> the Java HotSpot VM's signal handlers if the handlers conflict with
>>> those already installed by the Java HotSpot VM (B). Instead, these
> 
> so effectively the only way this can work if we somehow locate and
> LD_PRELOAD this libjsig.so somewhere before soffice.bin is invoked.
> 
> to me this does rather not sound like it would reduce the number of
> hoops to jump through or make anything more reliable.

Note that all this is only relevant for processes that ever call 
osl_addSignalHandler (where the first call to that causes the relevant 
sigaction calls to establish LO's OS-level signal handler function).

 From my 2017-06-19 comments in <https://gerrit.libreoffice.org/#/c/38916/>:

"We are calling osl_addSignalHandler -> initSignal -> onInitSignal from 
InitVCL (vcl/source/app/svmain.cxx, to add our general crash handler) 
and from Desktop::Init (desktop/source/app/app.cxx, to add a termination 
listener).  The latter is soffice-only and can be ignored.

"When InitVCL is called from soffice, it does so early, before 
potentially instantiating any JVM, so there are no problems with 
establishing our SEGV etc. handlers.  However, when InitVCL is called 
from a process where a JVM is already instantiated (either because it's 
a Java application, or because it's a native process that already 
instantiated a JVM), we must not install SEGV etc. handlers overriding 
the JVM ones.  The existing code takes care of that (assuming such a 
process's executable name doesn't contain "soffice").  However, a better 
approach overall would probably be to establish our crash handler not 
from InitVCL, but from soffice-specific code."

So, for soffice.bin we could potentially link statically against 
libjsig.so, but don't need to.  (Which is good, for cases where LO is 
run in an environment w/o Java, where we'd need to do LD_PRELOAD hacking 
in the sofice script; which wouldn't work on macOS; ...)

For processes other than soffice.bin (i.e., other executables that are 
part of the LO installation; and native 3rd party apps using UNO; and 
Java apps using UNO and thus potentially instantiating binary UNO within 
the JVM process), we go through some hoops in sal/osl/unx/signal.cxx to, 
upon calling osl_addSignalHandler, not establish OS-level signal 
handlers for signals that are relevant to the JVM (like SIGSEGV, to 
synthesise java.lang.NullPointerException).

This latter part is code that could arguably be cleaned up.  But IMO not 
by linking against libjsig.so (we'd e.g. need to tell 3rd party app 
providers to relink their apps), but rather by making sure that such 
processes don't call osl_addSignalHandler in the first place.

The likelihood that 3rd party code calls osl_addSingalHandler by 
themselves is probably rather small, and we can somewhat guard against 
that by deprecating that part of the URE interface (deprecating it for 
external code, LO-internally we'd still need it).  Then, the only 
remaining place through which such processes could call 
osl_addSingalHandler is by (indirectly) instantiating VCL, calling into 
InitVCL.  Hence, my above suggestion "to establish our crash handler not 
from InitVCL, but from soffice-specific code."  Once we've done that, we 
could clean up the code in sal/osl/unx/signal.cxx in a way as initially 
suggested in <https://gerrit.libreoffice.org/#/c/38916/3>, patch set 3 
of "sal: remove last of corner cased signal handler code".