OSX: DBus Session Lookup Always Fails when Run from Cron or a Parent Process - Direct Exec Always Works

Fri Dec 18 17:06:01 PST 2015

Success! Using launchd instead of cron is the workaround/solution.

I have never used launchctl, launchd or the required Apple xml/plist for scheduling jobs before, but it was pretty straightforward and is more flexible than cron.

As the DBus specs doc states, the mechanism is there within launchd to exchange the needed session environment variables.

As I was hoping, using launchd (and the same ENV mechanism) to schedule/run the parent python process DOES appear to pass on the needed environment.

I hope this helps you too.

The impact of this issue to my project was actually huge .. I will be going into this weekend and holiday quite happy about this result.

Cheers,
Jim

> On Dec 18, 2015, at 3:45 PM, Jim Mannix <j_mannix at apple.com> wrote:
> 
> It almost sounds like using launchd to trigger this part of my automation, thus replacing cron, might be a solution to look into.
> 
> The specification info even says as much with respect to launching dbus and refers directly to environment and sessions.
> I might give this a shot.
> My real solution is ideally to trigger a parent Python script ever minute and then sometimes launch the perl script which uses the perl module which uses Net::DBus for the required app.
> 
> I am going to try this with launchd.
> 
>> On Dec 18, 2015, at 3:39 PM, Jim Mannix <j_mannix at apple.com <mailto:j_mannix at apple.com>> wrote:
>> 
>> Found something in the specification doc which sheds a little light on the same area we are zeroing in on:
>> launchd
>> 
>> launchd is an open-source server management system that replaces init, inetd and cron on Apple Mac OS X versions 10.4 and above. It provides a common session bus address for each user and deprecates the X11-enabled D-Bus launcher on OSX.
>> 
>> launchd allocates a socket and provides it with the unix path through the DBUS_LAUNCHD_SESSION_BUS_SOCKET variable in launchd's environment. Every process spawned by launchd (or dbus-daemon, if it was started by launchd) can access it through its environment. Other processes can query for the launchd socket by executing: $ launchctl getenv DBUS_LAUNCHD_SESSION_BUS_SOCKET This is normally done by the D-Bus client library so doesn't have to be done manually.
>> 
>> —
>> 
>> Separately, I was digging through the Net::DBus source and found a debug switch which gives some additional output, but not really in the error scenario, only for my successfully runs.
>> 
>> Set this in your crontab or export it manually on the command line .. one can see that DBus does react to it in both scenarios (success and fail).
>> 
>> (* This in itself is also a separate new piece of information .. I know that DBus is seeing the environment variables in both success and fail (manual and cron) scenarios .. both trigger the debug level.)
>> 
>> Just set it to 1 so it is true and of course export it (crontab setting does not need export .. it is crontab format not shell):
>> 
>> NET_DBUS_DEBUG=1
>> 
>> Alas the info it exposes is mostly only high-level method names, addresses and a few args related to them, not much at all related to sessions or the issue in an obvious way.     
>> 
>> —
>> 
>> Based on finding the bit about launchd and launchctl and the session id env var I might look closer now at the daemon process and launchd and this environment mechanism, which sounds a little different than a regular ENV mechanism, on which I am no expert.
>> Just wanted to share these bits of info while I’ve got them all in front of me.
>> 
>> Jim
>> 
>> 
>>> On Dec 18, 2015, at 12:52 PM, Jim Mannix <j_mannix at apple.com <mailto:j_mannix at apple.com>> wrote:
>>> 
>>> Hi Rene,
>>> Thanks for your detailed response.
>>> I do think it could be just an environment variable issue but I would not be shocked if there was more to it than that.
>>> 
>>> 
>>>> On Dec 18, 2015, at 1:57 AM, René J.V. Bertin <rjvbertin at gmail.com <mailto:rjvbertin at gmail.com>> wrote:
>>>> 
>>>> On Thursday December 17 2015 18:07:57 Jim Mannix wrote:
>>>> 
>>>>> 
>>>>> I’ve tried a great many things to get a Qt/DBus application to work under automation, but it only works when run directly.manually by a human in a terminal .. and the application is intended for cron automation or automation by a parent process.
>>>> 
>>>> That is with Qt5 I presume? Was it written from scratch for OS X?
>>> I could not find a Qt version number, but it seems like the issue might just be in the Net::DBus->session() call and how that code is trying to determine a session identifier.
>>> I don’t have anything pointing me to the Qt part of it at this point.
>>> It might be an issue on OSX with some part of the Net::DBus module and probably specific to OSX.
>>> 
>>> I would imagine that folks would expect DBus to find the correct session even when run from cron or some upstream parent process.
>>> I do realize however that some technologies are sensitive to environment variables in this context, but I have tried everything I can possibly think of to solve a mere ENV variable issue.
>>> 
>>> Perhaps there is a very dynamic environement variable involved and it only exists briefly right at the problem point and that is why I have not seen it or been able to capture/pass it on or make sure it is inherited in some manner.
>>> Stabbing in the dark a little there .. but that’s where I am at.
>>> 
>>>> 
>>>> Not that I can help you, but it looks a lot like an issue I have ran into myself porting KF5 (KDE5) frameworks and applications:
>>>> 
>>>> KF5's secure wallet feature relies on a central daemon process (kwalletd5) which handles and serves all requests to access wallets; it uses gpgme when available and thus requires communication with an appropriate pinentry tool.
>>>> 
>>>> This works fine when kwalletd5 is started manually from a terminal. It does not however if the daemon is started automatically when an application tries to access a wallet, even if that parent process is started manually from a terminal. The same applies when starting kwalletd5 through a launchctl script or even through LaunchServices; building it as an app bundle or not doesn't make a difference, nor does the LSUIElement setting.
>>>> It's been a while since I last checked this particular issue, but the error message about loading the DBus session plist seems familiar in this context. Coupled with the OOM error it does look like it comes from a bit of code that doesn't really know what it's reporting on (or reports an inappropriate error code).
>>> 
>>> There is a daemon and launchctl involved in my scenario but I have not dived too deep into that part.
>>> 
>>> I did at one point get the same error I am reporting, but in this rare case it occurred when I was manually running things (the context in which they normally work fine.)
>>> But in this case I figured out that the daemon process was not running.
>>> I did an unload of the plist, confirming it was not running.
>>> Then I loaded again .. presumably no restarting the missing service.
>>> 
>>> The error went away; things ran fine again.
>>> This error is very generic in my opinion .. occurring both for a missing service as well as a scenario in which a session cannot be identified.
>>> (*Unless perhaps the actual scenario in both cases IS a missing service .. SOMETHING TO CONSIDER. i.e. The error message was the same in these two different scenarios because the underlying issue (similar to a missing/crashed service) is as good or better definition of the root cause as the inability to identify a session.) Gotta love it when you get vague/ambiguous error messages, but it happens.
>>> 
>>> I have read discussions touching on vagaries of how sessions are identified and while ENV vars have been mentioned, I have also see things mentioned referring to DBus looking at what user or what PID started the service in question .. as part of identifying the correct session.
>>> 
>>> I will probably be doing more experiments soon and will update with anything interesting.
>>> 
>>> Regards,
>>> Jim
>>> 
>>>> 
>>>> I'd be a bit surprised if the OS X implementation of gpgme uses DBus to communicate with the pinentry utility, but the communication between kwalletd5 and its clients is definitely done using DBus.
>>>> 
>>>> My first thought was that the DBus session address env.variable somehow gets lost or went stale; that appears not to be the case (it still shows up correctly in `launchctl export` and thus should be available to any process started no matter how).
>>>> 
>>>> Do DBus and/or Qt5 use any other env. variables that might not be available to applications that are not started manually from a terminal, on OS X?
>>>> 
>>>> FWIW ...
>>>> René
>>> 
>>> _______________________________________________
>>> dbus mailing list
>>> dbus at lists.freedesktop.org <mailto:dbus at lists.freedesktop.org>
>>> http://lists.freedesktop.org/mailman/listinfo/dbus <http://lists.freedesktop.org/mailman/listinfo/dbus>
>> 
>> _______________________________________________
>> dbus mailing list
>> dbus at lists.freedesktop.org <mailto:dbus at lists.freedesktop.org>
>> http://lists.freedesktop.org/mailman/listinfo/dbus
> 
> _______________________________________________
> dbus mailing list
> dbus at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dbus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/dbus/attachments/20151218/a0df8a7a/attachment-0001.html>