[systemd-devel] Journal API demo application: "tallow" - a fail2ban replacement

[David Strauss]

Units "subscribed" to specific message IDs has two negative effects:
(1) massive proliferation of units, since each one can supposedly only
handle one UUID and (2) unit names that are useless from a human
perspective.

It would be much cleaner to offer "message activation" though a unit
with an arbitrary name and a journalctl-style filter for incoming
messages.

I'd also consider a socket-based approach for event "subscriptions" as
long as it's efficient to stat() the non-existence of the appropriate
Message ID socket, since the common case would be no listener. If a
socket unit is the only supported way to listen, even the stat() would
be unnecessary, as systemd would know all active sockets to begin
with. There could be a standard naming convention for such sockets,
like /var/run/journal/msgid/<UUID>. This would allow message
"activation" of a listener and human-friendly unit names. It would
also allow listening to happen in both "accept" and non-"accept"
socket modes. Even a shell script could easily receive and handle
certain events.

On Thu, Nov 8, 2012 at 11:12 AM, Daniel J Walsh <dwalsh at redhat.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 11/08/2012 01:18 PM, Douglas, William wrote:
>> On Thu, Nov 8, 2012 at 8:54 AM, Kay Sievers <kay at vrfy.org> wrote:
>>> On Thu, Nov 8, 2012 at 8:31 AM, William Douglas
>>> <william.douglas at intel.com> wrote:
>>>> "Kok, Auke-jan H" <auke-jan.h.kok at intel.com> writes:
>>>>>
>>>>> I wrote a demo application that uses the journal API to scan for SSH
>>>>> bruteforce logs in the journal, called "tallow".
>>>>
>>>> Since Auke is on vacation now (and would *never* read email or work on
>>>> projects on vacation ;) ) and has finally put this could out there, I
>>>> want to ask about getting a general event response processor in the
>>>> journal/systemd.
>>>>
>>>> In order to avoid 30 daemons each doing an inotify on the journal for
>>>> their message of interest to pass by, I think having a simple
>>>> .log-event type unit file be used by the journal would be a nice to
>>>> have feature. An example of what I'm thinking of would be the following
>>>> to run a processing script for each core dump:
>>>>
>>>> [Event] MessageID=fc2e22bc6ee647b6b90729ab34a250b1 (or
>>>> SD_MESSAGE_COREDUMP)
>>>>
>>>> [Service] type=oneshot ExecStart=/usr/sbin/core-processor %data %pid
>>>>
>>>>
>>>> Where Event can contain any journal field, conditions are done as
>>>> normal ie ConditionFileExists and friends (including allowing !
>>>> prefixes). For the ExecStart line %data and %pid are examples of
>>>> passing journal fields though I'm definitely looking for opinions on
>>>> how this could best be done as it doesn't seem right to have them in
>>>> the [Service] group but having some kind of ExecArguments in [Event]
>>>> seems odd too.
>>>>
>>>> Anyway I'd like to hear people's opinions on why this feature is
>>>> unneeded and what other way I should be doing this or maybe even this
>>>> is a good idea and with some refinement could get merged in to
>>>> systemd.
>>>
>>> Right, we already planned to have "message activation" in systemd. It
>>> will probably its own unit type, called ".msgid", or something. The
>>> details are not entirely clear, it's just a bunch of ideas that moment.
>>>
>>
>> Awesome!
>>
>>> We need to sort out a few things, like what happens at startup, to
>>> prevent the "30 daemons" to see all the old messages again and again and
>>> getting activated for them. How to properly wake up an already running
>>> daemon when a new message arrives. How to handle re-activation for
>>> services which exit at idle.
>>>
>>
>> Yes start up is interesting. I imagine an admin would probably want to be
>> able to take action on messages that have happened before the journal and
>> message activation are online. Is the BOOT_ID field useful for determining
>> where to start processing these messages? Seems like it might be good to
>> take an optional AfterJournal=Bool so some units could by pass on those
>> earlier messages too.
>>
>> The already running daemon's I expect to have have a socket available for
>> getting these messages, though I guess a SIGUSR# could work if there isn't
>> need for much in the way of data transfer.
>>
>> For run once type processing, re-activation could work more like
>> daemon at .service files. Each message event would just do a run with their
>> cursor as the identifier. _______________________________________________
>> systemd-devel mailing list systemd-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>
> setroubleshoot would be interested in this also, since it would eliminate the
> need to run with the audit daemon.  But journal needs to be getting audit data.
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iEYEARECAAYFAlCcBCUACgkQrlYvE4MpobNa/gCdHkTSLU4AC/1PRce6zm/0ZwEB
> X8gAoLYRinLAPafNQc+cDHqxLz+Kk3pE
> =fDOK
> -----END PGP SIGNATURE-----
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/systemd-devel



-- 
David Strauss
   | david at davidstrauss.net
   | +1 512 577 5827 [mobile]