[systemd-devel] Logs from a service is not showing up in journalctl but showing up in syslog

[Debraj Manna]

Thanks Silvio for replying. I will check your suggestions.

But it appears this is some issue with systemd version 229 as mentioned in
https://unix.stackexchange.com/a/417632

On Thu, Jul 25, 2019 at 4:09 PM Silvio Knizek <killermoehre at gmx.net> wrote:

> Am Donnerstag, den 25.07.2019, 15:55 +0530 schrieb Debraj Manna:
> > I have unit file which looks like below. I am seeing some of the echo
> > are
> > showing up in syslog but not in journalctl. Can someone let me know
> > what is
> > going on?
> > systemd version 229 running on Ubuntu 16.
> >
> > [Unit]
> > Description=Kafka Service
> >
> > [Service]
> > Type=simple
> > Environment=KAFKA_HOME=/home/ubuntu/deploy/kafka
> > Environment=LIB_DIR=/var/lib/kafka
> > Environment=LOG_DIR=/var/log/kafka
> > Environment=TEMP_DIR=/home/ubuntu/tmp
> > Environment=TOOLS_JAR=/home/ubuntu/build-target/common-utils/tools-
> > 0.001-SNAPSHOT.jar
> > Environment=MIN_DATA_PARTITION_FREE_SPACE_PCT=10
> > Environment=MIN_DATA_PARTITION_FREE_SPACE_GB=10
> > Environment=DATA_PARTITION_NAME=/var
> >
> > ExecStartPre=-/bin/mkdir -p /var/log/kafka
> > ExecStartPre=-/bin/chown -R ubuntu:ubuntu /var/log/kafka
> > ExecStartPre=-/bin/mkdir -p /var/lib/kafka/kafka-logs
> > ExecStartPre=-/bin/chown -R ubuntu:ubuntu /var/lib/kafka/kafka-logs
> > ExecStartPre=-/bin/rm -f /var/log/kafka/kafka-logs/.lock
> > ExecStartPre=-/bin/mkdir -p /home/ubuntu/tmp
> > ExecStartPre=-/bin/chown -R ubuntu:ubuntu /home/ubuntu/tmp
> > ExecStartPre=-/bin/chmod -R 775 /home/ubuntu/tmp
> > ExecStartPre=-/bin/su ubuntu -c
> > "/home/ubuntu/build-target/kafka/kafka-systemd-prestart.sh"
> > ExecStart=/bin/su ubuntu -c
> > "/home/ubuntu/build-target/kafka/kafka-systemd-health.sh"
> > ExecStopPost=-/bin/bash
> > /home/ubuntu/build-target/kafka/kafka-systemd-poststop.sh
> > RestartSec=2s
> > Restart=always
> > LimitNOFILE=65535
> > KillSignal=SIGTERM
> > SendSIGKILL=no
> > SuccessExitStatus=1 143
> >
> > [Install]
> > WantedBy=multi-user.target
> >
> > kafka-systemd-prestart.sh looks like below
> >
> > echo "[`date +"%h %d %Y %H:%M:%S-%3N"`] Starting kafka..."
> > timeout --signal=sigkill 600s java -cp "$TOOLS_JAR"
> > com.vnera.tools.kafka.KafkaIndexValidator "$LIB_DIR/kafka-logs"
> > "$KAFKA_HOME/config/server.properties" true
> > broker_id=`sudo grep -F "broker.id"
> > $KAFKA_HOME/config/server.properties | awk -F '=' '{print $2}'`
> > zookeeper_list=`sudo grep -F "zookeeper.connect="
> > $KAFKA_HOME/config/server.properties | awk -F '=' '{print $2}'`
> > echo "attempting removal of broker id $broker_id"
> >
> > used_pct=`df ${DATA_PARTITION_NAME} --output=pcent | grep -v Use |
> > grep -o '[^ ]*[^ %]'`
> > free_pct=$(expr 100 - $used_pct)
> > free_gb=`df -h ${DATA_PARTITION_NAME} --output=avail --block-size G |
> > grep -v Avail | grep -o '[^ ]*[^ G]'`
> > echo "[`date +"%h %d %Y %H:%M:%S-%3N"`] partition
> > ${DATA_PARTITION_NAME} free% $free_pct required%
> > ${MIN_DATA_PARTITION_FREE_SPACE_PCT} freegb ${free_gb} required
> > ${MIN_DATA_PARTITION_FREE_SPACE_GB}"
> >
> > # Some other code
> >
> > kafka-systemd-poststop.sh looks like below
> >
> > -------------------
> >
> > echo "[`date +"%h %d %Y %H:%M:%S-%3N"`] Exiting kafka..."
> > cmd="ps -ef | grep -v grep | grep kafkaServer"
> >
> > # Some other code
> >
> > echo "completed exiting kafka"
> >
> > Doing sudo journalctl -u kafka.service looks like below
> >
> > Jul 25 07:41:39 platform2 systemd[1]: Started Kafka Service.
> > Jul 25 07:41:39 platform2 su[39160]: Successful su for ubuntu by root
> > Jul 25 07:41:39 platform2 su[39160]: + ??? root:ubuntu
> > Jul 25 07:41:39 platform2 su[39160]: pam_unix(su:session): session
> > opened for user ubuntu by (uid=0)
> > Jul 25 07:41:40 platform2 bash[39192]: [Jul 25 2019 07:41:40-572]
> > Exiting kafka...
> >
> > I am not seeing some of the echo from kafka-systemd-prestart.sh in
> > journatl but I am seeing those logs in syslog
> >
> > Jul 25 10:17:03 platform2 su[38464]: WatchedEvent state:SyncConnected
> > type:None path:null
> > Jul 25 10:17:03 platform2 su[38464]: Node does not exist:
> > /brokers/ids/2
> > Jul 25 10:17:03 platform2 su[38464]: [Jul 25 2019 10:17:03-343]
> > partition /var free% 9 required% 10 freegb 134 required 10
> > Jul 25 10:17:03 platform2 su[38464]: [Jul 25 2019 10:17:03-344]
> > Sufficient disk space is not available, sleeping for 60 seconds
> > before
> > exiting...
>
> Hi,
>
> first of all, take a look at man:tmpfiles.d to replace the whole
> mkdir/chmod ExecStartPre= stuff.
> Second, don't use »su« in .service. It breaks stuff by creating a new
> cgroup hierarchy because it's run through pam. Use User= instead.
> With both this changes your shell scripts shouldn't be necessary at all
> and than everything should land in the journal.
> Please don't re-invent the stuff systemd is already providing.
>
> BR
> Silvio
>
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20190725/c54eb40a/attachment-0001.html>