<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"> <style type="text/css" style="display:none;"></style> </head> <body dir="ltr"> <div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr"> <p style="margin-top:0;margin-bottom:0"></p> <div>Hi All,</div> <div><br> </div> <div>We are facing a problem that we believe to be caused by libdbus in a multithreaded environment. It's reproducibility is extremely low, we've only seen it twice so far. Can you please advise if this has been already addressed, or in some way a flaw in our usage of libdbus.</div> <div><br> </div> <div>Problem - </div> <div><br> </div> <div>The watchdog implementation in our system detected a deadlock in an application thread and killed the process. Upon examining the backtrace it was found that the process was stuck on a dbus_connection_send_with_reply_and_block call. </div> <div>What is perplexing is that the call is waiting for a reply that has already been processed by another thread in the process. </div> <div><br> </div> <div>From our logs -</div> <div><br> </div> <div>00:10:02.759: 3| 404| 1315|- |AbortDownload@kontactsynchronizer_p.h:366|Enter // The proxy API call </div> <div>00:10:02.759: 3| 404| 1315|- |addTimeout@kmainloop.cpp:234|CORE::Adding timer 361 with timeout of 23000 // libdbus asked us to add a timer</div> <div>00:10:02.768: 3| 404| 438|- |removeTimeout@kmainloop.cpp:302|CORE::Removing timer 361 // libdbus asked us to remove the added timer, indicating that the reply has been received in 9 ms.</div> <div><br> </div> <div>Our theory for the rootcause of the defect</div> <div><br> </div> <div>1. ThreadA calls dbus_connection_send_with_reply_and_block, which sends data out over the socket.</div> <div>2. ThreadA checks if the reply message has been received in _dbus_connection_block_pending_call. This method peeks the incoming message queue to see if the reply has already arrived. Answer is No, and hence it is decided that poll is needed.</div> <div>3. Context switch causes ThreadB to wake up, which happens to be the a threadpool thread monitoring the dbus socket for incoming data. This thread calls dbus_watch_handle and incoming message is moved to the pending queue.</div> <div>4. ThreadA now resumes, and it starts polling the socket for the pending reply, as it is unaware of the reply message already handled by ThreadB. </div> <div>5. Since the poll timeout is 25000 milliseconds, this call will now be blocked until some data arrives for this process. This means that in the case of complete radio silence, this call will take full 25 seconds to return even though the reply has already arrived in the process.</div> <div><br> </div> <div>If the poll begins before the message is received, or if the poll is woken up by some other incoming message, then libdbus behaves correctly. </div> <div><br> </div> <div>I see that something similar was being discussed here - https://sourceforge.net/p/windbus/mailman/windbus-svn/?viewmonth=200705. Apparently John Palmieri submitted a patch to address it, but the link is just a changelog and I cannot seem to track down the actual change.</div> <div>Excerpt from John's update on the link -</div> <div>"* dbus/dbus-connection.c: </div> <div>(_dbus_connection_block_pending_call):</div> <div>Check to see if our data has already been read off the connection</div> <div>by another blocking pending call before we block in poll.</div> <div>(check_for_reply_and_update_dispatch_unlocked):</div> <div>Code taken from _dbus_connection_block_pending_call - checks for</div> <div>an already read reply and updates the dispatch if there is one."</div> <div><br> </div> <div>Backtrace - </div> <div><br> </div> <div>#0 0xb6857490 in poll () at ../sysdeps/unix/syscall-template.S:81</div> <div>No locals.</div> <div>#1 0xb65bf044 in _dbus_poll (fds=<optimized out>, n_fds=<optimized out>, timeout_milliseconds=<optimized out>)</div> <div> at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-sysdeps-unix.c:2761</div> <div>No locals.</div> <div>#2 0xb65b88a8 in socket_do_iteration (transport=0x2483ff0, flags=6, timeout_milliseconds=<optimized out>)</div> <div> at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-transport-socket.c:1167</div> <div> saved_errno = <optimized out></div> <div> socket_transport = 0x2483ff0</div> <div> poll_fd = {fd = 5, events = 1, revents = 0}</div> <div> poll_res = <optimized out></div> <div> poll_timeout = <optimized out></div> <div>#3 0xb65b77f0 in _dbus_transport_do_iteration (transport=0x2483ff0, flags=1, timeout_milliseconds=24993)</div> <div> at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-transport.c:1001</div> <div>No locals.</div> <div>#4 0xb659e990 in _dbus_connection_do_iteration_unlocked (connection=0x2484488, pending=0x0, flags=6, timeout_milliseconds=24993)</div> <div> at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-connection.c:1227</div> <div>No locals.</div> <div>#5 0xb659f4dc in _dbus_connection_block_pending_call (pending=0x26b5638) at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-connection.c:2527</div> <div> start_tv_sec = 602</div> <div> start_tv_usec = 761037</div> <div> tv_sec = 602</div> <div> tv_usec = 768470</div> <div> status = DBUS_DISPATCH_DATA_REMAINS</div> <div> connection = 0x2484488</div> <div> client_serial = 3059348700</div> <div> timeout = 0x26b3768</div> <div> timeout_milliseconds = 25000</div> <div> elapsed_milliseconds = 7</div> <div>#6 0xb659fa00 in dbus_connection_send_with_reply_and_block (connection=<optimized out>, message=0xb41054f0, timeout_milliseconds=-1, error=0xa9e059e8)</div> <div> at /usr/src/debug/dbus/1.10.8-r0/git/dbus/dbus-connection.c:3575</div> <div> reply = <optimized out></div> <div> pending = 0x26b5638</div> <div> __FUNCTION__ = "dbus_connection_send_with_reply_and_block"</div> <div>#7 0xb6c56f78 in kivi::ipc::dbus::DBus::Send (this=0x2480e70, msg=..., result=...) at /usr/src/debug/kividbus/1.0-r0/git/dbusipc.cpp:446</div> <div><br> </div> Could you please advise how to address this in libdbus? <p></p> <p style="margin-top:0;margin-bottom:0"><br> </p> <div id="Signature"> <div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; background-color: rgb(255, 255, 255);"> <p class="x_MsoNormal" style="color:rgb(0,0,0); margin:0in 0in 0.0001pt; font-size:15.5555562973022px; font-family:'Times New Roman',serif; background-color:rgb(255,255,255)"> <span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">Thanks and Regards,</span></span><span style="font-size:11pt; font-family:Calibri,sans-serif; color:rgb(0,0,0); text-decoration:none"></span></p> <font face="Segoe UI, sans-serif" style="color:rgb(0,0,0); text-decoration:none"><span style="font-size:14.44444465637207px; color:rgb(0,0,0); text-decoration:none"><b style="color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">Manish Narang</span></b></span></font><br style="color:rgb(0,0,0); text-decoration:none"> <p class="x_MsoNormal" style="color:rgb(0,0,0); margin:0in 0in 0.0001pt; font-size:15.5555562973022px; font-family:'Times New Roman',serif; background-color:rgb(255,255,255)"> <span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">Design Lead – Infotainment</span></span><span style="font-size:11pt; font-family:Calibri,sans-serif; color:rgb(0,0,0); text-decoration:none"></span></p> <p class="x_MsoNormal" style="color:rgb(0,0,0); margin:0in 0in 0.0001pt; font-size:15.5555562973022px; font-family:'Times New Roman',serif; background-color:rgb(255,255,255)"> <span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">Mobile : +918983509995</span></span><strong style="color:rgb(0,0,0); text-decoration:none"><span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"> </span></strong><span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">| </span></span><span style="font-size:10pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none"></span><a href="mailto:manish.narang@kpit.com" tabindex="-1" disabled="true" id="LPNoLP"><span style="color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">manish.narang@k</span><span style="color:rgb(0,0,0); text-decoration:none"></span><span style="color:rgb(0,0,0); text-decoration:none">pit.com</span></span></a></span><span style="font-size:11pt; font-family:Calibri,sans-serif; color:rgb(31,73,125); font-size:11pt"></span></p> <p class="x_MsoNormal" style="margin:0in 0in 0.0001pt; font-size:15.5555562973022px; font-family:'Times New Roman',serif; background-color:rgb(255,255,255)"> <strong style="color:rgb(0,0,0); text-decoration:none"><span lang="EN-IN" style="font-size:11pt; font-family:'Segoe UI',sans-serif; color:rgb(0,0,0); text-decoration:none"><span style="color:rgb(0,0,0); text-decoration:none">————————————————————————————————</span></span></strong><span lang="EN-IN" style="color:rgb(127,127,127); font-size:10pt; font-family:'Segoe UI',sans-serif"><br> </span><strong style="color:rgb(33,33,33)"><span lang="EN-IN" style="font-size:11pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,255)">KPIT</span></strong><span lang="EN-IN" style="color:rgb(127,127,127); font-size:11pt; font-family:'Segoe UI',sans-serif">|</span><strong style="color:rgb(33,33,33)"><span lang="EN-IN" style="font-size:9pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,89)">Web:</span></strong><span lang="EN-IN" style="color:rgb(89,89,89); font-size:9pt; font-family:'Segoe UI',sans-serif"> </span><span lang="EN-IN" style="font-size:9pt; font-family:'Segoe UI',sans-serif"><font color="#5959ff"> <a href="http://www.kpit.com" id="LPNoLP">www.kpit.com</a></font></span><span lang="EN-IN" style="color:rgb(89,89,89); font-size:9pt; font-family:'Segoe UI',sans-serif"> | <strong>Social:</strong> </span><span lang="EN-IN" style="color:rgb(31,73,125); font-size:11pt; font-family:Calibri,sans-serif"><a href="http://www.kpit.com/facebook" target="_blank" tabindex="-1" disabled="true" id="LPNoLP"><span style="font-size:9pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,89)">Facebook</span></a></span><span lang="EN-IN" style="color:rgb(89,89,89); font-size:9pt; font-family:'Segoe UI',sans-serif"> | </span><span lang="EN-IN" style="color:rgb(31,73,125); font-size:11pt; font-family:Calibri,sans-serif"><a href="http://www.kpit.com/twitter" target="_blank" tabindex="-1" disabled="true" id="LPNoLP"><span style="font-size:9pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,89)">Twitter</span></a></span><span lang="EN-IN" style="color:rgb(89,89,89); font-size:9pt; font-family:'Segoe UI',sans-serif"> | </span><span lang="EN-IN" style="color:rgb(31,73,125); font-size:11pt; font-family:Calibri,sans-serif"><a href="http://www.kpit.com/linkedin" target="_blank" title="http://www.kpit.com/linkedin Ctrl+Click or tap to follow the link" tabindex="-1" disabled="true" id="LPNoLP"><span style="font-size:9pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,89)">Linkedin</span></a></span><span lang="EN-IN" style="color:rgb(89,89,89); font-size:9pt; font-family:'Segoe UI',sans-serif"> | </span><span lang="EN-IN" style="color:rgb(31,73,125); font-size:11pt; font-family:Calibri,sans-serif"><span style="font-size:9pt; font-family:'Segoe UI',sans-serif; color:rgb(89,89,89)"><a href="http://www.kpit.com/youtube" target="_blank" tabindex="-1" disabled="true" id="LPNoLP">Youtube</a></span></span></p> </div> </div> </div> This message contains information that may be privileged or confidential and is the property of the KPIT Technologies Ltd. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. KPIT Technologies Ltd. does not accept any liability for virus infected mails. </body> </html>