How to recover from serial device being force closed.

[Giacinto Cifelli]

Hello Jessy,

it would be great if you could verify the behavior on the main branch
(at least on the last stable 1.20.6), and give us some logs, so we can
have a better idea of the issue and how proceed, whether in a general
manner, or with some specific fixes.

Giacinto




On Mon, May 1, 2023 at 8:47 AM Jessy Exum <jessy.diamondman at gmail.com> wrote:
>
> Hello ModemManager devs,
>
> I have encountered an issue while running ModemManager (version 1.18.4) on a Cinterion PLAS8 modem. Sometimes, the serial device used for AT communication with the modem is force closed (due to G_IO_HUP) without any obvious reason.
>
> Despite the modem being unreachable over the dead connection, the modem remains listed in mmcli (without being marked as having an error), and any ModemManager command that would send an AT command to the modem fails with an error similar to:
>
> GDBus.Error:org.freedesktop.ModemManager1.Error.Core.Connected: Cannot run sequence: 'Could not open serial device ttyACM1: it has been forced close'
>
>
> When I first saw this issue, I thought it was the modem acting up (it would not be the first time), but I found that restarting ModemManager recovers the modem. This tells me that the modem was still functioning, and ModemManager was unable (or never tried) to re-open the channel. I have not had this error occur on a device with ModemManager debugging enabled, so I can not be sure if reconnection is attempted (I will try to find a way to manually trigger this issue so I can get better debugging).
>
> If I could reset the modem, that should cause it to be re-initiated in ModemManger and likely fix the issue. But as far as I know, there is no way to have ModemManager re-enumerate a modem without removing the device (physically, through a modem reset, etc). Unfortunately, I am also unable to recover the modem through ModemManager because a modem reset requires the ability to send AT commands to the modem. Currently, the only solutions I have found to recover the modem is to physically power cycle the modem (which is not always possible), or to stop and restart ModemManager (inconvenient if there are other modems). I may also be able to use an external script to detect ModemManager getting into this state, and directly send a reset command to the modem over the AT channel (without ModemManager), but I would prefer if ModemManager was able to recover this on its own.
>
> The error occurs rarely/randomly enough that I am not interested in chasing down why this happens (it could be the modem being flakey, electrical interference, whatever). I am more interested in being able to gracefully recover from this situation, so I want to know what ModemManager should be doing so I can try to help fix it.
>
> Any suggestions or information would be greatly appreciated.
>
> Thanks for all the hard work in this useful tool,
> Jessy Diamond Exum
>