First wds/wda command fails in approx 70% of all boots on SIMCom 7600E-H

[Tor Rune Skoglund]

Hi Aleksander,

thanks for your help.

Den 18.07.2020 09:14, skrev Aleksander Morgado:
>> I have a very weird problem with the SIMCom 7600E-H. In approx. 70% of all bootups, some wda/wds commands that apparently talk with the card always fail the first time (and possibly subsequent times) they are run. E.g. if I do:
>>
>> platform ~ # ip link set dev wwp0s20u5u1i5 down
>> platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-get-data-format
>> error: couldn't create client for the 'wda' service: CID allocation failed in the CTL client: Transaction timed out
>> platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-get-data-format
>> error: couldn't create client for the 'wda' service: QMI protocol error (3): 'Internal'
>>
> The type of error you received is in the CTL service, when attempting
> to allocate a new client. I would assume that the error you're seeing
> is just because the modem needs time to fully boot, could that be it?
It doesn't seem to be related to the boot up time. I did several more 
tests for an hour waiting 5 minutes on each boot before doing anything, 
and the behaviour stays the same.
> Does the command also fail consistently if you e.g. wait some minutes
> (just to be sure) before sending those commands?
Yes, i.e. it fails consistently randomly in 70%-80% of all boots also 
when waiting.
> Is the wda-get-data-format the first command you ever send to the device?

Yes.  This system is quite "pure" with no GUI, no ModemManager nor 
NetworkManager installed, no other modem tools run at boot, and no dhcpc 
daemon.

> In ModemManager we do some initial "probing" of the port, e.g.
> attempting to list the supported QMI services (an action that also
> requires the CTL service) and only once that has been completed, we go
> on with other things like several client allocations.
>
> If you want to do this kind of probing yourself, you could also run
> --wda-noop --client-no-release-cid so that qmicli attempts to just
> allocate a WDA client and not release it on exit. You can run this
> command in a loop until it succeeds, and when it does, you can then
> run --wda-get-data-format --client-cid=X to use the successfully
> allocated client. Or, well, you could also run in a loop the
> --wda-get-data-format command until it succeeds.

Reg looping to wait for wda until it works: I cannot get to work simply 
because if the first wda command fails, the subsequent ones always fail 
too, forever: Like this:

platform ~ # uptime
  13:33:22 up 8 min,  1 user,  load average: 0.01, 0.11, 0.08
platform ~ # qmicli -p -d /dev/cdc-wdm0 --get-expected-data-format
802-3
platform ~ # qmicli -p -d /dev/cdc-wdm0 --set-expected-data-format=raw-ip
[/dev/cdc-wdm0] expected data format set to: raw-ip
platform ~ # while ! qmicli -p -d /dev/cdc-wdm0 --wda-get-data-format; 
do echo "wda-get-data-format failed"; sleep 5; done
error: couldn't create client for the 'wda' service: CID allocation 
failed in the CTL client: Transaction timed out
wda-get-data-format failed
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
wda-get-data-format failed
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
wda-get-data-format failed
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
wda-get-data-format failed
.... and so on.....

However, if a wds command is run first, wda does never fail, no matter 
if the first wds command failed or not, e.g. (rebooted system):

platform ~ # qmicli -p -d /dev/cdc-wdm0 --wds-noop
platform ~ # qmicli -p -d /dev/cdc-wdm0 --get-expected-data-format
802-3
platform ~ # qmicli -p -d /dev/cdc-wdm0 --set-expected-data-format=raw-ip
[/dev/cdc-wdm0] expected data format set to: raw-ip
platform ~ # while ! qmicli -p -d /dev/cdc-wdm0 --wda-get-data-format; 
do echo "wda-get-data-format failed"; sleep 5; done
[/dev/cdc-wdm0] Successfully got data format
                    QoS flow header: no
                Link layer protocol: 'raw-ip'
   Uplink data aggregation protocol: 'disabled'
Downlink data aggregation protocol: 'disabled'
                      NDP signature: '0'
Downlink data aggregation max datagrams: '0'
Downlink data aggregation max size: '0'
platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-set-data-format=raw-ip
[/dev/cdc-wdm0] Successfully set data format
                         QoS flow header: no
                     Link layer protocol: 'raw-ip'
        Uplink data aggregation protocol: 'disabled'
      Downlink data aggregation protocol: 'disabled'
                           NDP signature: '0'
Downlink data aggregation max datagrams: '0'
      Downlink data aggregation max size: '0'

In the first case above, one could then think that running a wds command 
*if* wda fails also could be an option, but that does not work. The wda 
"Internal" error does not recover:

platform ~ # qmicli -p -d /dev/cdc-wdm0 --wds-noop
platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-get-data-format
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'

The wda-noop does not recover either:

platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-noop --client-no-release-cid
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-noop --client-no-release-cid
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
....

And it behaves like wda-get-data-format if run as the first command 
(rebooted system):

platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-noop --client-no-release-cid
error: couldn't create client for the 'wda' service: CID allocation 
failed in the CTL client: Transaction timed out
platform ~ # qmicli -p -d /dev/cdc-wdm0 --wda-noop --client-no-release-cid
error: couldn't create client for the 'wda' service: QMI protocol error 
(3): 'Internal'
...

I don't know... Could be that the modems are partly faulty. But these 
are two randomly picked modems from a batch of 130. If so, the whole 
batch is possibly bad. Or it could be that it is a (still unknown) 
problem with SIMCom 7600E-H, possibly in combination with qmi? But then, 
others should have experienced similar issues too. Or it might still be 
me doing something wrong somewhere.

It seems like adding a wds command first could actually be the solution 
at this time then. Another option is to reboot the modem until it works 
with e.g. echo "AT+CRESET^M" > /dev/ttyUSB2.

BR,
Tor Rune Skoglund