Help in establishing multiple simultaneous APNs: mmcli create/connect bearer
Aleksander Morgado
aleksander at aleksander.es
Thu Mar 17 09:59:18 UTC 2022
Hey,
> I attempted to use mmcli 1.18.6 and nmcli 1.36.0 with the create and connect bearer features of mmcli to create multiple simultaneous connections to two different APNs with a USB cellular modem on an Ubuntu Jammy Jellyfish 22.04 system. The ModemManager.service file had the debug and test-multiplex-requested options activated on the ExecStart line.
>
> Short summary of results:
>
> A sequence of mmcli commands to create a bearer, connect the bearer, create a second bearer, then connect the second bearer did result in multiple simultaneous packet data network connections. Some subsequent configuration with ip and route commands were necessary to make it work, but the second connection failed after a few minutes with error org.freedesktop.ModemManager1.Error.MobileEquipment.Unknown.
>
Hum, strange about that 2nd connection dropping like that.
> More details starting from the beginning:
>
> After booting and logging in, I plugged in the modem into the system, it reported this configuration:
>
> user at ubuntu22a:~/Documents$ ifconfig
>
> ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>
> inet6 fe80::ad94:11a5:b4a6:eada prefixlen 64 scopeid 0x20<link>
>
> ether 00:0c:29:df:a4:a1 txqueuelen 1000 (Ethernet)
>
> RX packets 0 bytes 0 (0.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 321 bytes 51137 (51.1 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> device interrupt 19 base 0x2000
>
>
>
> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>
> inet 127.0.0.1 netmask 255.0.0.0
>
> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>
> loop txqueuelen 1000 (Local Loopback)
>
> RX packets 8850 bytes 637347 (637.3 KB)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 8850 bytes 637347 (637.3 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> user at ubuntu22a:~/Documents$ ip link list
>
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>
> 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
>
> link/ether 00:0c:29:df:a4:a1 brd ff:ff:ff:ff:ff:ff
>
> altname enp2s1
>
> 3: wwan0: <BROADCAST,MULTICAST,NOARP> mtu 1428 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>
> link/ether 6a:d4:98:05:df:3a brd ff:ff:ff:ff:ff:ff
>
>
>
> link/ether 6a:d4:98:05:df:3a brd ff:ff:ff:ff:ff:ff
>
>
>
> user at ubuntu22a:~/Documents$ route
>
> Kernel IP routing table
>
> Destination Gateway Genmask Flags Metric Ref Use Iface
>
> <empty>
>
>
>
> Running the command “sudo mmcli -L” indicated that my modem was number 0. I ran the following command:
>
> sudo mmcli -m 0 –create-bearer=’apn=apnname1,ip-type=ipv4,allowed-auth=pap,user=apnuser1,password=apnpass1’
>
> This was successful and created bearer number 0.
>
> I then ran
>
> sudo mmcli -m 0 -b 0 --connect
>
> This was also successful.
>
> After doing this, the system’s networking configuration looked as follows:
>
> user at ubuntu22a:~/Documents$ ifconfig
>
> <ens33 and lo omitted for brevity’s sake; they are identical to first listing>
>
> mbimmux0.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> inet6 2001:db53:18ae:0:98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x0<global>
>
> inet6 2001:db53:18ae:0:3911:3b6a:6a96:92a4 prefixlen 64 scopeid 0x0<global>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 24 bytes 4191 (4.1 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> wwan0: flags=67<UP,BROADCAST,RUNNING> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 30 bytes 4271 (4.2 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> user at ubuntu22a:~/Documents$ ip link list
>
> <1: lo and 2: ens33 omitted for brevity’s sake; they are identical to first listing>
>
> 3: wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
> 4: mbimmux0.1 at wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1428 qdisc noqueue state UP mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
>
>
> user at ubuntu22a:~/Documents$ route
>
> Kernel IP routing table
>
> Destination Gateway Genmask Flags Metric Ref Use Iface
>
> <empty>
>
All this seems fine to me. MM takes care of bringing up the bearer
connection, and also takes care of creating the new virtual network
interface that is bound to that connection. The IP configuration and
routes of the new virtual network interface should be manually setup
in the interface based on the details exposed in the corresponding
Bearer object, MM won't do that.
>
>
> I then ran
>
> sudo mmcli -m 0 –create-bearer=’apn=apnname2,ip-type=ipv4,allowed-auth=pap,user=apnuser2,password=apnpass2’
>
> This was successful and created bearer number 1.
>
> I then ran
>
> sudo mmcli -m 0 -b 1 --connect
>
> This was also successful.
>
> After doing this, the system’s networking configuration looked as follows:
>
> user at ubuntu22a:~/Documents$ ifconfig
>
> <ens33 and lo omitted for brevity’s sake; they are identical to first listing>
>
> mbimmux0.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> inet6 2001:db53:18ae:0:98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x0<global>
>
> inet6 2001:db53:18ae:0:3911:3b6a:6a96:92a4 prefixlen 64 scopeid 0x0<global>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 27 bytes 4512 (4.5 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> wwan0: flags=67<UP,BROADCAST,RUNNING> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 36 bytes 4718 (4.7 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> user at ubuntu22a:~/Documents$ ip link list
>
> <1: lo and 2: ens33 omitted for brevity’s sake; they are identical to first listing>
>
> 3: wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
> 4: mbimmux0.1 at wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1428 qdisc noqueue state UP mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
> 5: mbimmux0.2 at wwan0: <BROADCAST> mtu 1428 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
>
>
> user at ubuntu22a:~/Documents$ route
>
> Kernel IP routing table
>
> Destination Gateway Genmask Flags Metric Ref Use Iface
>
> <empty>
>
>
>
> I noticed that mbimmux0.2 did not appear in the “ifconfig” output and that “ip link list” did not report it as up. So I ran “sudo ip link set mbimmux0.2 up” and then the interface configuration looked like this:
>
I think that's also fine. Bringing the virtual network interface up is
something the upper layers would do when setting it up. MM takes care
of making sure the master interface is up, not the new virtual ones.
> user at ubuntu22a:~/Documents$ ifconfig
>
> <ens33 and lo omitted for brevity’s sake; they are identical to first listing>
>
> mbimmux0.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> inet6 2001:db53:18ae:0:98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x0<global>
>
> inet6 2001:db53:18ae:0:3911:3b6a:6a96:92a4 prefixlen 64 scopeid 0x0<global>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 29 bytes 4726 (4.7 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> mbimmux0.2: flags=67<UP,BROADCAST,RUNNING> mtu 1428
>
> inet6 2001:db53:18ae:0:19fd:361e:7fdb:ea15 prefixlen 64 scopeid 0x0<global>
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> inet6 2001:db53:18ae:0:98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x0<global>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 10 bytes 868 (868.0 B)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> wwan0: flags=67<UP,BROADCAST,RUNNING> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 2 bytes 176 (176.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 49 bytes 5688 (5.6 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> user at ubuntu22a:~/Documents$ ip link list
>
> <1: lo and 2: ens33 omitted for brevity’s sake; they are identical to first listing>
>
> 3: wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
> 4: mbimmux0.1 at wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1428 qdisc noqueue state UP mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
> 5: mbimmux0.2 at wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc noqueue state UP mode DEFAULT group default qlen 1000
>
> link/ether 9a:b0:d0:6e:dc:9f brd ff:ff:ff:ff:ff:ff
>
>
>
> user at ubuntu22a:~/Documents$ route
>
> Kernel IP routing table
>
> Destination Gateway Genmask Flags Metric Ref Use Iface
>
> <empty>
>
>
>
> The status of the modem and the two bearers was then as follows:
>
> user at ubuntu22a:~/Documents$ mmcli -m 0
>
> --------------------------------
>
> General | path: /org/freedesktop/ModemManager1/Modem/0
>
> | device id: a883afd9cd8161007d0e42fe1e7a52bd9073f70d
>
> --------------------------------
>
> Hardware | manufacturer: Undisclosed, Inc.
>
> | model: A101
>
> | firmware revision: A101_01.2000.00
>
> | h/w revision: A101_V4.5_MBIM_V4.5
>
> | supported: gsm-umts, lte
>
> | current: gsm-umts, lte
>
> | equipment id: 101000000000008
>
> --------------------------------
>
> System | device: /sys/devices/pci0000:00/0000:00:11.0/0000:02:03.0/usb1/1-1
>
> | drivers: cdc_acm, cdc_mbim
>
> | plugin: generic
>
> | primary port: cdc-wdm0
>
> | ports: cdc-wdm0 (mbim), ttyACM0 (at), ttyACM2 (at), wwan0 (net)
>
> --------------------------------
>
> Status | unlock retries: sim-pin2 (3)
>
> | state: connected
>
> | power state: on
>
> | access tech: gprs
>
> | signal quality: 80% (cached)
>
> --------------------------------
>
> Modes | supported: allowed: 2g, 3g, 4g; preferred: none
>
> | current: allowed: 2g, 3g, 4g; preferred: none
>
> --------------------------------
>
> IP | supported: ipv4, ipv6, ipv4v6
>
> --------------------------------
>
> 3GPP | imei: 101000000000008
>
> | enabled locks: fixed-dialing
>
> | operator id: MCCMNC
>
> | operator name: MCCMNC
>
> | registration: roaming
>
> --------------------------------
>
> 3GPP EPS | ue mode of operation: csps-2
>
> --------------------------------
>
> SIM | primary sim path: /org/freedesktop/ModemManager1/SIM/0
>
> --------------------------------
>
> Bearer | paths: /org/freedesktop/ModemManager1/Bearer/1
>
> | /org/freedesktop/ModemManager1/Bearer/0
>
> user at ubuntu22a:~/Documents$ mmcli -m 0 -b 0
>
> ------------------------------------
>
> General | path: /org/freedesktop/ModemManager1/Bearer/0
>
> | type: default
>
> ------------------------------------
>
> Status | connected: yes
>
> | suspended: no
>
> | multiplexed: yes
>
> | interface: mbimmux0.1
>
> | ip timeout: 20
>
> ------------------------------------
>
> Properties | apn: apnname1
>
> | roaming: allowed
>
> | ip type: ipv4
>
> | allowed-auth: pap
>
> | user: apnuser1
>
> | password: apnpass1
>
> ------------------------------------
>
> IPv4 configuration | method: static
>
> | address: 10.3.1.2
>
> | prefix: 24
>
> | gateway: 10.3.1.1
>
> | dns: 208.67.222.222, 208.67.220.220
>
> ------------------------------------
>
> Statistics | duration: 390
>
> | attempts: 1
>
> | total-duration: 390
>
> user at ubuntu22a:~/Documents$ mmcli -m 0 -b 1
>
> ------------------------------------
>
> General | path: /org/freedesktop/ModemManager1/Bearer/1
>
> | type: default
>
> ------------------------------------
>
> Status | connected: yes
>
> | suspended: no
>
> | multiplexed: yes
>
> | interface: mbimmux0.2
>
> | ip timeout: 20
>
> ------------------------------------
>
> Properties | apn: apnname2
>
> | roaming: allowed
>
> | ip type: ipv4
>
> | allowed-auth: pap
>
> | user: apnuser2
>
> | password: apnpass2
>
> ------------------------------------
>
> IPv4 configuration | method: static
>
> | address: 10.3.1.3
>
> | prefix: 24
>
> | gateway: 10.3.1.1
>
Is this right? Are you getting in the 2nd APN connection an IP address
in the same subnet as in the 1st APN connection? It wouldn't be
completely wrong, but it kind of defeats the purpose of the multi APN
connection really (unless the network itself has explicit routing
rules for those different IPs even if in the same subnet).
> | dns: 208.67.222.222, 208.67.220.220
>
> ------------------------------------
>
> Statistics | attempts: 2
>
> | total-duration: 284
>
>
>
> I noticed that I would need to add some route information in order to be able to direct traffic through the mmbimux0.1 and mbimmux0.2 interfaces. This is fine.
Yes, as said, MM doesn't do any IP/route config by itself.
>
> However, after approximately five minutes of using and examining these interfaces, I noticed the second bearer connection had failed with an unknown error and mbimmux0.2 had been removed from the list of interfaces:
>
> user at ubuntu22a:~/Documents$ mmcli -m 0 -b 1
>
> --------------------------------------
>
> General | path: /org/freedesktop/ModemManager1/Bearer/1
>
> | type: default
>
> --------------------------------------
>
> Status | connected: no
>
> | connection error name: org.freedesktop.ModemManager1.Error.MobileEquipment.Unknown
>
> | connection error message: Unknown error
>
> | suspended: no
>
> | multiplexed: no
>
> | ip timeout: 20
>
> --------------------------------------
>
> Properties | apn: apnname2
>
> | roaming: allowed
>
> | ip type: ipv4
>
> | allowed-auth: pap
>
> | user: apnuser2
>
> | password: apnpass2
>
> --------------------------------------
>
> Statistics | duration: 294
>
> | attempts: 2
>
> | total-duration: 578
>
No idea why that error happens. That's likely an error reported by the
modem itself and we're converting it to "Unknown" ourselves, but there
is probably more info about the specific error happening in the MM
debug log itself. You should look in the MM debug log and look for a
MBIM indication reporting that disconnection at that time, and see if
there's a better error description there. It could be that the modem
reports Unknown itself, I wouldn't be surprised to be honest.
> user at ubuntu22a:~/Documents$ ifconfig
>
> <ens33 and lo omitted for brevity’s sake; they are identical to first listing>
>
> mbimmux0.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> inet6 2001:db53:18ae:0:98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x0<global>
>
> inet6 2001:db53:18ae:0:3911:3b6a:6a96:92a4 prefixlen 64 scopeid 0x0<global>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 1 bytes 88 (88.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 30 bytes 4833 (4.8 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> wwan0: flags=67<UP,BROADCAST,RUNNING> mtu 1428
>
> inet6 fe80::98b0:d0ff:fe6e:dc9f prefixlen 64 scopeid 0x20<link>
>
> ether 9a:b0:d0:6e:dc:9f txqueuelen 1000 (Ethernet)
>
> RX packets 3 bytes 264 (264.0 B)
>
> RX errors 0 dropped 0 overruns 0 frame 0
>
> TX packets 62 bytes 6621 (6.6 KB)
>
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
>
>
> After it fails, running “sudo mmcli -m 0 -b 1 --connect” then “sudo ip link set mbimmux0.2 up” puts things back in business. I did not expect to have to keep checking and then stand it back up when it fails.
>
MM does not do any reconnection logic by itself, because reconnections
may involve IP reconfiguration. All that should be done by upper
layers, the actual network management tool should do it.
> Questions:
>
> 1. How can I keep the second bearer connection from falling over?
>
If it's a network-reported disconnection, you cannot do anything. It
could be the network is disconnecting you because there's no traffic
ongoing or something, don't know.
> 2. Is it expected that “sudo ip link set mbimmux0.2 up” would be necessary after the second bearer connection is established? Is this the right thing to do?
>
Yes, that should be fine.
> 3. Is there a way to have the ifconfig and route entries managed by any of this software to avoid the need to “manually” manage them?
>
You need a network management tool to do that on top of ModemManager;
MM just sets everything up, and then the actual IP/route configuration
should be done by upper layers. In your case it involves manually
configuring the interface because "static" is the method reported, but
it could also be "dhcp" or "ppp" depending on the modem type and
protocol.
> 4. Any other advice to help me achieve happiness?
>
Oh, you're very close to having everything working together :) If you
script out a simple connection management logic using mmcli and
iproute2 for IP/route management, you already have the basic part
done.
If you're up to looking at NetworkManager's wwan plugin to see how
complex it would be to add the required multiplex logic support there,
that would benefit everyone, not just you :) Avoiding the explicit
disconnection on the device when a new connection is added would be a
start.
--
Aleksander
https://aleksander.es
More information about the ModemManager-devel
mailing list