Help in establishing multiple simultaneous APNs: nmcli and multiple connection keyfiles

Thu Mar 17 03:08:03 UTC 2022

On Fri, Mar 4, 2022 at 9:43 AM Aleksander Morgado <aleksander at aleksander.es>
wrote:

> Hey,
>
> > I need some help in establishing simultaneous interfaces to multiple
> APNs.
> >
> > I have an Ubuntu 20.04 system with a single cellular modem attached via
> USB in an IoT application. My problem is that I need to drive traffic
> through two different APNs when registered to the same cellular network.
> Each APN serves a different purpose -- different services with different
> billing are to occur on each.
> >
> > I've tried multiple ways to do this. While I can get connected with two
> APNs, I can only get an interface name for one of the connections. I need a
> separate interface name for each so that I can place the respective flow of
> traffic through the appropriate interface and onto the correct APN. My
> application needs both to be available simultaneously. Going back and forth
> between building up and tearing down each of them is not an acceptable
> solution.
> >
> > I tried three different things and none have brought happiness:
> >
> > (1) Using mmcli in debug mode with a set of AT commands for each APN:
> using the sequence AT+GDCONT, AT+XGAUTH, and AT+CGACT with parameters
> relevant for the first APN and then again with the sequence AT+GDCONT,
> AT+XGAUTH, and AT+CGACT for the second APN. This is successful, and I
> obtain assigned IP addresses for each APN. However, neither of the APNs are
> assigned to an interface and I don't know how to funnel traffic to each.
> >
> > (2) I have used the bearer features of mmcli. I have run "mmcli -m
> <modem-number> --create-bearer=<key-value pairs>" two times, each with the
> proper key-value pairs for the respective APNs. When I later run "mmcli -m
> <modem_number>", I see both bearers. I then run "mmcli -m <modem-number>
> --bearer <bearer-number> --connect". This works the first time for the
> first APN/bearer, and I have an interface assigned. But I get the following
> error when next running " mmcli -m <modem-number> --bearer <bearer-number>
> --connect" for the second APN/bearer:
> > error: couldn't connect the bearer: 'GDBusError:
> org.freedesktop.ModemManager1.Error.Core.NotFound: No valid data port to
> launch connection'
> >
> > (3) I've put multiple connection keyfiles in the
> /etc/NetworkManager/system-connections directory. I run "nmcli con reload"
> and then "nmcli --wait 120 con up id <name_of_first_connection>". This
> works as I get connected to the first APN, get assigned an IP address, and
> get an interface association (wwan0). However, when I run "nmcli --wait 120
> con up id <name_of_second_connection>", this works, but it displaces the
> first connection, so I am only left with the second connection on interface
> wwan0. The first connection is no longer available.
> >
> > My Ubuntu 20.04 system has mmcli 1.16.6 and nmcli 1.22.10. I could also
> install mbimcli 1.24.8, although it is not currently part of the system
> image.
> >
> > Ideas on how to accomplish what I am after?
>
> Your modem needs to be QMI or MBIM based, and you need to install
> ModemManager 1.18, which is the first version supporting multiplexing.
> Once you have that, you can:
>  *  run Modem.SimpleConnect() with"multiplex=required"
> Or:
>  * start the ModemManager daemon with the additional
> --test-multiplex-requested option.
>
> My suggestion is to use the 2nd option, just start MM with
> --test-multiplex-requested so that any attempt to start a connection
> (e.g. via NetworkManager) will trigger the multiplex setup. When
> running with multiplex enabled, ModemManager will create a new virtual
> network interface for each APN connection, and then you can handle
> custom routing rules in the host for each of them.
>
> I talked a bit about this feature in the FOSDEM2022 talk earlier this
> year, if you're interested:
> https://fosdem.org/2022/schedule/event/mobile_modemmanager/
>
> --
> Aleksander
> https://aleksander.es

I attempted to use mmcli 1.18.6 and nmcli 1.36.0 with network manager and
keyfiles to create multiple simultaneous connections to two different APNs
with a USB cellular modem on an Ubuntu Jammy Jellyfish 22.04 system. The
ModemManager.service file had the debug and test-multiplex-requested
options activated on the ExecStart line.

Short summary of results:

This sequence of commands (sudo nmcli con reload, sudo nmcli --wait 300 con
up id conn1,  sudo nmcli --wait 300 con up id conn2) did not produce
multiple simultaneous packet data network connections, but rather the
second connection displaced the first.

 More details starting from the beginning:

After booting and logging in, I plugged in the modem into the system, it
reported this configuration:

user at ubuntu22a:~$ ifconfig

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::ad94:11a5:b4a6:eada  prefixlen 64  scopeid 0x20<link>

        ether 00:0c:29:df:a4:a1  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 43  bytes 7024 (7.0 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

        device interrupt 19  base 0x2000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 407  bytes 30924 (30.9 KB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 407  bytes 30924 (30.9 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

user at ubuntu22a:~/Documents$ ip link list

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
state UNKNOWN mode DEFAULT group default qlen 1000

    link/ether 00:0c:29:df:a4:a1 brd ff:ff:ff:ff:ff:ff

    altname enp2s1

4: wwan0: <BROADCAST,MULTICAST,NOARP> mtu 1428 qdisc noop state DOWN
mode DEFAULT group default qlen 1000

    link/ether 2a:b3:6e:9a:d3:6a brd ff:ff:ff:ff:ff:ff

user at ubuntu22a:~/Documents$ route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

*<empty>*

I placed two files, named conn1 and conn2, into
/etc/NetworkManager/system-connections. These files were both structured
like the following, although the id, uuid, apn, username and password were
set to appropriately different values:

[connection]

id=conn1

uuid=201846b6-5c29-4475-8d07-91fca39beb35

type=gsm

autoconnect=false

secondaries=

interface-name=cdc-wdm0

[gsm]

apn=apnname1

number=*99#

password-flags=0

username=apnuser1

password=apnpass1

[ppp]

noauth=false

refuse-chap=true

refuse-eap=true

refuse-mschap=true

refuse-mschapv2=true

[ipv4]

dns-search=

method=auto

[ipv6]

addr-gen-mode=stable-privacy

dns-search=

ip6-privacy=0

method=ignore

I ran “sudo nmcli con reload” and then “sudo nmcli --wait 300 con up id
conn1”. This worked. This was the result on the networking arrangements:

user at ubuntu22a:~/Documents$ ifconfig

*<ens33 and lo omitted for brevity’s sake; they are identical to first listing>*

mbimmux0.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST>  mtu 1428

        inet 10.3.1.2  netmask 255.255.255.0  broadcast 10.3.1.255

        ether 2a:b3:6e:9a:d3:6a  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 28  bytes 3117 (3.1 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=67<UP,BROADCAST,RUNNING>  mtu 1428

        inet6 fe80::28b3:6eff:fe9a:d36a  prefixlen 64  scopeid 0x20<link>

        ether 2a:b3:6e:9a:d3:6a  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 33  bytes 3169 (3.1 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

user at ubuntu22a:~/Documents$ ip link list

*<1: lo and 2: ens33 omitted for brevity’s sake; they are identical to
first listing>*

4: wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc fq_codel state
UNKNOWN mode DEFAULT group default qlen 1000

    link/ether 2a:b3:6e:9a:d3:6a brd ff:ff:ff:ff:ff:ff

5: mbimmux0.1 at wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1428
qdisc noqueue state UP mode DEFAULT group default qlen 1000

    link/ether 2a:b3:6e:9a:d3:6a brd ff:ff:ff:ff:ff:ff

user at ubuntu22a:~/Documents$ route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         _gateway        0.0.0.0         UG    20700  0
0 mbimmux0.1

10.3.1.0        0.0.0.0         255.255.255.0   U     700    0
0 mbimmux0.1

link-local      0.0.0.0         255.255.0.0     U     1000   0
0 mbimmux0.1

The connectivity through this connection to the first APN worked fine.

I then attempted to establish a connection to the second APN with “sudo
nmcli --wait 300 con up id conn2”. The command reported “Connection
successfully activated (D-Bus active path:
/org/freedesktop/NetworkManager/ActiveConnection/14)”.

However, the connection to the first APN no longer worked and my
connectivity was only to the second APN. This was the result on the
networking arrangements:

user at ubuntu22a:~/Documents$ ifconfig

*<ens33 and lo omitted for brevity’s sake; they are identical to first listing>*

mbimmux0.1: flags=67<UP,BROADCAST,RUNNING>  mtu 1428

        inet 10.3.1.2  netmask 255.255.255.0  broadcast 10.3.1.255

        ether 2a:b3:6e:9a:d3:6a  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 3  bytes 126 (126.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=67<UP,BROADCAST,RUNNING>  mtu 1428

        inet6 fe80::28b3:6eff:fe9a:d36a  prefixlen 64  scopeid 0x20<link>

        ether 2a:b3:6e:9a:d3:6a  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 39  bytes 3556 (3.5 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

user at ubuntu22a:~/Documents$ ip link list

*<1: lo and 2: ens33 omitted for brevity’s sake; they are identical to
first listing>*

4: wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc fq_codel state
UNKNOWN mode DEFAULT group default qlen 1000

    link/ether 2a:b3:6e:9a:d3:6a brd ff:ff:ff:ff:ff:ff

6: mbimmux0.1 at wwan0: <BROADCAST,UP,LOWER_UP> mtu 1428 qdisc noqueue
state UP mode DEFAULT group default qlen 1000

    link/ether 2a:b3:6e:9a:d3:6a brd ff:ff:ff:ff:ff:ff

user at ubuntu22a:~/Documents$ route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         _gateway        0.0.0.0         UG    20700  0
0 mbimmux0.1

10.3.1.0        0.0.0.0         255.255.255.0   U     700    0
0 mbimmux0.1

link-local      0.0.0.0         255.255.0.0     U     1000   0
0 mbimmux0.1

Notice that in comparing the output from “ip link list” that the index
number for “mbimmux0.1 at wwan0” has changed from 5 to 6.

Thus, using this sequence of commands,

sudo nmcli con reload

nmcli --wait 300 con up id conn1

nmcli --wait 300 con up id conn2

 did not achieve two simultaneous packet data network connections, but
rather the second connection has displaced the first. This seems to have
involved the multiplexing in some way, as evidenced by the mbimmux.0.1
appearing in the “ifconfig” and “ip link list” outputs.

Any suggestions on how to make this work using the nmcli and connection
keyfiles?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/modemmanager-devel/attachments/20220316/44339e0f/attachment-0001.htm>