DSS usage

[Bjørn Mork]

Aleksander Morgado <aleksander at aleksander.es> writes:
> On Thu, Apr 10, 2014 at 5:10 PM, Dmytro Milinevskyy
> <dmilinevskyy at sequans.com> wrote:
>> I would like to add few cents to Arnaud's email.
>> Is smth preventing modem manager to keep wwan interface up if DSS channels
>> are in use? I expect that modem manager is aware of their presence.
>> In my opinion the link state of wwan interface should always be "up". If
>> there's a need to disable wwan interface the modem manager can flush IP
>> address(es) assigned to it and clean the routing table(which normally should
>> be done automatically).
>> This will eliminate any change to existing kernel drivers.
>
> ModemManager won't be aware of DSS channels, as those will (some day,
> when it gets finished) be allocated through the 'mbim-proxy' setup in
> libmbim, not through ModemManager.
>
> Also, as a side note, bringing the iface up and keeping it that way
> will be a task of NetworkManager, not ModemManager.
>
> I have mixed feelings in to whether this is a task of the kernel (the
> one which enables VLANs for the actual DSS channels in the net
> interface) or a task for userspace (the one actually requesting to
> create the DSS channels). NetworkManager will definitely not know
> whether a DSS channel is ongoing, unless we e.g. link libmbim-glib to
> it and we maintain the list of DSS channels setup in the mbim-proxy.
> It's kind of nasty to do that, though... having it managed by the
> kernel, if easy, would be more clear - at least for userspace, of
> course...

Yes, this is revealing some downsides to the current driver design.  I'd
like to thank Dmytro and Arnaud for bringing up the necessary
discussion. Most of this has so far been just decided by me, without
much discussion at all.  Mostly because noone showed much interest...
Not even Greg S, who wrote the initial version of the cdc_mbim driver,
had much to say about DSS handling.

The multiplexing nature of MBIM did not fit directly into the current
usbnet driver framework. To support multiplexed IP sessions we needed
some way for userspace to create additional network interfaces on
demand.  And we either had to drop usbnet or implement multiple network
interfaces within the (possibly extended) usbnet framework. usbnet has a
really tight mapping between a single network interface and two bulk
endpoints.  Changing that is not an easy task.  And we really wanted to
reuse cdc_ncm as much as possible, to avoid adding yet-another
implementation of the NCM NTB framing.  cdc_ncm was of course already a
usbnet driver.

Mapping the MBIM sessions to VLANs seemed like the obvious solution once
the idea turned up. There is an existing kernel API (or several,
actually) for managing VLAN interfaces, and command line tools using
this API is installed on every Linux system out there.  And even more
important: This meant that we could reuse cdc_ncm and the usbnet
framework without adding lots of support for mapping multiple network
devices to the same two bulk endpoints.

There was a wish that the parent interface (wwan0) to always be mapped
to IP session 0, for simplicity.  The reasoning was: This network
interface will be automatically created, and everything will work
without further setup.  Setting up IP session 0 is the common case,
which should Just Work(tm)

Now, the drawback is that you have to keep the "session 0" interface up
at all times, even if you only connect higher session IDs.  And if you
snoop on the wwan0 interface, you will not only see the untagged session
0 traffic but also any tagged traffic on any other active session.

But these issues seemed minor to me...  At least compared to the
additional complexity multiple network interfaces in usbnet would
present.  I still think this is a good compromise for IP sessions.

Dealing with DSS seemed a lot more difficult for a long time.  I really
did not know how to do that.  I played with driver specific APIs to add
and remove ttys, and I played with automatic tty creation based on MBIM
control channel snooping inside the driver. But none of this really
touched the issue that the transport still depended on the same two bulk
endpoints which were controlled by usbnet and tied to a network device.
And the tty code ended up being horrific hacks (possibly because I was
writing the code...).  In theory it should all be simple.  But when you
add the on demand device creation and the upstream interfacing to a
network device, it soon became ugly and most likely very buggy. I had
exactly *one* device with DSS support to test it on, and that support
was all but standard.

I ended up preparing the cdc_mbim driver for mainline without any DSS
support at all, thinking that it would be better to add it when we knew
how userspace would want to use it.

But then I got this brilliant (feel free to interprete that as a sarcasm
:-) idea: We had only used 255 (session 0 is untagged) of our 4094
VLANs.  Why not just map the DSS sessions to 256 other VLANs?  Yes, I
know DSS is defined as "not IP" and that this nowadays pretty much means
"not network traffic". The point is that the VLANs aren't intended as
the DSS session endpoints.  They are there as a mere "standard" kernel
API for the MBIM userspace tools.  More processing is definetely needed
between the VLAN interfaces and whatever userspace consumers we have,
whether those are minicom or gpsd or something completely different.  As
noted this API choice ends up requiring dummy ethernet headers, which
must be added and removed.

The advantages of the VLAN approach for DSS were many:
- Simple and existing API for adding and removing the kernel interfaces.
- Very easy integration with the usbnet transmit and receive code.
- Flexible use, supporting *any* type of DSS, not only applications
  where ttys are the primary kernel interface.
- Keeping frame boundaries intact from device to userspace, supporting
  DSS use cases where that might matter.

The downsides are as have been mentioned:
- dependency on the parent wwan0 interface, which belongs to IP session 0 
  management
- odd userspace interface, requiring dummy ethernet headers to be added
  and stripped in userspace

Of these, the dependecy on wwan0 is by far the worst IMHO.  And I would
have loved to just "fix" that in the driver.  But when Arnaud first made
me think about this, I realized that it wasn't really me who added
that... It's really an inheritage of the multiplexing protocol.  There
is no way you are going to transport those DSS fragments without
cooperating with all the other users of the two bulk endpoints on the
MBIM data interface.  There just has to be some top level policy manager
here "owning" both the MBIM data and control channels. wwan0 represents
the shared data channel, and /dev/cdc-wdm0 represents the shared control
channel.  Both these resources must be managed by one application to
facilitate resource sharing.

Yes, we have moved a lot of management stuff from the kernel to
userspace here.  Maybe too much?  It started out as a blueprint of the
qmi_wwan design, but the data multiplexing stuff adds another
dimension.  Maybe we should have let the driver do the resource
management?  But then we would have had to let it manage the control
channel as well.  Which would not have been a good solution, with an
extensible protocol like MBIM.

I don't think we are going to change much of this now in any case.  We
might be able to add some extra policy around the wwan0 link management
in the driver, to avoid userspace applications not aware of MBIM at all,
like generic DHCP clients, from taking the link down without approval
from the MBIM management application.  But I don't know exactly how to
do that.  And it is just taking some control away from userspace.  Not
sure about that at all.

I am not in favour of rushing to do anything here. It seems to me that
the kernel currently allows any possible use case we can come up with,
only with a few more gotchas than we wanted. I believe this is still
better than ending up removing support for some use case.  And we do
have to obey the kernel userspace API rules, which severely limits our
ability to change any major bits at this point. We can of course add
stuff and fix bugs, but we cannot really change or remove anything.


Bjørn