Promoting MM for another embedded consumer distro (how to?)

[matthew stanger]

Dylan,

> What is the problem here? Could you elaborate?


The desire is to have the fastest power on to cell connection. Folks point
out
the probing times for modem detection as an example of unneeded/wasted
performance vs. a custom solution that already knows the modem/API/dev's
that
it's talking to. This software supports retail consumers where time and
performance are very important. IMHO it's only AT command flows,
understandably, that can be slower. The RDK distro will only support modems
that support QMI/MBIM. No AT commands (maybe expect things like GPS, but
they'll have to be proxied).

No DBus, how will that go then for integration with other components?


I think this one is pretty out their myself, these systems use
hostapd/systemd/bluez already.. so trying to phase out dbus for performance
reasons seems like a tall order to me. A custom IPC/system-bus would be
used in
it's place, called RBUS, it's open-sourced (https://github.com/rdkcmf/rbus).
I
think this is a conceptual thing I can't image how that would work out.

Aleksander,

> I know there are some other protocols, like some binary proprietary
> protocol from MTK used in their SoCs, but if you ask me that is a step
> backwards overall (unless they publish the doc of the protocol).


We've already resolved this very thing. This is a perfect example of how we
can
help the community with direct influence to silicon makers.

you don't need to know whether the
> QMI protocol in use is very new and things like NAS System Info can be
> used or if the QMI protocol in use is much older and only NAS Serving
> System is supported; you don't need to know whether plain MBIM 1.0 is
> supported or whether the device supports MBIM Extensions from
> Microsoft (be it v1, v2 or v3); you don't need to know whether GNSS
> management is done via AT commands + NMEA traces or via QMI LOC or via
> QMI PDS.... and so on. MM is an abstraction layer over all those
> things.
>

These are very helpful points to me, TY!

Simplicity like
> this has its drawbacks obviously, as you would be tied to a single
> module and upgrade path in the future may be complicated.
>

This is how the work body is thinking of this now, but for FWA to be
successful
in this distro we'll need to allow MTK/QCOM module & direct modem support.
I am
trying really hard to get folks to stop thinking about designing in a few QC
modems. We don't need to support every modem possible like desktop tries to
do
but we do need to have the flexibility to scale generation after generation.
I'm trying to focus everyone on supporting the protocols as standards.

Would you want to set up a video meeting with some of them if they
> need to clarify things? I can definitely help clarify things and I'm
> sure others like Dylan wouldn't mind to join as well.


It'd be amazing if you/Dylan could drop in for 30 mins, and I think it'd
speak
volumes to the projects quality (I wanted to hire you for these things :)
FWIW
before you got snatched up). I'll need to get permission from the working
group
and get back to you. Meetings are Bi-weekly, I don't think it'd be an issue.

This is a key thing. Microsoft has extended MBIM with a ton of
> extensions, with different versions, and it gets very complex trying
> to support all.


Very good to know. MBIM is the target for any chipset that doesn't support
QMI and we're taking a hard line on that. This is the way.

QoS is something I've been recently discussing with others, especially
> tied to e.g. 5G slicing.
> multi-routing and APN data multiplexing is already supported in QMI
> and MBIM; the only missing piece could be MTK SoCs.


Any opinions on support for ENDC/SA, carrier aggregation, eSIM or low level
features
like this. Usually this is at the modem FW level but there are times when
this
stuff needs to be controlled (on/off really). IMO it makes sense to expose
these things through MM to me, but I'm not sure if the project has dealt
with
features at that level?

MM currently supports >100 different modules, and those are only the
> ones I've tested myself lately.


Another exciting opportunity is the testing hardware pipelines this distro
brings. Plus we'd be testing the whole stack w/ MM/lib's at the
carrier/PTCRB
level in NA/EU at min. I wonder if we could help drive to a regulation
standard
one day (big stretch), kind of like CRDA for Wifi. The stuff going on with
SXD55 and FCC unlocks looks a bit painful for the community?

Yes. Actually, something like 11 years ago there was already a patch
> from GIMP's maintainer Mitch to do that, but was not integrated
> because it was trying to disable too many things. But fully disabling
> features makes sense. One thing to keep in mind is that it's not only
> ifdefs to disable code; disabling a feature should also e.g. disable
> all AT URCs associated to that feature (e.g. AT+CRING regex match
> should still exist even if voice is disabled, and the regex should end
> up discarding the URC event silently if so).


 Not an issue, we have many OEM engineering teams available. As long as I
can
get them clear direction, like you stated here we can get that kind of work
done the way it'd be accepted by you. All of these tweaks would be a phase 2
thing anyways, first I need to get the win to lock this architecture in.
Then
we can figure out what tuning makes since, but I did want to shed some
light on
some tweaks that could happen.

This is not under scope at the moment. The whole MM codebase is very
> very very tied to DBus integration, and changing that would be a bit
> of a mess. We were able to get rid of the udev dependency for some
> systems like openwrt, but that still requires DBus. I know a lot of
> embedded systems that keep DBus only for ModemManager really, and
> they're not unhappy.


As I stated to Dylan above, this a head scratcher to me too but I'm just
trying
to get feedback and address everyone. I can't imagine how this would ever
work
out, as I mentioned above the distro already uses many dbus Linux programs
so
all that would have to be undone somehow... It's a very long term
goal(years)
for them but I think it's more of a tech debt stance. I understand this
isn't
something this community would want, no hard feelings here. I don't want it
either but I also don't want it to derail getting MM into the stack, so I'm
trying to address the concern carefully.

So this is really my fault :) I know I'm slow reviewing merge
> requests, but that is also sometimes due to how the merge requests are
> pushed. If the changes are pushed in a way that is easy for me to
> review (multiple commits with one logical change each, clear commit
> messages, and so on) then it's likely that I react to the MR in a
> better way. Also, multiple small MRs are much better to review than
> one single MR with a ton of changes. Anyway, I think I'm improving in
> all this lately, given that I can spend my worktime on reviewing
> stuff. Every MR that is flagged as Ready for Review will be reviewed
> shortly. As said earlier, more help would be welcome :) Regarding not
> accepting MRs, I've done that in the past mostly on technical grounds
> or because of the quality of the MR. If the submitter is able to
> provide technical reasoning or is able to rework the MRs to make them
> have better quality, I'd like to think that I'm reasonable and easy to
> convince. Sending the proposal to the mailing list or to gitlab (e.g.
> as a MR with the suggested changes in the API) before beginning the
> work is a key thing.


What? No, you are very quick IMO, lol. Case in point look at those PPA turn
around times in Ubuntu. I promise no code with synchronous sleeps will enter
the ML :) through us.

Hope my comments help :)


Yes, very much. Thanks for taking the time to respond to all this.



On Wed, Sep 28, 2022 at 1:03 AM Aleksander Morgado <
aleksandermj at chromium.org> wrote:

> Hey Matthew,
>
> Let me add some more things to what Dylan already said.
>
> >
> > TLDR: I'm seeking input to help champion MM as a core cell management
> component
> > to a large firmware collaborations body. If successful, IMHO, it'd have
> huge
> > benefits to WWAN/Linux wireless & MM as it'd bring in a lot of official
> > support, more WWAN standardization, and muscle to the community.
> >
>
> That would be great!
>
> >
> > Here's the situation.
> >
> > We're working with the RDK community to add Fixed Wireless Access
> support to
> > the project. If you've never heard of RDK before it's basically like
> Android
> > for STBs & modems (rdkcentral.com/rdk-profiles/)[+80
> <http://rdkcentral.com/rdk-profiles/)%5B+80> Million deployments].
> > Chances are if you have a modem/STB in the US or EU it runs this
> software. So
> > why does this matter to MM?
> >
> > Getting MM in this distro would bring in a form of direct support to
> > libQMI/libMBIM from the two silicon makers and a small army of
> developers. It
> > would help reinforce QMI/MBIM as standards across the WWAN industry and
> > enhance feature/quality for those libraries and MM.
> >
>
> For the most part, QMI and MBIM (mostly led by Microsoft these days)
> are already the two main standards used by everyone, even if 3GPP
> keeps pushing the AT reference as the default one.
> I know there are some other protocols, like some binary proprietary
> protocol from MTK used in their SoCs, but if you ask me that is a step
> backwards overall (unless they publish the doc of the protocol).
>
> > Currently there is debate on the best solution/architecture for RDK for
> cell
> > support (data-only modems). While I champion for the full MM +
> libQMI/MBIM
> > stack many want to only use libQMI/MBIM and propose a custom but
> technically
> > FOSS RDK cell manager
> > (
> https://code.rdkcentral.com/r/plugins/gitiles/rdkb/components/opensource/ccsp/RdkCellularManager/+/refs/heads/rdk-next/source/CellularManager
> ).
> >
> > The debate is over this:
> >
> > Prop 1:
> > RDK Network Manager -> RDK Cell Manager(everything to control/mng cell
> modems) -> libQMI(FreeDesktop) -> Modem
> >
> > Prop 2:
> > RDK Network Manager -> RDK Cell Manager(RDK to FOSS API adaptor) ->
> ModemManager -> libQMI/MBIM(FreeDesktop) -> Modem
> >
>
> There is one critical thing here, which is what role ModemManager
> plays in a connection stack. The role of MM is not to do a lot of
> custom control of what the device does; MM is just a somewhat "small"
> compatibility layer that exposes a single DBus API to manage any kind
> of modem behind. MM by itself won't do anything, it requires a user of
> its API (e.g. a connection manager on top of it) to connect or
> disconnect the modem, and so on. If you use MM, you don't need to know
> what protocol the modem is using; you don't need to know whether the
> QMI protocol in use is very new and things like NAS System Info can be
> used or if the QMI protocol in use is much older and only NAS Serving
> System is supported; you don't need to know whether plain MBIM 1.0 is
> supported or whether the device supports MBIM Extensions from
> Microsoft (be it v1, v2 or v3); you don't need to know whether GNSS
> management is done via AT commands + NMEA traces or via QMI LOC or via
> QMI PDS.... and so on. MM is an abstraction layer over all those
> things.
>
> There are cases where MM may not make sense, and I've found those
> myself over the past years when I was doing freelancing. The most
> clear example is if you need some quick connection manager that does
> very limited things, e.g. bring a very specific modem model up, run
> reconnections, and configure the IP settings itself. Simplicity like
> this has its drawbacks obviously, as you would be tied to a single
> module and upgrade path in the future may be complicated.
>
> >
> > Getting ~100+ engineers/architects across dozens of company's spanning
> the
> > globe to agree on code is a challenging task which is why I write to ask
> for
> > advice.
> >
>
> Would you want to set up a video meeting with some of them if they
> need to clarify things? I can definitely help clarify things and I'm
> sure others like Dylan wouldn't mind to join as well.
>
> > I'd value input on shortfalls/under-estimations of rebuilding a robust
> cell
> > manager, ie. MM equivalent, from scratch. Here are some things I think
> are under
> > estimated currently:
> >
> > - Ability to handle different versions/variants in QMI, & MBIB
>
> This is a key thing. Microsoft has extended MBIM with a ton of
> extensions, with different versions, and it gets very complex trying
> to support all. QMI may be a bit simpler, but that's just because the
> different version support has already been running in MM for many
> years so it's quite stable.
>
> > - Non-standard interfaces, such as GPS
>
> Yes, as soon as you want to do GNSS management, you would need to use
> per-manufacturer implementations.
>
> > - Advanced features such as VLANS, QoS, multi-routing/data multiplexing
>
> VLANs are really not managed by MM, they're one stage up in the stack.
> QoS is something I've been recently discussing with others, especially
> tied to e.g. 5G slicing.
> multi-routing and APN data multiplexing is already supported in QMI
> and MBIM; the only missing piece could be MTK SoCs.
>
> > - Overloadable feature architecture, like plugins, to handle custom
> > integrations
>
> MM has this already, yes.
>
> > - Handling of async modem events/telemetry
>
> Same.
>
> > - Dealing with AT commands (if required)
>
> The Modem.Command() API can be enabled on build time if needed.
>
> > - Code that can scale to support more than a few target modem modules
>
> MM currently supports >100 different modules, and those are only the
> ones I've tested myself lately.
>
> > - Sole burden of maintaining the above complex solution
> >
>
> I'm lucky to say that my current employer allows me to keep on
> maintaining the projects during my work time. Plus, more maintainers
> are always welcome! Dylan has been helping in reviewing merge requests
> lately, and I definitely wouldn't mind more helping hands around.
>
> >
> > The main concerns against using MM on the project are:
> > - Binary size (can remove plugin's and in future add more make flags to
> remove
> > features[would be contribution])
>
> Yes. Actually, something like 11 years ago there was already a patch
> from GIMP's maintainer Mitch to do that, but was not integrated
> because it was trying to disable too many things. But fully disabling
> features makes sense. One thing to keep in mind is that it's not only
> ifdefs to disable code; disabling a feature should also e.g. disable
> all AT URCs associated to that feature (e.g. AT+CRING regex match
> should still exist even if voice is disabled, and the regex should end
> up discarding the URC event silently if so).
>
> > - Performance,  (Allow list/udev tag's init modem quickly)
>
> If modem layout is under control, udev tags can definitely be added to
> make it quicker to probe, there is no issue here.
>
> > - Dbus, integration w/o it [distance future](Challenging but could be
> done
> > [would be contribution])
>
> This is not under scope at the moment. The whole MM codebase is very
> very very tied to DBus integration, and changing that would be a bit
> of a mess. We were able to get rid of the udev dependency for some
> systems like openwrt, but that still requires DBus. I know a lot of
> embedded systems that keep DBus only for ModemManager really, and
> they're not unhappy.
>
> > - Don't need voice or X feature (add more make flags to remove
> features[would
> > be contribution])
>
> As said above.
>
> > - Upstream will slow down project or not accept MRs (Yocto + git-patch,
> send
> > proposal to ML before beginning work)
> >
>
> So this is really my fault :) I know I'm slow reviewing merge
> requests, but that is also sometimes due to how the merge requests are
> pushed. If the changes are pushed in a way that is easy for me to
> review (multiple commits with one logical change each, clear commit
> messages, and so on) then it's likely that I react to the MR in a
> better way. Also, multiple small MRs are much better to review than
> one single MR with a ton of changes. Anyway, I think I'm improving in
> all this lately, given that I can spend my worktime on reviewing
> stuff. Every MR that is flagged as Ready for Review will be reviewed
> shortly. As said earlier, more help would be welcome :) Regarding not
> accepting MRs, I've done that in the past mostly on technical grounds
> or because of the quality of the MR. If the submitter is able to
> provide technical reasoning or is able to rework the MRs to make them
> have better quality, I'd like to think that I'm reasonable and easy to
> convince. Sending the proposal to the mailing list or to gitlab (e.g.
> as a MR with the suggested changes in the API) before beginning the
> work is a key thing.
>
> >
> > I think using MM has huge benefits over reinventing the wheel, what else
> could
> > I add. I've tried to highlight:
> > - +10 years & 100+ contributors
>
> Plus, nowadays, Qualcomm and Intel themselves are involved in the
> development.
>
> > - Proven on millions, at least, of deployments across many major Linux
> > distros
>
> Yes.
>
> > - Alread deployed in Tier 1 consumer devices - Chromebook / ChromeOS
>
> Yes, and that support is not going away.
>
> > - Awesome & responsive community of super smart folks
>
> We don't bite!
>
> > - Go fast alone, go far as a tribe
> >
>
> True.
>
> >
> > I really believe this could make such a big impact to the community so
> if you
> > can spare advice on how I can help sell MM I'd love to hear it.
> >
>
> Hope my comments help :)
>
> > Thanks for coming to my Ted talk.
>
> Hahah
>
> --
> Aleksander
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/modemmanager-devel/attachments/20220929/352e4c4c/attachment-0001.htm>