Promoting MM for another embedded consumer distro (how to?)

Fri Sep 30 04:55:56 UTC 2022

>
> Would you want to set up a video meeting with some of them if they
> need to clarify things? I can definitely help clarify things and I'm
> sure others like Dylan wouldn't mind to join as well.

I got this approved. We have Bi-Weekly calls, the next is actually tomorrow
at 8am PST / 5pm CET,
So if any bi-weekly dates on that timeframe works for you then let me know
and I'll get it arranged from there.

On Thu, Sep 29, 2022 at 2:35 PM matthew stanger <stangerm2 at gmail.com> wrote:

> Dylan,
>
>> What is the problem here? Could you elaborate?
>
>
> The desire is to have the fastest power on to cell connection. Folks point
> out
> the probing times for modem detection as an example of unneeded/wasted
> performance vs. a custom solution that already knows the modem/API/dev's
> that
> it's talking to. This software supports retail consumers where time and
> performance are very important. IMHO it's only AT command flows,
> understandably, that can be slower. The RDK distro will only support modems
> that support QMI/MBIM. No AT commands (maybe expect things like GPS, but
> they'll have to be proxied).
>
> No DBus, how will that go then for integration with other components?
>
>
> I think this one is pretty out their myself, these systems use
> hostapd/systemd/bluez already.. so trying to phase out dbus for performance
> reasons seems like a tall order to me. A custom IPC/system-bus would be
> used in
> it's place, called RBUS, it's open-sourced (https://github.com/rdkcmf/rbus).
> I
> think this is a conceptual thing I can't image how that would work out.
>
> Aleksander,
>
>> I know there are some other protocols, like some binary proprietary
>> protocol from MTK used in their SoCs, but if you ask me that is a step
>> backwards overall (unless they publish the doc of the protocol).
>
>
> We've already resolved this very thing. This is a perfect example of how
> we can
> help the community with direct influence to silicon makers.
>
> you don't need to know whether the
>> QMI protocol in use is very new and things like NAS System Info can be
>> used or if the QMI protocol in use is much older and only NAS Serving
>> System is supported; you don't need to know whether plain MBIM 1.0 is
>> supported or whether the device supports MBIM Extensions from
>> Microsoft (be it v1, v2 or v3); you don't need to know whether GNSS
>> management is done via AT commands + NMEA traces or via QMI LOC or via
>> QMI PDS.... and so on. MM is an abstraction layer over all those
>> things.
>>
>
> These are very helpful points to me, TY!
>
> Simplicity like
>> this has its drawbacks obviously, as you would be tied to a single
>> module and upgrade path in the future may be complicated.
>>
>
> This is how the work body is thinking of this now, but for FWA to be
> successful
> in this distro we'll need to allow MTK/QCOM module & direct modem support.
> I am
> trying really hard to get folks to stop thinking about designing in a few
> QC
> modems. We don't need to support every modem possible like desktop tries
> to do
> but we do need to have the flexibility to scale generation after
> generation.
> I'm trying to focus everyone on supporting the protocols as standards.
>
> Would you want to set up a video meeting with some of them if they
>> need to clarify things? I can definitely help clarify things and I'm
>> sure others like Dylan wouldn't mind to join as well.
>
>
> It'd be amazing if you/Dylan could drop in for 30 mins, and I think it'd
> speak
> volumes to the projects quality (I wanted to hire you for these things :)
> FWIW
> before you got snatched up). I'll need to get permission from the working
> group
> and get back to you. Meetings are Bi-weekly, I don't think it'd be an
> issue.
>
> This is a key thing. Microsoft has extended MBIM with a ton of
>> extensions, with different versions, and it gets very complex trying
>> to support all.
>
>
> Very good to know. MBIM is the target for any chipset that doesn't support
> QMI and we're taking a hard line on that. This is the way.
>
> QoS is something I've been recently discussing with others, especially
>> tied to e.g. 5G slicing.
>> multi-routing and APN data multiplexing is already supported in QMI
>> and MBIM; the only missing piece could be MTK SoCs.
>
>
> Any opinions on support for ENDC/SA, carrier aggregation, eSIM or low
> level features
> like this. Usually this is at the modem FW level but there are times when
> this
> stuff needs to be controlled (on/off really). IMO it makes sense to expose
> these things through MM to me, but I'm not sure if the project has dealt
> with
> features at that level?
>
> MM currently supports >100 different modules, and those are only the
>> ones I've tested myself lately.
>
>
> Another exciting opportunity is the testing hardware pipelines this distro
> brings. Plus we'd be testing the whole stack w/ MM/lib's at the
> carrier/PTCRB
> level in NA/EU at min. I wonder if we could help drive to a regulation
> standard
> one day (big stretch), kind of like CRDA for Wifi. The stuff going on with
> SXD55 and FCC unlocks looks a bit painful for the community?
>
> Yes. Actually, something like 11 years ago there was already a patch
>> from GIMP's maintainer Mitch to do that, but was not integrated
>> because it was trying to disable too many things. But fully disabling
>> features makes sense. One thing to keep in mind is that it's not only
>> ifdefs to disable code; disabling a feature should also e.g. disable
>> all AT URCs associated to that feature (e.g. AT+CRING regex match
>> should still exist even if voice is disabled, and the regex should end
>> up discarding the URC event silently if so).
>
>
>  Not an issue, we have many OEM engineering teams available. As long as I
> can
> get them clear direction, like you stated here we can get that kind of work
> done the way it'd be accepted by you. All of these tweaks would be a phase
> 2
> thing anyways, first I need to get the win to lock this architecture in.
> Then
> we can figure out what tuning makes since, but I did want to shed some
> light on
> some tweaks that could happen.
>
> This is not under scope at the moment. The whole MM codebase is very
>> very very tied to DBus integration, and changing that would be a bit
>> of a mess. We were able to get rid of the udev dependency for some
>> systems like openwrt, but that still requires DBus. I know a lot of
>> embedded systems that keep DBus only for ModemManager really, and
>> they're not unhappy.
>
>
> As I stated to Dylan above, this a head scratcher to me too but I'm just
> trying
> to get feedback and address everyone. I can't imagine how this would ever
> work
> out, as I mentioned above the distro already uses many dbus Linux programs
> so
> all that would have to be undone somehow... It's a very long term
> goal(years)
> for them but I think it's more of a tech debt stance. I understand this
> isn't
> something this community would want, no hard feelings here. I don't want it
> either but I also don't want it to derail getting MM into the stack, so I'm
> trying to address the concern carefully.
>
> So this is really my fault :) I know I'm slow reviewing merge
>> requests, but that is also sometimes due to how the merge requests are
>> pushed. If the changes are pushed in a way that is easy for me to
>> review (multiple commits with one logical change each, clear commit
>> messages, and so on) then it's likely that I react to the MR in a
>> better way. Also, multiple small MRs are much better to review than
>> one single MR with a ton of changes. Anyway, I think I'm improving in
>> all this lately, given that I can spend my worktime on reviewing
>> stuff. Every MR that is flagged as Ready for Review will be reviewed
>> shortly. As said earlier, more help would be welcome :) Regarding not
>> accepting MRs, I've done that in the past mostly on technical grounds
>> or because of the quality of the MR. If the submitter is able to
>> provide technical reasoning or is able to rework the MRs to make them
>> have better quality, I'd like to think that I'm reasonable and easy to
>> convince. Sending the proposal to the mailing list or to gitlab (e.g.
>> as a MR with the suggested changes in the API) before beginning the
>> work is a key thing.
>
>
> What? No, you are very quick IMO, lol. Case in point look at those PPA turn
> around times in Ubuntu. I promise no code with synchronous sleeps will
> enter
> the ML :) through us.
>
> Hope my comments help :)
>
>
> Yes, very much. Thanks for taking the time to respond to all this.
>
>
>
> On Wed, Sep 28, 2022 at 1:03 AM Aleksander Morgado <
> aleksandermj at chromium.org> wrote:
>
>> Hey Matthew,
>>
>> Let me add some more things to what Dylan already said.
>>
>> >
>> > TLDR: I'm seeking input to help champion MM as a core cell management
>> component
>> > to a large firmware collaborations body. If successful, IMHO, it'd have
>> huge
>> > benefits to WWAN/Linux wireless & MM as it'd bring in a lot of official
>> > support, more WWAN standardization, and muscle to the community.
>> >
>>
>> That would be great!
>>
>> >
>> > Here's the situation.
>> >
>> > We're working with the RDK community to add Fixed Wireless Access
>> support to
>> > the project. If you've never heard of RDK before it's basically like
>> Android
>> > for STBs & modems (rdkcentral.com/rdk-profiles/)[+80
>> <http://rdkcentral.com/rdk-profiles/)%5B+80> Million deployments].
>> > Chances are if you have a modem/STB in the US or EU it runs this
>> software. So
>> > why does this matter to MM?
>> >
>> > Getting MM in this distro would bring in a form of direct support to
>> > libQMI/libMBIM from the two silicon makers and a small army of
>> developers. It
>> > would help reinforce QMI/MBIM as standards across the WWAN industry and
>> > enhance feature/quality for those libraries and MM.
>> >
>>
>> For the most part, QMI and MBIM (mostly led by Microsoft these days)
>> are already the two main standards used by everyone, even if 3GPP
>> keeps pushing the AT reference as the default one.
>> I know there are some other protocols, like some binary proprietary
>> protocol from MTK used in their SoCs, but if you ask me that is a step
>> backwards overall (unless they publish the doc of the protocol).
>>
>> > Currently there is debate on the best solution/architecture for RDK for
>> cell
>> > support (data-only modems). While I champion for the full MM +
>> libQMI/MBIM
>> > stack many want to only use libQMI/MBIM and propose a custom but
>> technically
>> > FOSS RDK cell manager
>> > (
>> https://code.rdkcentral.com/r/plugins/gitiles/rdkb/components/opensource/ccsp/RdkCellularManager/+/refs/heads/rdk-next/source/CellularManager
>> ).
>> >
>> > The debate is over this:
>> >
>> > Prop 1:
>> > RDK Network Manager -> RDK Cell Manager(everything to control/mng cell
>> modems) -> libQMI(FreeDesktop) -> Modem
>> >
>> > Prop 2:
>> > RDK Network Manager -> RDK Cell Manager(RDK to FOSS API adaptor) ->
>> ModemManager -> libQMI/MBIM(FreeDesktop) -> Modem
>> >
>>
>> There is one critical thing here, which is what role ModemManager
>> plays in a connection stack. The role of MM is not to do a lot of
>> custom control of what the device does; MM is just a somewhat "small"
>> compatibility layer that exposes a single DBus API to manage any kind
>> of modem behind. MM by itself won't do anything, it requires a user of
>> its API (e.g. a connection manager on top of it) to connect or
>> disconnect the modem, and so on. If you use MM, you don't need to know
>> what protocol the modem is using; you don't need to know whether the
>> QMI protocol in use is very new and things like NAS System Info can be
>> used or if the QMI protocol in use is much older and only NAS Serving
>> System is supported; you don't need to know whether plain MBIM 1.0 is
>> supported or whether the device supports MBIM Extensions from
>> Microsoft (be it v1, v2 or v3); you don't need to know whether GNSS
>> management is done via AT commands + NMEA traces or via QMI LOC or via
>> QMI PDS.... and so on. MM is an abstraction layer over all those
>> things.
>>
>> There are cases where MM may not make sense, and I've found those
>> myself over the past years when I was doing freelancing. The most
>> clear example is if you need some quick connection manager that does
>> very limited things, e.g. bring a very specific modem model up, run
>> reconnections, and configure the IP settings itself. Simplicity like
>> this has its drawbacks obviously, as you would be tied to a single
>> module and upgrade path in the future may be complicated.
>>
>> >
>> > Getting ~100+ engineers/architects across dozens of company's spanning
>> the
>> > globe to agree on code is a challenging task which is why I write to
>> ask for
>> > advice.
>> >
>>
>> Would you want to set up a video meeting with some of them if they
>> need to clarify things? I can definitely help clarify things and I'm
>> sure others like Dylan wouldn't mind to join as well.
>>
>> > I'd value input on shortfalls/under-estimations of rebuilding a robust
>> cell
>> > manager, ie. MM equivalent, from scratch. Here are some things I think
>> are under
>> > estimated currently:
>> >
>> > - Ability to handle different versions/variants in QMI, & MBIB
>>
>> This is a key thing. Microsoft has extended MBIM with a ton of
>> extensions, with different versions, and it gets very complex trying
>> to support all. QMI may be a bit simpler, but that's just because the
>> different version support has already been running in MM for many
>> years so it's quite stable.
>>
>> > - Non-standard interfaces, such as GPS
>>
>> Yes, as soon as you want to do GNSS management, you would need to use
>> per-manufacturer implementations.
>>
>> > - Advanced features such as VLANS, QoS, multi-routing/data multiplexing
>>
>> VLANs are really not managed by MM, they're one stage up in the stack.
>> QoS is something I've been recently discussing with others, especially
>> tied to e.g. 5G slicing.
>> multi-routing and APN data multiplexing is already supported in QMI
>> and MBIM; the only missing piece could be MTK SoCs.
>>
>> > - Overloadable feature architecture, like plugins, to handle custom
>> > integrations
>>
>> MM has this already, yes.
>>
>> > - Handling of async modem events/telemetry
>>
>> Same.
>>
>> > - Dealing with AT commands (if required)
>>
>> The Modem.Command() API can be enabled on build time if needed.
>>
>> > - Code that can scale to support more than a few target modem modules
>>
>> MM currently supports >100 different modules, and those are only the
>> ones I've tested myself lately.
>>
>> > - Sole burden of maintaining the above complex solution
>> >
>>
>> I'm lucky to say that my current employer allows me to keep on
>> maintaining the projects during my work time. Plus, more maintainers
>> are always welcome! Dylan has been helping in reviewing merge requests
>> lately, and I definitely wouldn't mind more helping hands around.
>>
>> >
>> > The main concerns against using MM on the project are:
>> > - Binary size (can remove plugin's and in future add more make flags to
>> remove
>> > features[would be contribution])
>>
>> Yes. Actually, something like 11 years ago there was already a patch
>> from GIMP's maintainer Mitch to do that, but was not integrated
>> because it was trying to disable too many things. But fully disabling
>> features makes sense. One thing to keep in mind is that it's not only
>> ifdefs to disable code; disabling a feature should also e.g. disable
>> all AT URCs associated to that feature (e.g. AT+CRING regex match
>> should still exist even if voice is disabled, and the regex should end
>> up discarding the URC event silently if so).
>>
>> > - Performance,  (Allow list/udev tag's init modem quickly)
>>
>> If modem layout is under control, udev tags can definitely be added to
>> make it quicker to probe, there is no issue here.
>>
>> > - Dbus, integration w/o it [distance future](Challenging but could be
>> done
>> > [would be contribution])
>>
>> This is not under scope at the moment. The whole MM codebase is very
>> very very tied to DBus integration, and changing that would be a bit
>> of a mess. We were able to get rid of the udev dependency for some
>> systems like openwrt, but that still requires DBus. I know a lot of
>> embedded systems that keep DBus only for ModemManager really, and
>> they're not unhappy.
>>
>> > - Don't need voice or X feature (add more make flags to remove
>> features[would
>> > be contribution])
>>
>> As said above.
>>
>> > - Upstream will slow down project or not accept MRs (Yocto + git-patch,
>> send
>> > proposal to ML before beginning work)
>> >
>>
>> So this is really my fault :) I know I'm slow reviewing merge
>> requests, but that is also sometimes due to how the merge requests are
>> pushed. If the changes are pushed in a way that is easy for me to
>> review (multiple commits with one logical change each, clear commit
>> messages, and so on) then it's likely that I react to the MR in a
>> better way. Also, multiple small MRs are much better to review than
>> one single MR with a ton of changes. Anyway, I think I'm improving in
>> all this lately, given that I can spend my worktime on reviewing
>> stuff. Every MR that is flagged as Ready for Review will be reviewed
>> shortly. As said earlier, more help would be welcome :) Regarding not
>> accepting MRs, I've done that in the past mostly on technical grounds
>> or because of the quality of the MR. If the submitter is able to
>> provide technical reasoning or is able to rework the MRs to make them
>> have better quality, I'd like to think that I'm reasonable and easy to
>> convince. Sending the proposal to the mailing list or to gitlab (e.g.
>> as a MR with the suggested changes in the API) before beginning the
>> work is a key thing.
>>
>> >
>> > I think using MM has huge benefits over reinventing the wheel, what
>> else could
>> > I add. I've tried to highlight:
>> > - +10 years & 100+ contributors
>>
>> Plus, nowadays, Qualcomm and Intel themselves are involved in the
>> development.
>>
>> > - Proven on millions, at least, of deployments across many major Linux
>> > distros
>>
>> Yes.
>>
>> > - Alread deployed in Tier 1 consumer devices - Chromebook / ChromeOS
>>
>> Yes, and that support is not going away.
>>
>> > - Awesome & responsive community of super smart folks
>>
>> We don't bite!
>>
>> > - Go fast alone, go far as a tribe
>> >
>>
>> True.
>>
>> >
>> > I really believe this could make such a big impact to the community so
>> if you
>> > can spare advice on how I can help sell MM I'd love to hear it.
>> >
>>
>> Hope my comments help :)
>>
>> > Thanks for coming to my Ted talk.
>>
>> Hahah
>>
>> --
>> Aleksander
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/modemmanager-devel/attachments/20220929/f57c18b3/attachment-0001.htm>