From nietzsche at lysator.liu.se Mon Oct 20 10:40:45 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Mon, 20 Oct 2014 19:40:45 +0200 Subject: Ethernet-briding a MBIM-interface Message-ID: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> Hi, has anyone worked out how to bridge an MBIM-interface with a regular ethernet interface? I think there are 2 ways to do it: 1) Using ebtables. 2) Something like this: http://www.spinics.net/lists/linux-usb/msg96303.html Any ideas? //Markus - The panama-hat hacker -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bjorn at mork.no Thu Oct 23 04:41:41 2014 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Thu, 23 Oct 2014 13:41:41 +0200 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> (Markus Gothe's message of "Mon, 20 Oct 2014 19:40:45 +0200") References: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> Message-ID: <87wq7rko7e.fsf@nemi.mork.no> Markus Gothe writes: > Hi, > has anyone worked out how to bridge an MBIM-interface with a regular ethernet interface? > I think there are 2 ways to do it: > > 1) Using ebtables. > 2) Something like this: http://www.spinics.net/lists/linux-usb/msg96303.html True bridging of LTE/3G interfaces is simply impossible. We can only fake a bridge by routing the IP packets across point-to-point links and moving the global IP address to the other end of this link. This is what the modem does when it presents its local end of the GTP tunnel as an MBIM IP session to the host. All LTE/3G modems presenting some sort of ethernet like interface to the host does something like this. Bridging these fake ethernet interfaces might be possible, depending in the implementation. But I believe it's impossible to bridge the interface created by the Linux MBIM driver due to the way it forces the usage of the local MAC address as a destination. The ethernet frames delievered by this driver will always go to the local host. Creating a bridge will not change the behaviour. You can probably do MAC address rewriting using ebtables to overcome the problem, but that is unnecessarily complicated IMHO. There is nothing preventing you from continuing the game played by the modem over more point-to-point links, including ethernet links. And you can make the last one of these look pretty much like an ethernet LAN with a DHCP server handing out a single address and a default gateway answering ARP requests. Making the modem firmware take care of the DHCP requests will just complicate this (it can be done, but I do not see the point). Run your own DHCP server on the ethernet LAN interface instead, using a config based on the MBIM IP configuraton data. The rest is plain IP forwarding between the ethernet and MBIM interface: A default route out the MBIM interface and a /32 host route out the ethernet interface, using proxy ARP on the ethernet interface to ensure that it will answer requests for the (fake) gateway address. Alternatively you can configure the gateway address on the ethernet interface, but it will then become unreachable for the connected host. Whether that is a problem or not is another question, but it does make the setup less transparent. Bj?rn From nietzsche at lysator.liu.se Thu Oct 23 09:30:43 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Thu, 23 Oct 2014 18:30:43 +0200 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <87wq7rko7e.fsf@nemi.mork.no> References: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> <87wq7rko7e.fsf@nemi.mork.no> Message-ID: <67283673-E69A-4183-8EBF-B395A8D237FB@lysator.liu.se> Hi, Bj?rn and thanks for your feedback. 2G/3G networks are circuit switch running PPP over the backhaul. LTE is circuit switch and talking IP over the backhaul. This is why Sierra?s DirectIP / sierra_net.c is a half-bridge. It is by an old design. Hiding the PPP and using a DHCP-hack (older DirectIP devices actually wants the DHCP-server to be implemented in sierra_net.c using CnS/HIP). The local MAC-issue will be overcome by using eatables to do MAC-DNAT (been there done that before). The use case for being able to bridge MBIM is of course embedded devices. Not computers. Playing around with proxy arp / parprouted might be a good starter. :-) So basically you are saying that MBIM and Sierra DirecIP is reassembling each other very close? //M On 23 Oct 2014, at 13:41 , Bj?rn Mork wrote: > Markus Gothe writes: > >> Hi, >> has anyone worked out how to bridge an MBIM-interface with a regular ethernet interface? >> I think there are 2 ways to do it: >> >> 1) Using ebtables. >> 2) Something like this: http://www.spinics.net/lists/linux-usb/msg96303.html > > True bridging of LTE/3G interfaces is simply impossible. We can only > fake a bridge by routing the IP packets across point-to-point links and > moving the global IP address to the other end of this link. This is > what the modem does when it presents its local end of the GTP tunnel as > an MBIM IP session to the host. > > All LTE/3G modems presenting some sort of ethernet like interface to the > host does something like this. Bridging these fake ethernet interfaces > might be possible, depending in the implementation. > > But I believe it's impossible to bridge the interface created by the > Linux MBIM driver due to the way it forces the usage of the local MAC > address as a destination. The ethernet frames delievered by this driver > will always go to the local host. Creating a bridge will not change the > behaviour. You can probably do MAC address rewriting using ebtables to > overcome the problem, but that is unnecessarily complicated IMHO. > > There is nothing preventing you from continuing the game played by the > modem over more point-to-point links, including ethernet links. And you > can make the last one of these look pretty much like an ethernet LAN > with a DHCP server handing out a single address and a default gateway > answering ARP requests. Making the modem firmware take care of the DHCP > requests will just complicate this (it can be done, but I do not see the > point). Run your own DHCP server on the ethernet LAN interface instead, > using a config based on the MBIM IP configuraton data. The rest is > plain IP forwarding between the ethernet and MBIM interface: A default > route out the MBIM interface and a /32 host route out the ethernet > interface, using proxy ARP on the ethernet interface to ensure that it > will answer requests for the (fake) gateway address. Alternatively you > can configure the gateway address on the ethernet interface, but it will > then become unreachable for the connected host. Whether that is a > problem or not is another question, but it does make the setup less > transparent. > > > Bj?rn //Markus - The panama-hat hacker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bjorn at mork.no Fri Oct 24 00:04:23 2014 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Fri, 24 Oct 2014 09:04:23 +0200 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <67283673-E69A-4183-8EBF-B395A8D237FB@lysator.liu.se> (Markus Gothe's message of "Thu, 23 Oct 2014 18:30:43 +0200") References: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> <87wq7rko7e.fsf@nemi.mork.no> <67283673-E69A-4183-8EBF-B395A8D237FB@lysator.liu.se> Message-ID: <87iojaeyo8.fsf@nemi.mork.no> Markus Gothe writes: > Hi, Bj?rn and thanks for your feedback. > > 2G/3G networks are circuit switch running PPP over the backhaul. LTE > is circuit switch and talking IP over the backhaul. This is why > Sierra?s DirectIP / sierra_net.c is a half-bridge. It is by an old > design. Hiding the PPP and using a DHCP-hack (older DirectIP devices > actually wants the DHCP-server to be implemented in sierra_net.c using > CnS/HIP). Yes, I have wondered if that wouldn't have been a better design after all. Having a DHCP server inside the driver would be messy, of course. But having it inside the modem firmware isn't less messy, and prevents us from fixing it if that should become necessary. Luckily it hasn't. For MBIM, I believe the best option for the host is to use the IP configuration request instead of trusting some random DHCP server implementation in the firmware. > The local MAC-issue will be overcome by using eatables to do MAC-DNAT > (been there done that before). The use case for being able to bridge > MBIM is of course embedded devices. Not computers. > > Playing around with proxy arp / parprouted might be a good starter. :-) > > So basically you are saying that MBIM and Sierra DirecIP is > reassembling each other very close? Yup. And also any other modem presenting an ethernet like interface, like NCM or ECM (with or without a QMI channel) Some might argue that MBIM should be different because there are no ethernet headers, but we messed that up in the Linux driver :-) Bj?rn From nietzsche at lysator.liu.se Fri Oct 24 00:23:20 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Fri, 24 Oct 2014 09:23:20 +0200 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <87iojaeyo8.fsf@nemi.mork.no> References: <4654DA40-047A-4CE1-A5DC-1AD49026C369@lysator.liu.se> <87wq7rko7e.fsf@nemi.mork.no> <67283673-E69A-4183-8EBF-B395A8D237FB@lysator.liu.se> <87iojaeyo8.fsf@nemi.mork.no> Message-ID: Well? The DirectIP and CDC ECM behaves differently at least because you put the logic inside the modem firmware :-) Having a DHCP-server inside cdc_mbim.c should not be too difficult, look at an old version of sierra_net.c Well, MBIM is supposed to be like QMI ?raw-ip?-mode. Maybe we could do the same mess up for that case. ;-) //M On 24 Oct 2014, at 09:04 , Bj?rn Mork wrote: > Markus Gothe writes: > >> Hi, Bj?rn and thanks for your feedback. >> >> 2G/3G networks are circuit switch running PPP over the backhaul. LTE >> is circuit switch and talking IP over the backhaul. This is why >> Sierra?s DirectIP / sierra_net.c is a half-bridge. It is by an old >> design. Hiding the PPP and using a DHCP-hack (older DirectIP devices >> actually wants the DHCP-server to be implemented in sierra_net.c using >> CnS/HIP). > > Yes, I have wondered if that wouldn't have been a better design after > all. Having a DHCP server inside the driver would be messy, of course. > But having it inside the modem firmware isn't less messy, and prevents > us from fixing it if that should become necessary. Luckily it hasn't. > > For MBIM, I believe the best option for the host is to use the IP > configuration request instead of trusting some random DHCP server > implementation in the firmware. > >> The local MAC-issue will be overcome by using eatables to do MAC-DNAT >> (been there done that before). The use case for being able to bridge >> MBIM is of course embedded devices. Not computers. >> >> Playing around with proxy arp / parprouted might be a good starter. :-) >> >> So basically you are saying that MBIM and Sierra DirecIP is >> reassembling each other very close? > > Yup. And also any other modem presenting an ethernet like interface, > like NCM or ECM (with or without a QMI channel) > > Some might argue that MBIM should be different because there are no > ethernet headers, but we messed that up in the Linux driver :-) > > > Bj?rn > _______________________________________________ > libmbim-devel mailing list > libmbim-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/libmbim-devel //Markus - The panama-hat hacker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: Message signed with OpenPGP using GPGMail URL: From nietzsche at lysator.liu.se Mon Oct 27 11:56:47 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Mon, 27 Oct 2014 19:56:47 +0100 Subject: Ethernet-briding a MBIM-interface Message-ID: <20141027185650.6EB1D4002E@mail.lysator.liu.se> The rationale for having a null mac adress as the ethernet source adress would be nice to know. Wouldnt it be better to keep it the same as the ethernet destination adress (like some CDC ECM implementations)??? This is afaik to be considered a bug. :-) //M Skickat ifrån min mobil. Den 24 okt 2014 09:23 skrev Markus Gothe : > > Well… > The DirectIP and CDC ECM behaves differently at least because you put the logic inside the modem firmware :-) > > Having a DHCP-server inside cdc_mbim.c should not be too difficult, look at an old version of sierra_net.c > > Well, MBIM is supposed to be like QMI ‘raw-ip’-mode. Maybe we could do the same mess up for that case. ;-) > > //M > > On 24 Oct 2014, at 09:04 , Bjørn Mork wrote: > > > Markus Gothe writes: > > > >> Hi, Björn and thanks for your feedback. > >> > >> 2G/3G networks are circuit switch running PPP over the backhaul. LTE > >> is circuit switch and talking IP over the backhaul.  This is why > >> Sierra’s DirectIP / sierra_net.c is a half-bridge. It is by an old > >> design. Hiding the PPP and using a DHCP-hack (older DirectIP devices > >> actually wants the DHCP-server to be implemented in sierra_net.c using > >> CnS/HIP). > > > > Yes, I have wondered if that wouldn't have been a better design after > > all.  Having a DHCP server inside the driver would be messy, of course. > > But having it inside the modem firmware isn't less messy, and prevents > > us from fixing it if that should become necessary.  Luckily it hasn't. > > > > For MBIM, I believe the best option for the host is to use the IP > > configuration request instead of trusting some random DHCP server > > implementation in the firmware. > > > >> The local MAC-issue will be overcome by using eatables to do MAC-DNAT > >> (been there done that before).  The use case for being able to bridge > >> MBIM is of course embedded devices. Not computers. > >> > >> Playing around with proxy arp / parprouted might be a good starter. :-) > >> > >> So basically you are saying that MBIM and Sierra DirecIP is > >> reassembling each other very close? > > > > Yup. And also any other modem presenting an ethernet like interface, > > like NCM or ECM (with or without a QMI channel) > > > > Some might argue that MBIM should be different because there are no > > ethernet headers, but we messed that up in the Linux driver :-) > > > > > > Bjørn > > _______________________________________________ > > libmbim-devel mailing list > > libmbim-devel at lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/libmbim-devel > > > > //Markus - The panama-hat hacker > > _______________________________________________ > libmbim-devel mailing list > libmbim-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/libmbim-devel From bjorn at mork.no Tue Oct 28 00:41:56 2014 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Tue, 28 Oct 2014 08:41:56 +0100 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <20141027185650.6EB1D4002E@mail.lysator.liu.se> (Markus Gothe's message of "Mon, 27 Oct 2014 19:56:47 +0100") References: <20141027185650.6EB1D4002E@mail.lysator.liu.se> Message-ID: <87a94g4p4r.fsf@nemi.mork.no> Markus Gothe writes: > The rationale for having a null mac adress as the ethernet source > adress would be nice to know. This just wasn't given much thought. I couldn't imagine where it would make a difference, and not copying anything seemed like slightly less work than copying something which wouldn't matter anyway. The driver give the frames to usbnet_skb_return() which does eth_type_trans() on them. This pulls the ethernet header from the skb after looking at the h_proto and h_dest fields. I obviously didn't think about what happens to frames being snooped or bridged... > Wouldnt it be better to keep it the same as the ethernet destination > adress (like some CDC ECM implementations)??? This is afaik to be > considered a bug. :-) Yes, that would be more consistent also with the IFF_NOARP frames coming from the modem. I believe we can change this if you can point to any use case where it actually matters. I'm not sure I care enough to try to push the change without any justification. Is bridging such a use case? Would it sort of work if the source address was saner? Bjørn From nietzsche at lysator.liu.se Tue Oct 28 00:52:33 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Tue, 28 Oct 2014 08:52:33 +0100 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <87a94g4p4r.fsf@nemi.mork.no> References: <20141027185650.6EB1D4002E@mail.lysator.liu.se> <87a94g4p4r.fsf@nemi.mork.no> Message-ID: I’ve a patch for with improved ARP-handling and setting the mac source, which gives me a DHCP-lease when briding a device… I’ll try more when I got everything at my hands. Running Linux inside a VM isn’t a good test case. //M On 28 Oct 2014, at 08:41 , Bjørn Mork wrote: > Markus Gothe writes: > >> The rationale for having a null mac adress as the ethernet source >> adress would be nice to know. > > This just wasn't given much thought. I couldn't imagine where it would > make a difference, and not copying anything seemed like slightly less > work than copying something which wouldn't matter anyway. The driver > give the frames to usbnet_skb_return() which does eth_type_trans() on > them. This pulls the ethernet header from the skb after looking at the > h_proto and h_dest fields. I obviously didn't think about what happens > to frames being snooped or bridged... > >> Wouldnt it be better to keep it the same as the ethernet destination >> adress (like some CDC ECM implementations)??? This is afaik to be >> considered a bug. :-) > > Yes, that would be more consistent also with the IFF_NOARP frames coming > from the modem. I believe we can change this if you can point to any > use case where it actually matters. I'm not sure I care enough to try > to push the change without any justification. > > Is bridging such a use case? Would it sort of work if the source > address was saner? > > > Bjørn > _______________________________________________ > libmbim-devel mailing list > libmbim-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/libmbim-devel //Markus - The panama-hat hacker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bjorn at mork.no Tue Oct 28 01:14:49 2014 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Tue, 28 Oct 2014 09:14:49 +0100 Subject: Ethernet-briding a MBIM-interface In-Reply-To: (Markus Gothe's message of "Tue, 28 Oct 2014 08:52:33 +0100") References: <20141027185650.6EB1D4002E@mail.lysator.liu.se> <87a94g4p4r.fsf@nemi.mork.no> Message-ID: <87wq7k391i.fsf@nemi.mork.no> Markus Gothe writes: > I’ve a patch for with improved ARP-handling and setting the mac source, > which gives me a DHCP-lease when briding a device… Yes, you are right. Looking at net/bridge/br_input.c shows that the all zero source address is a showstopper: rx_handler_result_t br_handle_frame(struct sk_buff **pskb) { .. if (!is_valid_ether_addr(eth_hdr(skb)->h_source)) goto drop; That's good enough for me. Let's attempt to change it. As for ARP handling in the driver: I don't think that will happen. It's just too ugly. Yes, I know I wrote some testing code, but that was just to try to figure out how to deal with an IPv6 related firmware issue. If you need ARP then you should either enable proxy ARP or run some userspace application to handle it. Bjørn From nietzsche at lysator.liu.se Tue Oct 28 01:26:40 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Tue, 28 Oct 2014 09:26:40 +0100 Subject: Ethernet-briding a MBIM-interface Message-ID: <20141028082642.BA4124004E@mail.lysator.liu.se> Explains why the fix_dst-stuff breaks briding in qmi_wwan.c as well I guess. However even if setting a null mac there it seems to be replaced by the modem fw, right? Sure I can do ARP-replies with ebtables / arptables as well I suppose. :-) ARP is all about preferences. //M Skickat ifrån min mobil. Den 28 okt 2014 09:14 skrev =?ISO-8859-1?Q?Bj=F8rn_Mork?= : > > Markus Gothe writes: > > > I’ve a patch for with improved ARP-handling and setting the mac source, > >  which gives me a DHCP-lease when briding a device… > > Yes, you are right.  Looking at net/bridge/br_input.c shows that the all > zero source address is a showstopper: > > rx_handler_result_t br_handle_frame(struct sk_buff **pskb) > { > .. >         if (!is_valid_ether_addr(eth_hdr(skb)->h_source)) >                 goto drop; > > > That's good enough for me.  Let's attempt to change it. > > As for ARP handling in the driver: I don't think that will happen.  It's > just too ugly.  Yes, I know I wrote some testing code, but that was just > to try to figure out how to deal with an IPv6 related firmware issue. > > If you need ARP then you should either enable proxy ARP or run some > userspace application to handle it. > > > Bjørn From bjorn at mork.no Tue Oct 28 02:13:07 2014 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Tue, 28 Oct 2014 10:13:07 +0100 Subject: Ethernet-briding a MBIM-interface In-Reply-To: <20141028082642.BA4124004E@mail.lysator.liu.se> (Markus Gothe's message of "Tue, 28 Oct 2014 09:26:40 +0100") References: <20141028082642.BA4124004E@mail.lysator.liu.se> Message-ID: <87oasw36cc.fsf@nemi.mork.no> Markus Gothe writes: > Explains why the fix_dst-stuff breaks briding in qmi_wwan.c as well I > guess. However even if setting a null mac there it seems to be > replaced by the modem fw, right? No, you're right. If we hit the code dealing with raw IP packets in qmi_wwan, then the h_source of the genereated header will be 0. Which makes those packets fail in a brigde. That should also be fixed. > Sure I can do ARP-replies with ebtables / arptables as well I > suppose. :-) ARP is all about preferences. Yes. Bjørn From nietzsche at lysator.liu.se Tue Oct 28 12:10:50 2014 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Tue, 28 Oct 2014 20:10:50 +0100 Subject: Ethernet-briding a MBIM-interface Message-ID: <20141028191053.1274740006@mail.lysator.liu.se> I can conclude this by saying that using a non-null unique source and destination mac + briding rules and ARP magic solved it all for a simple Huawei IPv4-only stick... So yes the root cause is the null mac. I will try to modify usbnet.h to get a decent patch for upstream. //M Skickat ifrån min mobil. Den 28 okt 2014 10:13 skrev =?ISO-8859-1?Q?Bj=F8rn_Mork?= : > > Markus Gothe writes: > > > Explains why the fix_dst-stuff breaks briding in qmi_wwan.c as well I > > guess. However even if setting a null mac there it seems to be > > replaced by the modem fw, right? > > > No, you're right.  If we hit the code dealing with raw IP packets in > qmi_wwan, then the h_source of the genereated header will be 0. Which > makes those packets fail in a brigde. > > That should also be fixed. > > > Sure I can do ARP-replies with ebtables / arptables as well I > > suppose. :-) ARP is all about preferences. > > Yes. > > > Bjørn