From aleksander at lanedo.com Mon May 7 05:47:00 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Mon, 07 May 2012 14:47:00 +0200 Subject: SYNC indications In-Reply-To: <87mx5xo59g.fsf@nemi.mork.no> References: <4F9A6423.3000003@lanedo.com> <87mx5xo59g.fsf@nemi.mork.no> Message-ID: <4FA7C444.1090402@lanedo.com> >> Does anyone know how to properly process SYNC indications in the CTL >> service? I tried to reply with a SYNC request for each indication I get, >> and the device replies with a proper SYNC-response, but it seems it >> doesn't prevent receiving more of such indications. Any idea what the >> SYNC indications are for in the first place? > > I think I once found that sending SYNC will reset all allocated Client > IDs. Which makes it useful if you have lost track of client state and > want to ensure that it is clean, but not the thing you would want to do > when receiving a SYNC indication. Needs verification... > Verified. I temporarily modified qmicli not to perform the CID release before exiting, and after 5-6 CID allocations, the UML290 started to complain: $> sudo qmicli -d /dev/cdc-wdm1 --wds-get-packet-service-status error: couldn't create WDS client: CID allocation failed in the CTL client: CID allocation reply parsing failed: QMI protocol error (5): 'client-ids-exhausted' $> sudo qmicli -d /dev/cdc-wdm1 --wds-get-packet-service-status error: couldn't create WDS client: CID allocation failed in the CTL client: CID allocation reply parsing failed: QMI protocol error (5): 'client-ids-exhausted' ... If after that I make sure to send a SYNC request as soon as the device gets open (with the new --device-open-sync switch in qmicli), the problem disappears: $> sudo qmicli -d /dev/cdc-wdm1 --wds-get-packet-service-status --device-open-sync [/dev/cdc-wdm1] Connection status: 'disconnected' I would think of always running the SYNC as soon as the QmiDevice gets open, but that doesn't work in my internal Gobi2k, which doesn't reply to the SYNC at all :-/ Cheers, -- Aleksander From bjorn at mork.no Fri May 11 13:02:18 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Fri, 11 May 2012 22:02:18 +0200 Subject: libqmi-glib In-Reply-To: <4F953BF1.6080609@lanedo.com> (Aleksander Morgado's message of "Mon, 23 Apr 2012 13:24:33 +0200") References: <4F953BF1.6080609@lanedo.com> Message-ID: <87havm1o6t.fsf@nemi.mork.no> Hello, it's fun to see the progress of this library. I've actually been using the qmicli utility to manage my connection the last few days, resulting in a few more bugs found in the cdc-wdm driver (like poll going crazy if you unplug the USB device). Anyway, today I noticed more features added. Unfortunately one of them is not going to work: You allow the client to exit after starting a connection, releasing the WDS client ID. That's not supported. You have to keep the client ID while the connection is up. Once the client ID is released, the connection goes down. (You may ask what the point of the handle is then.... I have no idea, but I assume that doing a stop with the handle and then release the ID is "cleaner" than just releasing the ID) Bj?rn From bjorn at mork.no Fri May 11 13:06:10 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Fri, 11 May 2012 22:06:10 +0200 Subject: libqmi-glib In-Reply-To: <87havm1o6t.fsf@nemi.mork.no> (=?utf-8?Q?=22Bj=C3=B8rn?= Mork"'s message of "Fri, 11 May 2012 22:02:18 +0200") References: <4F953BF1.6080609@lanedo.com> <87havm1o6t.fsf@nemi.mork.no> Message-ID: <87d36a1o0d.fsf@nemi.mork.no> Bj?rn Mork writes: > Anyway, today I noticed more features added. Unfortunately one of them > is not going to work: You allow the client to exit after starting a > connection, releasing the WDS client ID. That's not supported. You > have to keep the client ID while the connection is up. Once the client > ID is released, the connection goes down. And then I noticed the --client-no-release-cid option. I'm more than a bit slow. That will make it work. Probably best to bundle it with --wds-start-network, but then again I guess this is a test utility for people supposed to know what they're doing. I.e. not really intented for me :-) Bj?rn From ivoronin at gmail.com Sat May 12 14:35:33 2012 From: ivoronin at gmail.com (Ilya Voronin) Date: Sun, 13 May 2012 01:35:33 +0400 Subject: Wireshark Dissector for QMI Protocol Message-ID: I was playing with QMI-capable devices a lot the last few days and created a dissector for Wireshark, that helps me to monitor Windows-based connection manager running in VirtualBox and my own app. I'm sure you will find it useful too: https://gist.github.com/2641557 From aleksander at lanedo.com Sun May 13 03:23:18 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Sun, 13 May 2012 12:23:18 +0200 Subject: libqmi-glib In-Reply-To: <87d36a1o0d.fsf@nemi.mork.no> References: <4F953BF1.6080609@lanedo.com> <87havm1o6t.fsf@nemi.mork.no> <87d36a1o0d.fsf@nemi.mork.no> Message-ID: <4FAF8B96.5080505@lanedo.com> Hey hey, > >> Anyway, today I noticed more features added. Unfortunately one of them >> is not going to work: You allow the client to exit after starting a >> connection, releasing the WDS client ID. That's not supported. You >> have to keep the client ID while the connection is up. Once the client >> ID is released, the connection goes down. > > And then I noticed the --client-no-release-cid option. I'm more than a > bit slow. > Yes, that's the option to avoid releasing the CID when qmicli exits; then you can use --client-cid=X to recover it the next time you call qmicli. > That will make it work. Probably best to bundle it with --wds-start-network, > but then again I guess this is a test utility for people supposed to > know what they're doing. > If you run --wds-start-network WITH --wds-follow-network; the program will keep running with the connection open until you do Ctrl+C. In that way, you don't need to play with --client-no-release-cid, --client-cid and --wds-stop-network, as that is done for you automatically. If you still want the qmicli to exit, while keeping the CID unreleased and all that, try to use the new `qmi-network' script, where you can just: $> qmi-network /dev/cdc-wdm0 start $> qmi-network /dev/cdc-wdm0 status $> qmi-network /dev/cdc-wdm0 stop That will keep the packet data handle and CID in a state file under /tmp; quite similar to what you did in your script. Note that that script is still quite untested, I wrote it last Friday. One last thing; if you ever get the 'exhausted client IDs' error, you can just run qmicli with --device-open-sync, which will run the SYNC request as a step while opening the QmiDevice. Cheers! -- Aleksander From bjorn at mork.no Sun May 13 10:27:12 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Sun, 13 May 2012 19:27:12 +0200 Subject: libqmi-glib In-Reply-To: <4FAF8B96.5080505@lanedo.com> (Aleksander Morgado's message of "Sun, 13 May 2012 12:23:18 +0200") References: <4F953BF1.6080609@lanedo.com> <87havm1o6t.fsf@nemi.mork.no> <87d36a1o0d.fsf@nemi.mork.no> <4FAF8B96.5080505@lanedo.com> Message-ID: <87d368yosv.fsf@nemi.mork.no> Aleksander Morgado writes: > Hey hey, > >> >>> Anyway, today I noticed more features added. Unfortunately one of them >>> is not going to work: You allow the client to exit after starting a >>> connection, releasing the WDS client ID. That's not supported. You >>> have to keep the client ID while the connection is up. Once the client >>> ID is released, the connection goes down. >> >> And then I noticed the --client-no-release-cid option. I'm more than a >> bit slow. >> > > Yes, that's the option to avoid releasing the CID when qmicli exits; > then you can use --client-cid=X to recover it the next time you call qmicli. Yes, I noticed. >> That will make it work. Probably best to bundle it with --wds-start-network, >> but then again I guess this is a test utility for people supposed to >> know what they're doing. >> > > If you run --wds-start-network WITH --wds-follow-network; the program > will keep running with the connection open until you do Ctrl+C. In that > way, you don't need to play with --client-no-release-cid, --client-cid > and --wds-stop-network, as that is done for you automatically. Yes, that's how I have been using it until now. Works really well, and you get the indications as well. And I found the poll error in the driver because the file was open when my laptop was suspended ;-) > If you still want the qmicli to exit, while keeping the CID unreleased > and all that, try to use the new `qmi-network' script, where you can just: > $> qmi-network /dev/cdc-wdm0 start > $> qmi-network /dev/cdc-wdm0 status > $> qmi-network /dev/cdc-wdm0 stop > > That will keep the packet data handle and CID in a state file under > /tmp; quite similar to what you did in your script. Note that that > script is still quite untested, I wrote it last Friday. Will try. Note that I don't necessarily believe what I did necessarily is a good solution. I was concerned with integrating the script into my local Debian networking config, and saving the state like that was a quick and dirty way to achieve that. I don't think it fits most users requirements. But having something like this for testing is certainly useful. > One last thing; if you ever get the 'exhausted client IDs' error, you > can just run qmicli with --device-open-sync, which will run the SYNC > request as a step while opening the QmiDevice. Yes, I know. BTW, there was a request for a quick way to detect QMI devices on linux-usb today, and I revisited the idea about a libusb based method. I took a quick look at qmi-device.{c,h} with the intent to try to make something, but found that it would take me more time than I had, given my severely limited knowledge of C and glib... So I coded another quick and dirty hack instead. In perl. Attaching it, as it still may be useful to someone here too. A libusb based device type for libqmi-glib would be a much nicer solution, though. But something like that will never be anything but a test utility (since there won't be any network device) so I guess there isn't much point in putting a lot of work into it. The perl script may suffice. Bj?rn -------------- next part -------------- A non-text attachment was scrubbed... Name: qmiver.pl Type: text/x-perl Size: 6047 bytes Desc: not available URL: From bjorn at mork.no Tue May 15 01:12:12 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Tue, 15 May 2012 10:12:12 +0200 Subject: Wireshark Dissector for QMI Protocol In-Reply-To: (Ilya Voronin's message of "Sun, 13 May 2012 01:35:33 +0400") References: Message-ID: <87zk99c17n.fsf@nemi.mork.no> Ilya Voronin writes: > I was playing with QMI-capable devices a lot the last few days and > created a dissector for Wireshark, > that helps me to monitor Windows-based connection manager running in > VirtualBox and my own app. > I'm sure you will find it useful too: https://gist.github.com/2641557 I'm sure I will. That's really nice. Great idea! For anyone not having looked at this yet: Ilya uses the existing usbmon capabilities of wireshark, so this dissector can be used regardless of how you communicate with your QMI device. cdc-wdm, qcqmi (Gobi driver), or virtual machine passthrough - anything can be snooped. Would be even nicer though if there was some way to (semi-)automatically generate the message and TLV tables from existing data in e.g. libqmi, to avoid having yet another place where these must be entered manually... Bj?rn From aleksander at lanedo.com Tue May 15 12:10:02 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Tue, 15 May 2012 21:10:02 +0200 Subject: Wireshark Dissector for QMI Protocol In-Reply-To: <87zk99c17n.fsf@nemi.mork.no> References: <87zk99c17n.fsf@nemi.mork.no> Message-ID: <4FB2AA0A.50208@lanedo.com> >> I was playing with QMI-capable devices a lot the last few days and >> created a dissector for Wireshark, >> that helps me to monitor Windows-based connection manager running in >> VirtualBox and my own app. >> I'm sure you will find it useful too: https://gist.github.com/2641557 > > I'm sure I will. That's really nice. Great idea! > > For anyone not having looked at this yet: Ilya uses the existing usbmon > capabilities of wireshark, so this dissector can be used regardless of > how you communicate with your QMI device. cdc-wdm, qcqmi (Gobi driver), > or virtual machine passthrough - anything can be snooped. > That is super cool; I've written protocol dissectors for Wireshark in the past myself, I just didn't know you could also use it for USB. > Would be even nicer though if there was some way to (semi-)automatically > generate the message and TLV tables from existing data in e.g. libqmi, > to avoid having yet another place where these must be entered > manually... > Dan's `qmidb' in the upstream repo tries to create packed structs for specific messages directly from the Gobi database files; so that could help. I've also been playing around with something related but a bit different; which is defining the messages myself manually in a JSON file (see [1]) and then autogenerating the message creator/parser code directly from there with some python-based codegen (see [2]). The 'qmi-codegen' branch in my libqmi-glib tree does all this (actually, it still fails to generate something compilable... :-). Anyway, this approach tries to handle additional things than just reading every possible TLV from a specific QMI message, like (a) the case where some TLVs are given only if some other TLV exists and has a specific value (e.g. 'call end reason' TLV is only given in the 'wds start network' response if the response status was FAILURE and the error code was 'call failed'); or (b), the case where some given TLV has one meaning or another based on a previous TLV (e.g. 3gpp or 3gpp2 specific values on the same TLV, based on a previous TLV which defined whether it is 3gpp or 3gpp2). But what I think we all agree is that libqmi should really expose not only a client-based high-level API, it should also include the low-level message creation/parsing API as well, which this dissector could use to get the message contents translated. [1] https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json [2] https://gitorious.org/lanedo/libqmi-glib/trees/qmi-codegen/build-aux/qmi-codegen -- Aleksander From nietzsche at lysator.liu.se Tue May 15 13:29:05 2012 From: nietzsche at lysator.liu.se (Markus Gothe) Date: Tue, 15 May 2012 22:29:05 +0200 Subject: Wireshark Dissector for QMI Protocol In-Reply-To: <4FB2AA0A.50208@lanedo.com> References: <87zk99c17n.fsf@nemi.mork.no> <4FB2AA0A.50208@lanedo.com> Message-ID: <2CCCE79E-3684-4A12-9B4F-40B81987AF15@lysator.liu.se> Hi, I found this one very interesting... Didn't knew you could dissectors in LUA for Wireshark these days. However I didn't get it working with Wireshark 1.4.x... :-/ Just segfaulting in tshark on Ubuntu. The 'qmidb' repo...? Is it anything interesting that's not available from CodeAurora (yes, the C++ code they released is horrible ;-) ) Best, //Markus - The panama-hat hacker On May 15, 2012, at 21:10, Aleksander Morgado wrote: > >>> I was playing with QMI-capable devices a lot the last few days and >>> created a dissector for Wireshark, >>> that helps me to monitor Windows-based connection manager running in >>> VirtualBox and my own app. >>> I'm sure you will find it useful too: https://gist.github.com/2641557 >> >> I'm sure I will. That's really nice. Great idea! >> >> For anyone not having looked at this yet: Ilya uses the existing usbmon >> capabilities of wireshark, so this dissector can be used regardless of >> how you communicate with your QMI device. cdc-wdm, qcqmi (Gobi driver), >> or virtual machine passthrough - anything can be snooped. >> > > That is super cool; I've written protocol dissectors for Wireshark in > the past myself, I just didn't know you could also use it for USB. > >> Would be even nicer though if there was some way to (semi-)automatically >> generate the message and TLV tables from existing data in e.g. libqmi, >> to avoid having yet another place where these must be entered >> manually... >> > > > Dan's `qmidb' in the upstream repo tries to create packed structs for > specific messages directly from the Gobi database files; so that could help. > > I've also been playing around with something related but a bit > different; which is defining the messages myself manually in a JSON file > (see [1]) and then autogenerating the message creator/parser code > directly from there with some python-based codegen (see [2]). The > 'qmi-codegen' branch in my libqmi-glib tree does all this (actually, it > still fails to generate something compilable... :-). Anyway, this > approach tries to handle additional things than just reading every > possible TLV from a specific QMI message, like (a) the case where some > TLVs are given only if some other TLV exists and has a specific value > (e.g. 'call end reason' TLV is only given in the 'wds start network' > response if the response status was FAILURE and the error code was 'call > failed'); or (b), the case where some given TLV has one meaning or > another based on a previous TLV (e.g. 3gpp or 3gpp2 specific values on > the same TLV, based on a previous TLV which defined whether it is 3gpp > or 3gpp2). > > But what I think we all agree is that libqmi should really expose not > only a client-based high-level API, it should also include the low-level > message creation/parsing API as well, which this dissector could use to > get the message contents translated. > > > > [1] > https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json > [2] > https://gitorious.org/lanedo/libqmi-glib/trees/qmi-codegen/build-aux/qmi-codegen > > > -- > Aleksander > _______________________________________________ > libqmi-devel mailing list > libqmi-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/libqmi-devel From bjorn at mork.no Wed May 16 01:16:13 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Wed, 16 May 2012 10:16:13 +0200 Subject: Wireshark Dissector for QMI Protocol In-Reply-To: <4FB2AA0A.50208@lanedo.com> (Aleksander Morgado's message of "Tue, 15 May 2012 21:10:02 +0200") References: <87zk99c17n.fsf@nemi.mork.no> <4FB2AA0A.50208@lanedo.com> Message-ID: <877gwcbkxe.fsf@nemi.mork.no> Aleksander Morgado writes: > But what I think we all agree is that libqmi should really expose not > only a client-based high-level API, it should also include the low-level > message creation/parsing API as well, which this dissector could use to > get the message contents translated. Yup. I had a brief thought about "dictionaries" as a result of the Wireshark dissector (since I use RADIUS a lot and enjoy the usage of standard FreeRADIUS dictionaries with the Wireshark RADIUS dissector), but concluded that the protocol it too complex to make that fly. JSON might do it. But there are lots of protocol complexities of the type "if (higher-level-condition-x) then everything-is-completely-different" Like the completely unmotivated transaction ID size for QMI_CTL. Or formats for the *same* system/msgid/tlv depending on whether it's a request, reply or indication. Or the required set of TLVs depending on some outside factor, like which radio system we happen to be connected to. And many of the TLVs are also unnessarily complex IMHO, packing a number of different settings into a single TLV. That's only a struct typedef, but if the number of them are expanded by system/msg/msgtype/tlv, then it is going to be messy. It's not as bad as that of course, but still... But your JSON looks like it could handle most of this. Not too sure about the "mandatory" TLV flag. The answer might be "depends" in a lot of cases. Bj?rn From aleksander at lanedo.com Wed May 16 01:29:08 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Wed, 16 May 2012 10:29:08 +0200 Subject: Wireshark Dissector for QMI Protocol In-Reply-To: <877gwcbkxe.fsf@nemi.mork.no> References: <87zk99c17n.fsf@nemi.mork.no> <4FB2AA0A.50208@lanedo.com> <877gwcbkxe.fsf@nemi.mork.no> Message-ID: <4FB36554.4040603@lanedo.com> >> But what I think we all agree is that libqmi should really expose not >> only a client-based high-level API, it should also include the low-level >> message creation/parsing API as well, which this dissector could use to >> get the message contents translated. > > Yup. I had a brief thought about "dictionaries" as a result of the > Wireshark dissector (since I use RADIUS a lot and enjoy the usage of > standard FreeRADIUS dictionaries with the Wireshark RADIUS dissector), > but concluded that the protocol it too complex to make that fly. > > JSON might do it. But there are lots of protocol complexities of the > type > "if (higher-level-condition-x) then everything-is-completely-different" > > Like the completely unmotivated transaction ID size for QMI_CTL. Or > formats for the *same* system/msgid/tlv depending on whether it's a > request, reply or indication. Or the required set of TLVs depending on > some outside factor, like which radio system we happen to be connected > to. The different size of TID in QMI_CTL is just handled directly in the generated code, no extra rule setup just for that. For the remaining 'rules', I was hoping to be able to use different prerequisites; like if a given TLV has a different meaning depending on a previous TLV; I can always write the TLV twice (same ID) with different struct members *and* different/opposite prerequisites. The JSON will start to get complex in that case anyway. > > And many of the TLVs are also unnessarily complex IMHO, packing a number > of different settings into a single TLV. That's only a struct typedef, > but if the number of them are expanded by system/msg/msgtype/tlv, then > it is going to be messy. It's not as bad as that of course, but > still... > Yes, they really try to make it super complex. One of my assumptions, which I hope will always be valid, is that already defined TLVs won't change in future releases. Adding new TLVs to existing messages or full new messages (e.g. Get Bearer Data Technology & Get Current Bearer Data Technology) shouldn't be a problem. > But your JSON looks like it could handle most of this. Not too sure > about the "mandatory" TLV flag. The answer might be "depends" in a lot > of cases. > Well, a key with prerequisites is mandatory only if the prerequisites are satisfied; so yes, actually depends. The only mandatory key ever is the response status TLV in the response messages. Anyway, this codegen is just a quick try of what could be done, it may end up going nowhere. -- Aleksander From aleksander at lanedo.com Wed May 16 23:23:59 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Thu, 17 May 2012 08:23:59 +0200 Subject: libqmi-glib In-Reply-To: <87d368yosv.fsf@nemi.mork.no> References: <4F953BF1.6080609@lanedo.com> <87havm1o6t.fsf@nemi.mork.no> <87d36a1o0d.fsf@nemi.mork.no> <4FAF8B96.5080505@lanedo.com> <87d368yosv.fsf@nemi.mork.no> Message-ID: <4FB4997F.6020809@lanedo.com> On 05/13/2012 07:27 PM, Bj?rn Mork wrote: > BTW, there was a request for a quick way to detect QMI devices on > linux-usb today, and I revisited the idea about a libusb based method. > I took a quick look at qmi-device.{c,h} with the intent to try to make > something, but found that it would take me more time than I had, given > my severely limited knowledge of C and glib... So I coded another quick > and dirty hack instead. In perl. Attaching it, as it still may be > useful to someone here too. > > A libusb based device type for libqmi-glib would be a much nicer > solution, though. But something like that will never be anything but a > test utility (since there won't be any network device) so I guess there > isn't much point in putting a lot of work into it. The perl script may > suffice. I'm not sure I fully get why libusb would be a good thing to have when detecting a QMI device. Is it just to completely avoid any kernel driver handling the device and talking directly to it with raw USB transfers? -- Aleksander From bjorn at mork.no Thu May 17 03:17:57 2012 From: bjorn at mork.no (=?ISO-8859-1?Q?Bj=F8rn_Mork?=) Date: Thu, 17 May 2012 12:17:57 +0200 Subject: libqmi-glib In-Reply-To: <4FB4997F.6020809@lanedo.com> References: <4F953BF1.6080609@lanedo.com> <87havm1o6t.fsf@nemi.mork.no> <87d36a1o0d.fsf@nemi.mork.no> <4FAF8B96.5080505@lanedo.com> <87d368yosv.fsf@nemi.mork.no> <4FB4997F.6020809@lanedo.com> Message-ID: <7321ac69-7a62-48a3-8c43-902920e980f6@email.android.com> Aleksander Morgado wrote: >I'm not sure I fully get why libusb would be a good thing to have when >detecting a QMI device. Is it just to completely avoid any kernel >driver >handling the device and talking directly to it with raw USB transfers? Yes. It is completely useless for already supported devices. But verifying new devices is currently a lot of hassle due to missing support for new_id in qmi_wwan (generic problem with usbnet drivers), and the strict probe in cdc-wdm (will refuse any interface with more than 1 endpoint). A libusb based tool allow users to quickly test all interfaces on a new and unsupported device without having to patch and build a new driver first, but that is all it is useful for. So it might not be anything for libqmi-glib. The perl script will do after some refinement. Getting proper new_id support in qmi_wwan would be a better solution, and would probably benefit other usbnet drivers too. But I still have not found an acceptable solution to that problem. Bj?rn From tschaefer at t-online.de Sat May 19 01:41:33 2012 From: tschaefer at t-online.de (Thomas =?iso-8859-1?q?Sch=E4fer?=) Date: Sat, 19 May 2012 10:41:33 +0200 Subject: compile-problem Message-ID: <201205191041.34377.tschaefer@t-online.de> Hello, while testing K5005-surfstick I tried to use the newest libqmi-cli. After checking out, ./autogen.sh and make I have got the following message: thomas at hpmini:~/lib-qmi/libqmi-glib> LANG=C make make all-recursive make[1]: Entering directory `/home/thomas/lib-qmi/libqmi-glib' Making all in . make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib' make[2]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib' Making all in build-aux make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib/build-aux' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib/build-aux' Making all in src make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib/src' CC libqmi_glib_la-qmi-message-ctl.lo qmi-message-ctl.c: In function 'qmi_message_ctl_version_info_reply_parse': qmi-message-ctl.c:181:60: error: format '%u' expects argument of type 'unsigned int', but argument 6 has type 'long unsigned int' [-Werror=format] cc1: all warnings being treated as errors make[2]: *** [libqmi_glib_la-qmi-message-ctl.lo] Error 1 make[2]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib' make: *** [all] Error 2 thomas at hpmini:~/lib-qmi/libqmi-glib> After removing all Werror (ignoring warnings, not treating as erroe) in the makefiles it compiles and works. Regards, Thomas From aleksander at lanedo.com Sat May 19 03:47:02 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Sat, 19 May 2012 12:47:02 +0200 Subject: compile-problem In-Reply-To: <201205191041.34377.tschaefer@t-online.de> References: <201205191041.34377.tschaefer@t-online.de> Message-ID: <4FB77A26.2060003@lanedo.com> On 19/05/12 10:41, Thomas Sch?fer wrote: > Hello, > > > while testing K5005-surfstick I tried to use the newest libqmi-cli. > > After checking out, ./autogen.sh and make I have got the following message: > > thomas at hpmini:~/lib-qmi/libqmi-glib> LANG=C make > make all-recursive > make[1]: Entering directory `/home/thomas/lib-qmi/libqmi-glib' > Making all in . > make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib' > make[2]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib' > Making all in build-aux > make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib/build-aux' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/home/thomas/lib-qmi/libqmi-glib/build-aux' > Making all in src > make[2]: Entering directory `/home/thomas/lib-qmi/libqmi-glib/src' > CC libqmi_glib_la-qmi-message-ctl.lo > qmi-message-ctl.c: In function 'qmi_message_ctl_version_info_reply_parse': > qmi-message-ctl.c:181:60: error: format '%u' expects argument of type > 'unsigned int', but argument 6 has type 'long unsigned int' [-Werror=format] > cc1: all warnings being treated as errors > Fixed and pushed, thanks. -- Aleksander From aleksander at lanedo.com Wed May 23 10:49:52 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Wed, 23 May 2012 19:49:52 +0200 Subject: qmi-codegen in libqmi-glib Message-ID: <4FBD2340.6050706@lanedo.com> Hey hey, I've been playing a bit more with the idea of automatically generating code for message handling from input JSON files, and got already something working in the 'qmi-codegen' branch of my libqmi-glib tree: git://gitorious.org/lanedo/libqmi-glib.git The current set of supported messages are defined in the following JSON files: https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json And adding new messages should need, ideally, just updating these JSON files. I guess the qmi-codegen will need updates from time to time when new basic types are found in the QMI messages. Currently the following types are supported: * integers (uint8/uint16/uint32/int8/int16/int32) * strings * structs with integer contents * arrays of structs The new qmi-codegen python program will read the per-service input JSON files, and for each request/response, we will define GIO Async methods, like: void qmi_client_ctl_allocate_cid ( QmiClientCtl *self, QmiMessageCtlAllocateCidInput *input, guint timeout, GCancellable *cancellable, GAsyncReadyCallback callback, gpointer user_data); QmiMessageCtlAllocateCidOutput *qmi_client_ctl_allocate_cid_finish ( QmiClientCtl *self, GAsyncResult *res, GError **error); We then provide methods to create/ref/unref the QmiMessageCtlAllocateCidInput bundle, as well as methods to get/set specific TLVs, like: gboolean qmi_message_ctl_allocate_cid_input_get_service ( QmiMessageCtlAllocateCidInput *self, QmiService *value, GError **error); gboolean qmi_message_ctl_allocate_cid_input_set_service ( QmiMessageCtlAllocateCidInput *self, QmiService value, GError **error); QmiMessageCtlAllocateCidInput *qmi_message_ctl_allocate_cid_input_ref (QmiMessageCtlAllocateCidInput *self); void qmi_message_ctl_allocate_cid_input_unref (QmiMessageCtlAllocateCidInput *self); QmiMessageCtlAllocateCidInput *qmi_message_ctl_allocate_cid_input_new (void); And the same for output bundles like the QmiMessageCtlAllocateCidOutput: typedef struct _QmiMessageCtlAllocateCidOutputAllocationInfo { QmiService service; guint8 cid; } QmiMessageCtlAllocateCidOutputAllocationInfo; gboolean qmi_message_ctl_allocate_cid_output_get_result ( QmiMessageCtlAllocateCidOutput *self, GError **error); gboolean qmi_message_ctl_allocate_cid_output_get_allocation_info ( QmiMessageCtlAllocateCidOutput *self, QmiMessageCtlAllocateCidOutputAllocationInfo *value, GError **error); QmiMessageCtlAllocateCidOutput *qmi_message_ctl_allocate_cid_output_ref (QmiMessageCtlAllocateCidOutput *self); void qmi_message_ctl_allocate_cid_output_unref (QmiMessageCtlAllocateCidOutput *self); In order to keep API/ABI compatibility whenever the QMI protocol is updated, all input TLVs and output TLVs are packed into the input and output 'bundles'. Here I'm assuming that QMI protocol updates are either new messages or full new TLVs in already existing messages; or in other words, assuming that existing TLVs are not supposed to get modified (which is why for struct TLVs we define the struct itself in the public interface). When compiling with -DMESSAGE_ENABLE_TRACE, it enables tracing of the ongoing messages, which now also includes properly printing the TLV values (when known), like: <<<<<< QMUX: <<<<<< length = 16 <<<<<< flags = 0x00 <<<<<< service = "ctl" <<<<<< client = 0 <<<<<< QMI: <<<<<< flags = "none" <<<<<< transaction = 3 <<<<<< tlv_length = 5 <<<<<< message = "Release CID" (0x0023) <<<<<< TLV: <<<<<< type = "Release Info" (0x01) <<<<<< length = 2 <<<<<< value = 02:13 <<<<<< translated = <<<<<< [Service = 2] <<<<<< [Cid = 19] Tests with this branch are highly appreciated, as I would like to merge it to libqmi-glib master sooner than later. Cheers! -- Aleksander From dcbw at redhat.com Wed May 23 12:22:14 2012 From: dcbw at redhat.com (Dan Williams) Date: Wed, 23 May 2012 14:22:14 -0500 Subject: qmi-codegen in libqmi-glib In-Reply-To: <4FBD2340.6050706@lanedo.com> References: <4FBD2340.6050706@lanedo.com> Message-ID: <1337800934.25721.14.camel@dcbw.foobar.com> On Wed, 2012-05-23 at 19:49 +0200, Aleksander Morgado wrote: > Hey hey, > > I've been playing a bit more with the idea of automatically generating > code for message handling from input JSON files, and got already > something working in the 'qmi-codegen' branch of my libqmi-glib tree: > git://gitorious.org/lanedo/libqmi-glib.git > > The current set of supported messages are defined in the following JSON > files: > > https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json > > https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json > > https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json > > https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json > > And adding new messages should need, ideally, just updating these JSON > files. I guess the qmi-codegen will need updates from time to time when > new basic types are found in the QMI messages. Currently the following > types are supported: > * integers (uint8/uint16/uint32/int8/int16/int32) > * strings > * structs with integer contents > * arrays of structs At some point we'll need unions too, where the member of the union that's actually used depends on a specific value from earlier in the struct. That's how stuff like "get the access technology and status" works where you've got a big struct like: struct { bool 3gpp; // true = 3gpp, false = 3gpp2 union { struct { uint mcc; uint mnc; uint lac; uint cid; enum rat_type; } 3gpp; struct { bool is_1x_or_hdr; union { struct { 1x junk; } 1x; struct { evdo junk; } hdr; } u2; } 3gpp2; } u; } that's some evil stuff, but basically normal protocol packet parsing :( Anyway, I do like the idea of doing this with JSON. I despair of actually being able to use the Gobi database except for reference since generating the code out of them is horrendously complicated. Dan > > The new qmi-codegen python program will read the per-service input JSON > files, and for each request/response, we will define GIO Async methods, > like: > > void qmi_client_ctl_allocate_cid ( > QmiClientCtl *self, > QmiMessageCtlAllocateCidInput *input, > guint timeout, > GCancellable *cancellable, > GAsyncReadyCallback callback, > gpointer user_data); > QmiMessageCtlAllocateCidOutput *qmi_client_ctl_allocate_cid_finish ( > QmiClientCtl *self, > GAsyncResult *res, > GError **error); > > We then provide methods to create/ref/unref the > QmiMessageCtlAllocateCidInput bundle, as well as methods to get/set > specific TLVs, like: > > gboolean qmi_message_ctl_allocate_cid_input_get_service ( > QmiMessageCtlAllocateCidInput *self, > QmiService *value, > GError **error); > gboolean qmi_message_ctl_allocate_cid_input_set_service ( > QmiMessageCtlAllocateCidInput *self, > QmiService value, > GError **error); > QmiMessageCtlAllocateCidInput *qmi_message_ctl_allocate_cid_input_ref > (QmiMessageCtlAllocateCidInput *self); > void qmi_message_ctl_allocate_cid_input_unref > (QmiMessageCtlAllocateCidInput *self); > QmiMessageCtlAllocateCidInput *qmi_message_ctl_allocate_cid_input_new > (void); > > And the same for output bundles like the QmiMessageCtlAllocateCidOutput: > > typedef struct _QmiMessageCtlAllocateCidOutputAllocationInfo { > QmiService service; > guint8 cid; > } QmiMessageCtlAllocateCidOutputAllocationInfo; > > gboolean qmi_message_ctl_allocate_cid_output_get_result ( > QmiMessageCtlAllocateCidOutput *self, > GError **error); > gboolean qmi_message_ctl_allocate_cid_output_get_allocation_info ( > QmiMessageCtlAllocateCidOutput *self, > QmiMessageCtlAllocateCidOutputAllocationInfo *value, > GError **error); > QmiMessageCtlAllocateCidOutput *qmi_message_ctl_allocate_cid_output_ref > (QmiMessageCtlAllocateCidOutput *self); > void qmi_message_ctl_allocate_cid_output_unref > (QmiMessageCtlAllocateCidOutput *self); > > > In order to keep API/ABI compatibility whenever the QMI protocol is > updated, all input TLVs and output TLVs are packed into the input and > output 'bundles'. Here I'm assuming that QMI protocol updates are either > new messages or full new TLVs in already existing messages; or in other > words, assuming that existing TLVs are not supposed to get modified > (which is why for struct TLVs we define the struct itself in the public > interface). > > When compiling with -DMESSAGE_ENABLE_TRACE, it enables tracing of the > ongoing messages, which now also includes properly printing the TLV > values (when known), like: > > <<<<<< QMUX: > <<<<<< length = 16 > <<<<<< flags = 0x00 > <<<<<< service = "ctl" > <<<<<< client = 0 > <<<<<< QMI: > <<<<<< flags = "none" > <<<<<< transaction = 3 > <<<<<< tlv_length = 5 > <<<<<< message = "Release CID" (0x0023) > <<<<<< TLV: > <<<<<< type = "Release Info" (0x01) > <<<<<< length = 2 > <<<<<< value = 02:13 > <<<<<< translated = > <<<<<< [Service = 2] > <<<<<< [Cid = 19] > > > Tests with this branch are highly appreciated, as I would like to merge > it to libqmi-glib master sooner than later. Neato. Dan From bjorn at mork.no Wed May 23 12:55:30 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Wed, 23 May 2012 21:55:30 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <4FBD2340.6050706@lanedo.com> (Aleksander Morgado's message of "Wed, 23 May 2012 19:49:52 +0200") References: <4FBD2340.6050706@lanedo.com> Message-ID: <87mx4yr7t9.fsf@nemi.mork.no> Aleksander Morgado writes: > Hey hey, > > I've been playing a bit more with the idea of automatically generating > code for message handling from input JSON files, and got already > something working in the 'qmi-codegen' branch of my libqmi-glib tree: > git://gitorious.org/lanedo/libqmi-glib.git Minor build error: qmi-message.c: In function 'qmi_message_get_tlv_printable': qmi-message.c:672:34: error: format '%u' expects argument of type 'unsigned int', but argument 6 has type 'gsize' [-Werror=format] Bj?rn From bjorn at mork.no Wed May 23 13:12:53 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Wed, 23 May 2012 22:12:53 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <87mx4yr7t9.fsf@nemi.mork.no> (=?utf-8?Q?=22Bj=C3=B8rn?= Mork"'s message of "Wed, 23 May 2012 21:55:30 +0200") References: <4FBD2340.6050706@lanedo.com> <87mx4yr7t9.fsf@nemi.mork.no> Message-ID: <87ipfmr70a.fsf@nemi.mork.no> Bj?rn Mork writes: > Aleksander Morgado writes: > >> Hey hey, >> >> I've been playing a bit more with the idea of automatically generating >> code for message handling from input JSON files, and got already >> something working in the 'qmi-codegen' branch of my libqmi-glib tree: >> git://gitorious.org/lanedo/libqmi-glib.git > > Minor build error: > > qmi-message.c: In function 'qmi_message_get_tlv_printable': > qmi-message.c:672:34: error: format '%u' expects argument of type 'unsigned int', but argument 6 has type 'gsize' [-Werror=format] More of those in the generated code. How about bjorn at nemi:/usr/local/src/git/libqmi-glib$ git diff diff --git a/build-aux/qmi-codegen/Message.py b/build-aux/qmi-codegen/Message.py index c05640c..422c96c 100644 --- a/build-aux/qmi-codegen/Message.py +++ b/build-aux/qmi-codegen/Message.py @@ -312,7 +312,7 @@ class Message: ' g_string_append_printf (ctx->printable,\n' ' "%sTLV:\\n"\n' ' "%s type = \\"%s\\" (0x%02x)\\n"\n' - ' "%s length = %u\\n"\n' + ' "%s length = %zu\\n"\n' ' "%s value = %s\\n"\n' ' "%s translated = %s\\n",\n' ' ctx->line_prefix,\n' diff --git a/src/qmi-message.c b/src/qmi-message.c index f9b1f15..58a3668 100644 --- a/src/qmi-message.c +++ b/src/qmi-message.c @@ -664,7 +664,7 @@ qmi_message_get_tlv_printable (QmiMessage *self, value_hex = qmi_utils_str_hex (value, length, ':'); printable = g_strdup_printf ("%sTLV:\n" "%s type = 0x%02x\n" - "%s length = %u\n" + "%s length = %zu\n" "%s value = %s\n", line_prefix, line_prefix, type, ? Bj?rn From aleksander at lanedo.com Wed May 23 13:16:46 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Wed, 23 May 2012 22:16:46 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <87ipfmr70a.fsf@nemi.mork.no> References: <4FBD2340.6050706@lanedo.com> <87mx4yr7t9.fsf@nemi.mork.no> <87ipfmr70a.fsf@nemi.mork.no> Message-ID: <4FBD45AE.50403@lanedo.com> On 05/23/2012 10:12 PM, Bj?rn Mork wrote: > Bj?rn Mork writes: > >> Aleksander Morgado writes: >> >>> Hey hey, >>> >>> I've been playing a bit more with the idea of automatically generating >>> code for message handling from input JSON files, and got already >>> something working in the 'qmi-codegen' branch of my libqmi-glib tree: >>> git://gitorious.org/lanedo/libqmi-glib.git >> >> Minor build error: >> >> qmi-message.c: In function 'qmi_message_get_tlv_printable': >> qmi-message.c:672:34: error: format '%u' expects argument of type 'unsigned int', but argument 6 has type 'gsize' [-Werror=format] > > More of those in the generated code. How about > Just fixed and pushed both of these things, using G_GSIZE_FORMAT. Thanks! -- Aleksander From aleksander at lanedo.com Wed May 23 13:21:24 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Wed, 23 May 2012 22:21:24 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <1337800934.25721.14.camel@dcbw.foobar.com> References: <4FBD2340.6050706@lanedo.com> <1337800934.25721.14.camel@dcbw.foobar.com> Message-ID: <4FBD46C4.7090603@lanedo.com> >> >> I've been playing a bit more with the idea of automatically generating >> code for message handling from input JSON files, and got already >> something working in the 'qmi-codegen' branch of my libqmi-glib tree: >> git://gitorious.org/lanedo/libqmi-glib.git >> >> The current set of supported messages are defined in the following JSON >> files: >> >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json >> >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json >> >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json >> >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json >> >> And adding new messages should need, ideally, just updating these JSON >> files. I guess the qmi-codegen will need updates from time to time when >> new basic types are found in the QMI messages. Currently the following >> types are supported: >> * integers (uint8/uint16/uint32/int8/int16/int32) >> * strings >> * structs with integer contents >> * arrays of structs > > At some point we'll need unions too, where the member of the union > that's actually used depends on a specific value from earlier in the > struct. That's how stuff like "get the access technology and status" > works where you've got a big struct like: > > struct { > bool 3gpp; // true = 3gpp, false = 3gpp2 > union { > struct { > uint mcc; > uint mnc; > uint lac; > uint cid; > enum rat_type; > } 3gpp; > struct { > bool is_1x_or_hdr; > union { > struct { > 1x junk; > } 1x; > struct { > evdo junk; > } hdr; > } u2; > } 3gpp2; > } u; > } > > that's some evil stuff, but basically normal protocol packet parsing :( > Anyway, I do like the idea of doing this with JSON. I despair of > actually being able to use the Gobi database except for reference since > generating the code out of them is horrendously complicated. I'll check that message during the next days and also try to provide codegen for it. Still have other fixes in mind for it, but at least as it is now it should be equivalent feature-wise to the master branch. -- Aleksander From dcbw at redhat.com Wed May 23 13:53:59 2012 From: dcbw at redhat.com (Dan Williams) Date: Wed, 23 May 2012 15:53:59 -0500 Subject: qmi-codegen in libqmi-glib In-Reply-To: <4FBD46C4.7090603@lanedo.com> References: <4FBD2340.6050706@lanedo.com> <1337800934.25721.14.camel@dcbw.foobar.com> <4FBD46C4.7090603@lanedo.com> Message-ID: <1337806439.25721.35.camel@dcbw.foobar.com> On Wed, 2012-05-23 at 22:21 +0200, Aleksander Morgado wrote: > >> > >> I've been playing a bit more with the idea of automatically generating > >> code for message handling from input JSON files, and got already > >> something working in the 'qmi-codegen' branch of my libqmi-glib tree: > >> git://gitorious.org/lanedo/libqmi-glib.git > >> > >> The current set of supported messages are defined in the following JSON > >> files: > >> > >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json > >> > >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json > >> > >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json > >> > >> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json > >> > >> And adding new messages should need, ideally, just updating these JSON > >> files. I guess the qmi-codegen will need updates from time to time when > >> new basic types are found in the QMI messages. Currently the following > >> types are supported: > >> * integers (uint8/uint16/uint32/int8/int16/int32) > >> * strings > >> * structs with integer contents > >> * arrays of structs > > > > At some point we'll need unions too, where the member of the union > > that's actually used depends on a specific value from earlier in the > > struct. That's how stuff like "get the access technology and status" > > works where you've got a big struct like: > > > > struct { > > bool 3gpp; // true = 3gpp, false = 3gpp2 > > union { > > struct { > > uint mcc; > > uint mnc; > > uint lac; > > uint cid; > > enum rat_type; > > } 3gpp; > > struct { > > bool is_1x_or_hdr; > > union { > > struct { > > 1x junk; > > } 1x; > > struct { > > evdo junk; > > } hdr; > > } u2; > > } 3gpp2; > > } u; > > } > > > > that's some evil stuff, but basically normal protocol packet parsing :( > > Anyway, I do like the idea of doing this with JSON. I despair of > > actually being able to use the Gobi database except for reference since > > generating the code out of them is horrendously complicated. > > > I'll check that message during the next days and also try to provide > codegen for it. > > Still have other fixes in mind for it, but at least as it is now it > should be equivalent feature-wise to the master branch. eg sWDSEventReportIndication_DataSystems in GobiConnectionMgmtAPIStructs.h. That's basically a worst-case scenario, but unfortunately one that is kinda important. The bitfields and padding there just make it worse, though ideally we can reduce the bitfields to flag enums or something. That struct isn't quite as bad as it looks, since in reality all of sNetworkTypeIs1 can be represented by: enum CdmaType { /* 1x, evdo0, evdoA, and evdoB are mutually exclusive, * but eHRPD could be set when any of evdo0, evdoA, * or evdoB are set. */ 1x = 0x01; evdo0 = 0x02; evdoA = 0x04; evdoB = 0x08; eHRPD = 0x10; }; enum Cdma1xInfo { Cdma1x_is95 = 0x01; Cdma1x_is2000 = 0x02; Cdma1x_is2000RelA = 0x04; }; enum Evdo0Info { Evdo0_dpa = 0x01; }; enum EvdoAInfo { EvdoA_dpa = 0x01; EvdoA_mfpa = 0x02; EvdoA_empa = 0x04; EvdoA_empa_ehrpd = 0x08; }; enum EvdoBInfo { EvdoB_dpa = 0x01; EvdoB_mfpa = 0x02; EvdoB_empa = 0x04; EvdoB_empa_ehrpd = 0x08; EvdoB_mmpa_ehrpd = 0x10; EvdoB_mmpa_ehrpd = 0x20; }; struct sNetworkTypeIs1 { enum:uint8 CdmaType cdma_type; uint8 padding1[2]; uint8 null_bearer; // 0x80 = TRUE, 0x0 = FALSE union { enum:uint8 Cdma1xInfo 1xinfo; enum:uint8 Evdo0Info e0info; enum:uint8 EvdoAInfo eAinfo; enum:uint8 EvdoBInfo eBinfo; }; uint8 padding2[3]; }; Dan From aleksander at lanedo.com Thu May 24 00:01:46 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Thu, 24 May 2012 09:01:46 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <1337806439.25721.35.camel@dcbw.foobar.com> References: <4FBD2340.6050706@lanedo.com> <1337800934.25721.14.camel@dcbw.foobar.com> <4FBD46C4.7090603@lanedo.com> <1337806439.25721.35.camel@dcbw.foobar.com> Message-ID: <4FBDDCDA.50701@lanedo.com> On 05/23/2012 10:53 PM, Dan Williams wrote: > On Wed, 2012-05-23 at 22:21 +0200, Aleksander Morgado wrote: >>>> >>>> I've been playing a bit more with the idea of automatically generating >>>> code for message handling from input JSON files, and got already >>>> something working in the 'qmi-codegen' branch of my libqmi-glib tree: >>>> git://gitorious.org/lanedo/libqmi-glib.git >>>> >>>> The current set of supported messages are defined in the following JSON >>>> files: >>>> >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json >>>> >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json >>>> >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json >>>> >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json >>>> >>>> And adding new messages should need, ideally, just updating these JSON >>>> files. I guess the qmi-codegen will need updates from time to time when >>>> new basic types are found in the QMI messages. Currently the following >>>> types are supported: >>>> * integers (uint8/uint16/uint32/int8/int16/int32) >>>> * strings >>>> * structs with integer contents >>>> * arrays of structs >>> >>> At some point we'll need unions too, where the member of the union >>> that's actually used depends on a specific value from earlier in the >>> struct. That's how stuff like "get the access technology and status" >>> works where you've got a big struct like: >>> >>> struct { >>> bool 3gpp; // true = 3gpp, false = 3gpp2 >>> union { >>> struct { >>> uint mcc; >>> uint mnc; >>> uint lac; >>> uint cid; >>> enum rat_type; >>> } 3gpp; >>> struct { >>> bool is_1x_or_hdr; >>> union { >>> struct { >>> 1x junk; >>> } 1x; >>> struct { >>> evdo junk; >>> } hdr; >>> } u2; >>> } 3gpp2; >>> } u; >>> } >>> >>> that's some evil stuff, but basically normal protocol packet parsing :( >>> Anyway, I do like the idea of doing this with JSON. I despair of >>> actually being able to use the Gobi database except for reference since >>> generating the code out of them is horrendously complicated. >> >> >> I'll check that message during the next days and also try to provide >> codegen for it. >> >> Still have other fixes in mind for it, but at least as it is now it >> should be equivalent feature-wise to the master branch. > > eg sWDSEventReportIndication_DataSystems in > GobiConnectionMgmtAPIStructs.h. That's basically a worst-case scenario, > but unfortunately one that is kinda important. The bitfields and > padding there just make it worse, though ideally we can reduce the > bitfields to flag enums or something. That struct isn't quite as bad as > it looks, since in reality all of sNetworkTypeIs1 can be represented by: > > enum CdmaType { > /* 1x, evdo0, evdoA, and evdoB are mutually exclusive, > * but eHRPD could be set when any of evdo0, evdoA, > * or evdoB are set. > */ > 1x = 0x01; > evdo0 = 0x02; > evdoA = 0x04; > evdoB = 0x08; > eHRPD = 0x10; > }; > > enum Cdma1xInfo { > Cdma1x_is95 = 0x01; > Cdma1x_is2000 = 0x02; > Cdma1x_is2000RelA = 0x04; > }; > > enum Evdo0Info { > Evdo0_dpa = 0x01; > }; > > enum EvdoAInfo { > EvdoA_dpa = 0x01; > EvdoA_mfpa = 0x02; > EvdoA_empa = 0x04; > EvdoA_empa_ehrpd = 0x08; > }; > > enum EvdoBInfo { > EvdoB_dpa = 0x01; > EvdoB_mfpa = 0x02; > EvdoB_empa = 0x04; > EvdoB_empa_ehrpd = 0x08; > EvdoB_mmpa_ehrpd = 0x10; > EvdoB_mmpa_ehrpd = 0x20; > }; > > struct sNetworkTypeIs1 { > enum:uint8 CdmaType cdma_type; > uint8 padding1[2]; > uint8 null_bearer; // 0x80 = TRUE, 0x0 = FALSE > union { > enum:uint8 Cdma1xInfo 1xinfo; > enum:uint8 Evdo0Info e0info; > enum:uint8 EvdoAInfo eAinfo; > enum:uint8 EvdoBInfo eBinfo; > }; > uint8 padding2[3]; > }; > I already handled something like that in the current code, the result for the "Get Current Data Bearer Technology". Instead of the real enums being specified in the last union, I just used a generic guint32 'so_mask'. Then, each enum/flags defined has proper documentation on where to use it, like: /** * QmiWdsSoCdma1x: * @QMI_WDS_SO_CDMA1X_NONE: Unknown, to be ignored. * @QMI_WDS_SO_CDMA1X_IS95: IS95. * @QMI_WDS_SO_CDMA1X_IS2000: IS2000. * @QMI_WDS_SO_CDMA1X_IS2000_REL_A: IS2000 RelA. * * Flags specifying the Service Option when the bearer network type is * @QMI_WDS_NETWORK_TYPE_3GPP2 and when the Radio Access Technology mask * contains @QMI_WDS_RAT_3GPP2_CDMA1X. */ typedef enum { QMI_WDS_SO_CDMA1X_NONE = 0, QMI_WDS_SO_CDMA1X_IS95 = 1 << 0, QMI_WDS_SO_CDMA1X_IS2000 = 1 << 1, QMI_WDS_SO_CDMA1X_IS2000_REL_A = 1 << 2 } QmiWdsSoCdma1x; The user of the library *needs* to know when the so_mask contains a specific set of flags. The user needs to know it both in this case, where you get a generic mask, in order to know which set of flags to use; but it also needs to know it if we provide a union of different fields, in order to know which field of the union is the appropriate one. Aren't then the unions (at least in this case) just an additional complication that we can just completely skip? -- Aleksander From dcbw at redhat.com Thu May 24 09:16:28 2012 From: dcbw at redhat.com (Dan Williams) Date: Thu, 24 May 2012 11:16:28 -0500 Subject: qmi-codegen in libqmi-glib In-Reply-To: <4FBDDCDA.50701@lanedo.com> References: <4FBD2340.6050706@lanedo.com> <1337800934.25721.14.camel@dcbw.foobar.com> <4FBD46C4.7090603@lanedo.com> <1337806439.25721.35.camel@dcbw.foobar.com> <4FBDDCDA.50701@lanedo.com> Message-ID: <1337876188.12378.3.camel@dcbw.foobar.com> On Thu, 2012-05-24 at 09:01 +0200, Aleksander Morgado wrote: > On 05/23/2012 10:53 PM, Dan Williams wrote: > > On Wed, 2012-05-23 at 22:21 +0200, Aleksander Morgado wrote: > >>>> > >>>> I've been playing a bit more with the idea of automatically generating > >>>> code for message handling from input JSON files, and got already > >>>> something working in the 'qmi-codegen' branch of my libqmi-glib tree: > >>>> git://gitorious.org/lanedo/libqmi-glib.git > >>>> > >>>> The current set of supported messages are defined in the following JSON > >>>> files: > >>>> > >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-common.json > >>>> > >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-ctl.json > >>>> > >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-dms.json > >>>> > >>>> https://gitorious.org/lanedo/libqmi-glib/blobs/qmi-codegen/data/qmi-service-wds.json > >>>> > >>>> And adding new messages should need, ideally, just updating these JSON > >>>> files. I guess the qmi-codegen will need updates from time to time when > >>>> new basic types are found in the QMI messages. Currently the following > >>>> types are supported: > >>>> * integers (uint8/uint16/uint32/int8/int16/int32) > >>>> * strings > >>>> * structs with integer contents > >>>> * arrays of structs > >>> > >>> At some point we'll need unions too, where the member of the union > >>> that's actually used depends on a specific value from earlier in the > >>> struct. That's how stuff like "get the access technology and status" > >>> works where you've got a big struct like: > >>> > >>> struct { > >>> bool 3gpp; // true = 3gpp, false = 3gpp2 > >>> union { > >>> struct { > >>> uint mcc; > >>> uint mnc; > >>> uint lac; > >>> uint cid; > >>> enum rat_type; > >>> } 3gpp; > >>> struct { > >>> bool is_1x_or_hdr; > >>> union { > >>> struct { > >>> 1x junk; > >>> } 1x; > >>> struct { > >>> evdo junk; > >>> } hdr; > >>> } u2; > >>> } 3gpp2; > >>> } u; > >>> } > >>> > >>> that's some evil stuff, but basically normal protocol packet parsing :( > >>> Anyway, I do like the idea of doing this with JSON. I despair of > >>> actually being able to use the Gobi database except for reference since > >>> generating the code out of them is horrendously complicated. > >> > >> > >> I'll check that message during the next days and also try to provide > >> codegen for it. > >> > >> Still have other fixes in mind for it, but at least as it is now it > >> should be equivalent feature-wise to the master branch. > > > > eg sWDSEventReportIndication_DataSystems in > > GobiConnectionMgmtAPIStructs.h. That's basically a worst-case scenario, > > but unfortunately one that is kinda important. The bitfields and > > padding there just make it worse, though ideally we can reduce the > > bitfields to flag enums or something. That struct isn't quite as bad as > > it looks, since in reality all of sNetworkTypeIs1 can be represented by: > > > > enum CdmaType { > > /* 1x, evdo0, evdoA, and evdoB are mutually exclusive, > > * but eHRPD could be set when any of evdo0, evdoA, > > * or evdoB are set. > > */ > > 1x = 0x01; > > evdo0 = 0x02; > > evdoA = 0x04; > > evdoB = 0x08; > > eHRPD = 0x10; > > }; > > > > enum Cdma1xInfo { > > Cdma1x_is95 = 0x01; > > Cdma1x_is2000 = 0x02; > > Cdma1x_is2000RelA = 0x04; > > }; > > > > enum Evdo0Info { > > Evdo0_dpa = 0x01; > > }; > > > > enum EvdoAInfo { > > EvdoA_dpa = 0x01; > > EvdoA_mfpa = 0x02; > > EvdoA_empa = 0x04; > > EvdoA_empa_ehrpd = 0x08; > > }; > > > > enum EvdoBInfo { > > EvdoB_dpa = 0x01; > > EvdoB_mfpa = 0x02; > > EvdoB_empa = 0x04; > > EvdoB_empa_ehrpd = 0x08; > > EvdoB_mmpa_ehrpd = 0x10; > > EvdoB_mmpa_ehrpd = 0x20; > > }; > > > > struct sNetworkTypeIs1 { > > enum:uint8 CdmaType cdma_type; > > uint8 padding1[2]; > > uint8 null_bearer; // 0x80 = TRUE, 0x0 = FALSE > > union { > > enum:uint8 Cdma1xInfo 1xinfo; > > enum:uint8 Evdo0Info e0info; > > enum:uint8 EvdoAInfo eAinfo; > > enum:uint8 EvdoBInfo eBinfo; > > }; > > uint8 padding2[3]; > > }; > > > > I already handled something like that in the current code, the result > for the "Get Current Data Bearer Technology". Instead of the real enums > being specified in the last union, I just used a generic guint32 > 'so_mask'. Then, each enum/flags defined has proper documentation on > where to use it, like: > > /** > * QmiWdsSoCdma1x: > * @QMI_WDS_SO_CDMA1X_NONE: Unknown, to be ignored. > * @QMI_WDS_SO_CDMA1X_IS95: IS95. > * @QMI_WDS_SO_CDMA1X_IS2000: IS2000. > * @QMI_WDS_SO_CDMA1X_IS2000_REL_A: IS2000 RelA. > * > * Flags specifying the Service Option when the bearer network type is > * @QMI_WDS_NETWORK_TYPE_3GPP2 and when the Radio Access Technology mask > * contains @QMI_WDS_RAT_3GPP2_CDMA1X. > */ > typedef enum { > QMI_WDS_SO_CDMA1X_NONE = 0, > QMI_WDS_SO_CDMA1X_IS95 = 1 << 0, > QMI_WDS_SO_CDMA1X_IS2000 = 1 << 1, > QMI_WDS_SO_CDMA1X_IS2000_REL_A = 1 << 2 > } QmiWdsSoCdma1x; > > The user of the library *needs* to know when the so_mask contains a > specific set of flags. The user needs to know it both in this case, > where you get a generic mask, in order to know which set of flags to > use; but it also needs to know it if we provide a union of different > fields, in order to know which field of the union is the appropriate > one. Aren't then the unions (at least in this case) just an additional > complication that we can just completely skip? It depends on how complex the structures inside the union are. In this case, the RAT-specific flags (*Info in my example) fit nicely together and don't overflow a uint8. But I'm not sure about others. If it's a more complex structure, obviously there are two ways to go about parsing it: use unions and access the members directly, or start casting void* to the type you want. Unions make the code a bit nicer, at least until you start getting 2 or 3 unions deep like some of this stuff. At that point I've found casting is usually clearer. I've worked with a lot of firmware protocol parsing stuff for kernel wifi drivers, but the protocols there were never as complicated as QMI. Dan From bjorn at mork.no Thu May 24 11:34:32 2012 From: bjorn at mork.no (=?utf-8?Q?Bj=C3=B8rn_Mork?=) Date: Thu, 24 May 2012 20:34:32 +0200 Subject: qmi-codegen in libqmi-glib In-Reply-To: <4FBD2340.6050706@lanedo.com> (Aleksander Morgado's message of "Wed, 23 May 2012 19:49:52 +0200") References: <4FBD2340.6050706@lanedo.com> Message-ID: <87txz5bf7r.fsf@nemi.mork.no> Aleksander Morgado writes: > Tests with this branch are highly appreciated, as I would like to merge > it to libqmi-glib master sooner than later. Have been playing a bit with this branch today, and FWIW it works fine for me. Looking forward to the next step :-) Bj?rn From tschaefer at t-online.de Mon May 28 03:13:59 2012 From: tschaefer at t-online.de (Thomas =?iso-8859-1?q?Sch=E4fer?=) Date: Mon, 28 May 2012 12:13:59 +0200 Subject: utils/qmi-network doc? Message-ID: <201205281213.59437.tschaefer@t-online.de> Hello, Since the support of Huawei E398 and Vodafone K5005 , K3565Z I am using the qmi-ifupdown script by Bjorn. It works well. But I think this was only for testing purposes. libqmi-glib contains utils/qmi-network But I am not able to use/test it. Are there some descriptions or examples available? Regards, Thomas From aleksander at lanedo.com Mon May 28 04:01:05 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Mon, 28 May 2012 13:01:05 +0200 Subject: utils/qmi-network doc? In-Reply-To: <201205281213.59437.tschaefer@t-online.de> References: <201205281213.59437.tschaefer@t-online.de> Message-ID: <4FC35AF1.2090602@lanedo.com> > > Since the support of Huawei E398 and Vodafone K5005 , K3565Z I am using the > qmi-ifupdown script by Bjorn. It works well. But I think this was only for > testing purposes. > > libqmi-glib contains utils/qmi-network > > But I am not able to use/test it. Are there some descriptions or examples > available? > Well, that script is to be considered also for testing purposes, although it can be useful. There's not much to say about it, you just need to run (assuming it's cdc-wdm0 your QMI-speaking port): $> qmi-network /dev/cdc-wdm0 start to issue the QMI network start command or $> qmi-network /dev/cdc-wdm0 stop to issue the QMI network stop command. Once the start command has been issued, you are now able to run a DHCP client on the associated wwan interface, e.g: $> dhclient wwan0 I don't think that the qmi-network script gives you much more (if anything) than Bj?rn's script, but anyway... -- Aleksander From aleksander at lanedo.com Wed May 30 09:08:16 2012 From: aleksander at lanedo.com (Aleksander Morgado) Date: Wed, 30 May 2012 18:08:16 +0200 Subject: New update to the qmi-codegen branch Message-ID: <4FC645F0.4090600@lanedo.com> Hey, Using packed structs to directly cast the TLV value buffer will not always work; e.g. when two variable-length values (like strings) are given in the same TLV. In order to avoid that, and also to simplify the code generation tool, I reworked a bit the way the TLVs are read/written in the QMI messages. Now, each basic type variable (integers, strings) will be read separately, regardless of whether the variable is the only one in the TLV, or part of a sequence of variables in the same TLV, or a member in an array of such variables. This change also makes much easier to work with more complex things like structs with array members. The code is already in the qmi-codegen branch of my libqmi-glib tree, along with some additional DMS-related actions. Cheers! -- Aleksander