Idea to have qmicli output "parseable" data

[Aleksander Morgado]

Hey,

>
> I have had this exact same question before. There is a MR https://gitlab.freedesktop.org/mobile-broadband/libqmi/-/merge_requests/190 that does exactly this. I have not made contributions to it so I don't want to take credit for it but I'm quite excited for when it is merged!
>
> Yes, I was going to point to that merge request. The auto-generation
> of JSON data for all data types is already available in that MR, but
> it still requires to go over all qmicli commands one by one adding a
> few lines to be able to print the JSON output. The MR contains support
> for just a few commands, the ones that the company that hired me to do
> that needed :)
>
>
> Thanks, this looks excellent!
>
> I wonder if the error reporting is worth merging separately? This was one of the oddities I had noticed that slightly complicated parsing
>

Error reports are given in TLVs as any other TLV in the response
message. I think that would be the best thing to do also in the JSON
output, so that the parsers can expect this TLV to exist ALWAYS in all
response messages:

  "result": {
    "error-status": "0",
    "error-code": "0"
  },


> However, as you have done some work, can I ask what your opinion is on some of the more complicated outputs? I don't have the code open, but several commands seemed to be doing significant manipulation of the output structure while printing it. These seemed to be the complex ones to have to convert really?

Look at the NAS Get Cell Location Info example here, that is very
complex already:
https://gitlab.freedesktop.org/mobile-broadband/libqmi/-/merge_requests/190#note_739649

For some of the messages, it's true that qmicli also does some magic
here and there, but I believe printing the raw contents of the
response in JSON isn't a bad initial approach.

>
> Also, inserting lots of " if (qmicli_output_json ()) {" feels like a lot of typing...? Can we do better?
>

Heh. I'm sure we can. For mmcli's output in ModemManager, I did a full
refactor on how everything is printed so that the same code printing
the human pretty output can also print the JSON or key-value pair
outputs without many changes in the generic code. We could do
something similar here, I'm open to suggestions really.

> I wonder if we can't go a further up and consider the individual commands to be knocking some output map alike structure into shape, then the output is just a structure we either output as text, or output as json as we prefer? I wonder at the moment if the JSON and test outputs would be so similar?

That's basically what we did in ModemManager, yes, see
https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/blob/master/cli/mmcli-output.c
Even the "human pretty output" is generated by the output engine.

>
>
> How did you propose to handle the cases where there is both a value for a key and a sub value? For example in "--nas-get-serving-system" we have
>
>     Radio interfaces: '1'
>         [0]: 'lte'
>
> What I did was to store the '1" in a "_value" key under the "Radio interfaces" key
>

Really, I don't remember what was done for every type. What I did was
to improve the qmi-codegen output to also support generating the JSON
output, it's very much the same as the printable message outputs we
have when traces are enabled in libqmi, the very same idea. Arrays are
converted into JSON arrays, complex types are converted into JSON
structs, and so on.

> I also provided some heuristics to parse keys like:
>
>     3GPP daylight saving time adjustment: '1' hours
>
> into ['1', 'hours']
>

That would be too much IMO. The original idea was that the JSON output
in libqmi should print raw values as received from the modem. But
then, a lot of fields in QMI messages have "additional" formatting
required, as e.g. with the signal quality values in several messages.
For example, the SNR for LTE in "NAS Get Signal Info" is not given in
dB directly, you need to divide the given value by 10 in order to get
the dBs. What should the JSON output report in this case? the original
value or the one already processed to get dBs? It's tricky, and it's
really a lot of work, but it may be a good idea to give the processed
values also in the JSON output, for simplicity (i.e. we do it once in
libqmi, instead of N times by N different users). My original
implementation would need to be changed a lot to support that I'm
afraid.

> However, there is a certain lack of consistency here in the output, so it only "mostly works"
>
>
> Anyway, exciting that there is interest. I think it's worth kicking around the exact steps as it's quite a lot of typing to get this done. Any observations?

All this is a lot of work, help would be greatly appreciated :)

-- 
Aleksander
https://aleksander.es