EOL in sight for loext: namespace (?) / Smoke Tests & CodeCognita of LO Code - earlier: Re: Code Coverage LibreOffice

Sun Feb 7 22:40:46 UTC 2021

Hej,

Today at the FOSDEM <https://fosdem.org/2021/>, I did two lightning talks
(5min) on ODF Features in the LibreOffice developers room
<https://fosdem.org/2021/schedule/track/libreoffice/>.

   1. ODF Feature priority handling of the ODF TC
   <https://youtu.be/TjjhaF3rK3I>
   I would suggest in the future a priority handling for new ODF Features
   from ODF applications and create right away an ODF community specification
   to allow vendors to already use OASIS ODF namespaces instead
   intermediate ones. This would require a faster (semi-automated) creation of
   our specification as we started already on GitHub:
   http://github.com/oasis-tcs/odf-tc
   2. ODF Feature testing <https://youtu.be/GmKT91fkc1k>
   I would suggest to to use coverage tests of ODF Feature to create a
   'Code Cognita' (a map of the source codes, where features are being
   implemented).
   Especially, this approach is applicable to any ODF applications (e.g.
   Gnumeric & MS Office).
   ODF Features are likely to become the new denominator.

These are just drafts of ideas and quite far from being professional (kids
at home & too little time), but perhaps you are able to connect the dots...
;-)

Cheers,
Svante

Am Di., 3. Nov. 2020 um 13:11 Uhr schrieb Svante Schubert <
svante.schubert at gmail.com>:

> Dear LO developers,
>
> A simple/naive idea:
> Loading ODT files into LO while doing a coverage test.
> The source code coverage DIFF will reveal the source code being used
> exclusively for the ODT feature DIFF of the two documents.
>
> For example:
>
>    1. Load a very simple test document, like a single paragraph with
>    "Hello World" with LibreOffice and during load doing a code coverage test
>    to identify all the source code, which is being used for loading the ODF
>    test document.
>    2. Enhance the simple test document by a single feature, like making
>    the text bold and do the loading & code coverage again.
>    3. The difference (DIFF) between the two source code coverages
>    represents source code exclusively relevant to the feature bold.
>    4. Future commits changing source code of such an "ODF feature area"
>    would trigger the load/save roundtrip test of these special ODF test
>    document(s).
>    *The advantage is to have a relevant minimum regression test not
>    taking long to execute.*
>
> I already interviewed Maarten, yesterday, who wrote some years ago scripts
> to run the coverage tests on LO, but they seem not to be executed for a
> long time:
> https://dev-builds.libreoffice.org/lcov_reports/master-2015-06-10_23.49.39/
>
> Is there some LO build VM available that can be extended to reactivate the
> code coverage tests, so interested parties might follow this approach?
> Especially, I am longing for a "black box script" where I can give a text
> file with a list of ODF files as an input parameter to be code-cover-tested
> and receive for each input ODF the code coverage as a text file. Is there
> any script that can e used?
> In post-processing, the source code DIFF would than be created afterwards.
> Shouldn't be too difficult (famous last words)... ;-)
>
> Is anyone interested to join this adventure? :-)
>
> Cheers,
> Svante
>
> ---------- Forwarded message ---------
> Von: Maarten Hoes <hoes.maarten at gmail.com>
> Date: Mo., 2. Nov. 2020 um 22:35 Uhr
> Subject: Re: Code Coverage LibreOffice
> To: Svante Schubert <svante.schubert at gmail.com>
> Cc: Michael Stahl <mst at libreoffice.org>
>
>
> Hi Svante/Michael,
>
>
> I'm glad to be of help, if I can be. :)
>
> First of all, a disclaimer: I'm not a developer. I can write Unix/Linux
> shell scripts, and used to be a professional Unix/Linux system
> administrator, but I couldn't write a 'hello world' program in C or C++ if
> my life depended on it.
>
> Perhaps I should start with some basics. When talking about 'code
> coverage', people generally mean 'measure what source code gets executed'
> (generally by the execution of a test suite), so that you can see how much
> and what parts of the source code gets covered by the test suite.
>
> There is a wiki page [1] that describes how to set up lcov/gcov on Linux
> for the LibreOffice codebase, but I can't tell how accurate or up to date
> that wiki page currently is. Years ago, I wrote a shell script to automate
> that [2] process. If I recall correctly, it used to be run automatically
> (daily ?) by Jenkins, but the host/vm the job ran on became unavailable
> ages ago, and no-one seemed to notice or care enough to restore it. Old
> examples of what the generated reports look like can be found on the
> dev-builds.libreoffice.org website [3].
>
> The process basically looks somewhat like this:
>
> - Compile your code using additional flags, so that a record will be kept
> of what code gets executed.
> - Run the test suite (or whatever else of which you want to see what
> source code gets executed by running it).
> - Generate a human readable report that shows what sourcecode got executed.
>
> Now, on to your specific questions:
>
> 1. Yes, lcov/gcov code coverage will show you all the source code lines
> that get executed by loading your 'Hello World' test document.
> 2. Yes, loading the same document multiple times will execute the exact
> same source code every time you do it, provided you don't change any of the
> conditions (like changing the source code in between runs).
> 3. Yes, I believe that if you take your 'Hello World' test document, and
> make that text bold, then the difference between the two runs should be the
> additional code that gets executed by making the text bold.
>
> One thing to note though, as it may not be immediately obvious: When you
> start LibreOffice, and open a document, a lot more code will get executed
> than 'just' the code needed to display the text in the document. All of
> this additional code will get marked as having been executed in the report.
> I know of no way around that. So I am assUming (but you know what happens
> when we assUme) that you need to be intimately familiar with the entire
> LibreOffice codebase in order to differentiate between 'this is code that
> gets executed on startup, this is code needed for the GUI, this is code
> needed for parsing the document' and 'this is the code for the bold feature
> I want to test'. So I'm not so sure if this gets you specifically what you
> are looking for. (PS: There is a way to filter out / omit source code that
> you are not interested in in the final report, but this filtering is done
> on a per directory-basis. So you can for example filter out
> 'src/but/not/this/directory', but then you need to be very sure up-front
> that the code you are looking for is not contained in that directory. This
> also won't help you if you have 'code you are interested in' and 'code not
> interested in' that are both located in the same directory).
>
>
> I hope this may assist to get you started. As I may have raised more
> questions than I attempted to answer, please feel free to let me know, and
> I will try my best to be of further assistance.
>
>
>
> - Maarten
>
>
>
> [1]
> https://wiki.documentfoundation.org/Development/Lcov
>
> [2]
> https://git.libreoffice.org/buildbot/+/refs/heads/master/lcov-report/
>
> [3]
> https://dev-builds.libreoffice.org/lcov_reports/
>
>
>
>
>
> On Mon, Nov 2, 2020 at 8:55 PM Svante Schubert <svante.schubert at gmail.com>
> wrote:
>
>> Hi Maarten,
>>
>> Michael Stahl - who is as me a TDF member - dropped your name after the
>> OASIS ODF TC call, in which we are both participating.
>> I was curious about LO Code Coverage, but I am more working on ODF than
>> on LibreOffice itself, for instance, the
>> https://github.com/tdf/odftoolkit
>> There is a little experiment I have in mind, my hope is that you might
>> get curious enough to join it ;-)
>>
>> The simple idea:
>>
>>    1. Load a very simple test document, like a single paragraph with
>>    "Hello World" with LibreOffice and during load doing a code coverage of all
>>    the source code being used for loading the test document.
>>    2. Load the simple test document multiple times, is it always the
>>    same code coverage? (As I said I am new to LO Code and do not know the
>>    result)
>>    3. If it is the same (or changes within a predictable range), enhance
>>    the simple test document by a single feature, like making the text bold and
>>    do the loading & code coverage again. Is the difference between the two
>>    coverages the feature bold?
>>
>> The question is: Is it possible to create an ODF feature <=> source code
>> relation by doing the above?
>> The advantage, if certain parts of source code are being changed, we have
>> better insights on the ODF Testdocuments we are in need.
>>
>> The reason why I am asking is that the ODF Toolkit has a library ODFDOM,
>> which is able to transform an ODT document
>> <https://github.com/tdf/odftoolkit/blob/master/docs/docs/presentations/character-styles.odt>
>> into an equivalent list of user changes (in JSON)
>> <https://github.com/tdf/odftoolkit/blob/master/docs/docs/presentations/character-styles.json>
>> as if the document was created from top to bottom. By this, we are able to
>> define more precisely the feature delta of the coverage scenario.
>>
>> In the upcoming year, I am trying to do several things:
>>
>>    1. Add to the OASIS ODF specification these user changes (you may
>>    call them as well "ODF user features" or "ODF user semantics" or simply
>>    "ODF feature")
>>    In addition, to add a semantic dictionary.
>>    2. Refactor the work on the ODFDOM of ODF Toolkit...(many details)...
>>
>> I would be very curious to set-up an environment, which I can use as a
>> black box, providing a document as input and receive the code coverage as
>> output!🤗
>> Would you be so kind to assist me in that, Maarten? As I have frankly
>> little idea where to start ;-)
>>
>> Best regards,
>> Svante
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice/attachments/20210207/a57fe347/attachment.htm>