[Libreoffice-qa] GUI automation

[Michael Stahl]

On 14/03/13 02:05, Markus Mohrhard wrote:
> 
> 2013/3/13 Michael Meeks <michael.meeks at suse.com>:
>>
>> On Wed, 2013-03-13 at 16:17 +0100, Vladimir Benes wrote:

>>> There are several options how to approach this. The first is LDTP [1]
>>> and the other one (in my opinion far better) is Dogtail [2]. If LO uses
>>> standard gtk widgets so they're a11y friendly we can easily access them.
>>> This works for accessing menu items, dialogs and so on.
>>
>>         We used to have a huge chunk of code that did this GUI testing. The
>> general consensus is/was that it is unreliable, and broadly worthless -
>> breaking on minor UI changes, requiring lots of maintenance, incredibly
>> slow to run, etc. We removed it all :-)
> 
> I agree with Michael that the old GUI testing code was horrible and
> unreliable and my big fear is that our horrible accessibility code
> will not make it easier to use existing GUI testing frameworks but I'm
> happy if someone prooves me wrong. I have no idea how IA2 and the new
> widget dialogs play into that. Michael and Caolan may know better
> which consequences they have on our accesibility code.

AFAIK the "old" testautomation/testtool thing used some specific hooks
in the VCL toolkit to do its thing and yes it was notoriously never
reliable.

the A11y interfaces on the other hand seem to me like a good way to do
automated UI tests, and importantly, the A11y stuff is supposed to
_work_ in any case, so any tests using these sound good to me.

>>> Is there anything where we can start or at least help with? What I would
>>> expect is at least some priority one functionality that should be
>>> covered first or so.
> 
> So a few things on my list of untested functionality that is not
> related to GUI testing and has higher priority on my list:
> 
> * Copy & paste
> * Draw is totally untested
> * Improved export tests: optional validation of the exported files (ODF/OOXML)
> * Porting the failing API tests to C++ and re-enable them
> * Exploring and integrating test documents from different providers:
> gnumeric, abiword, calligra, OpenOffice
> * Performance tests

we also have some document print/graphical comparison test prototype
that is not automated yet...

also great to have would be a ODF diff tool so we can do automated
load/save and compare against file stored by reference version (writing
such tool is somewhere on my todo list).