Implementing accessibility non-regression check tool

[Samuel Thibault]

Jan-Marek Glogowski, le mer. 29 août 2018 06:53:52 +0000, a ecrit:
> Am August 28, 2018 4:04:37 PM UTC schrieb Samuel Thibault <sthibault at hypra.fr>:
> >Samuel Thibault, le lun. 12 févr. 2018 15:30:59 +0100, a ecrit:
> >> - At some point we'll get confident that we won't introduce other
> >> big classes of warnings over hundreds of .ui files. That's the point
> >> where we can say "ok, let's start fixing the existing issues over
> >> all .ui files once for good". We can then run through .ui files one
> >> by one, fixing the issues and removing the corresponding suppression
> >> lines. These could be used as "easy hacks" entries, they are usually
> >> just a few lines to fix.
> 
> I'm not sure we want this handled as "easy hacks". The goal was to enable the checks always for the build. Is this implemented and can I enable it?

The checks are already enabled, and any new issue will fail the
build.  The goal now is to fix existing issues (currently suppressed by
suppression rules).

> My preferred solution would be, that generating the error files in the current build wouldn't break it, but spill the errors to the console to annoy people, to get this fixed in time. Mind I have no ideas about the amount of output / the current state.

The current amount of warnings is ~2000 lines, so I don't think we want
to just enable it: new warnings (i.e. regressions, which is what we
strictly want to avoid) would be buried in the flow.

> >> The progression of all of this could be monitored with statistics reported e.g. in the minutes of ESC calls.
> So who would be responsible to fix it? If we don't test it on patch submission and break on error, it would just pile up again.

The principle is that the patch submission contains both the fix and
the removal of the suppression rules, i.e. the build breaks if the fix
doesn't actually fix the warnings which used to be suppressed by the
rules.

Of course, review is still needed to check that the .ui change used to
fix the warning does make sense and is not a "quick hack" to get the
warning away.

Samuel