[Mesa-ci-intel] [Intel CI]. Questions related to integration code coverage report into the CI and workflow for blacklisted cases

[Denys]

Hello Clayton and Intel team!


We have several points that we'd like to share and discuss with you.

1) Code coverage

Currently, we're working on a code quality improvement by measuring code 
coverage and detecting the parts of the Mesa code which aren't covered 
enough with the tests.

To start with, we've measured code coverage with the tests using only 
one architecture (Kabylake). Though, there are a lot of code paths which 
are architecture specific. So, to get the complete report we need to 
take into account all of them as well as all possible test frameworks used.

The best option here is to use Intel CI for report generating since it 
already runs all the available tests at all processor's architectures.

So, the question is whether it is possible to generate a code coverage 
report using CI or maybe you have any other tools which might help with 
this issue.


2) Blacklist Mesa CI files

If we understand correctly there are the files with the skipped tests 
(the ones which won't be run) in Blacklist Mesa CI files.

If so, we're wondering what is the process of moving the tests out from 
the list? Can we create the corresponding MRs if we confirm the test 
cases became stable and not lead to the problems anymore?

For now, we've tested several test cases for x32 and x64 Mesa versions 
(iris) locally and they look good (they were fixed with a patch below):


# these crash on iris on m64 and fail on m32
piglit.spec.ext_image_dma_buf_import.ext_image_dma_buf_import-sample_nv12
piglit.spec.ext_image_dma_buf_import.ext_image_dma_buf_import-sample_yuv420
piglit.spec.ext_image_dma_buf_import.ext_image_dma_buf_import-sample_yvu420

Fix:
https://gitlab.freedesktop.org/mesa/mesa/commit/4b5e8eb3c8d709bd7c6d1a33a114bf4b002548f8


And these test cases (checked on 19.2.0-devel-21.05-810b95e02c5-DEBUG 
Mesa version x32 and x64, on KBL - both tests are passing):

MESA_LOADER_DRIVER_OVERRIDE=iris ./point-vertex-id gl_VertexID divisor 
-auto -fbo
MESA_LOADER_DRIVER_OVERRIDE=iris ./glslparsertest 
/home/den/repository/piglit/tests/glslparsertest/shaders/CorrectFull.frag 
pass 1.10


To summarize, our questions are:


Is it possible to generate the code coverage report by using Intel CI 
and if it's already available can you please share it with us?
What is the process of moving these tests out of the blacklist?
Can we send the MRs if we confirm that the tests should be moved from 
the blacklist back to the config file?

Looking forward to your prompt response.

Thanks!

-- 
Denys Kostin | QA engineer
GlobalLogic Inc.
S evalle363
www.globallogic.com
  
http://www.globallogic.com/email_disclaimer.txt