[PATCH i-g-t 1/3] igt_hook: Add feature

[Gustavo Sousa]

Quoting Kamil Konieczny (2024-05-16 13:30:47-03:00)
>Hi Gustavo,
>On 2024-05-16 at 09:19:26 -0300, Gustavo Sousa wrote:
>> Quoting Kamil Konieczny (2024-05-16 07:40:58-03:00)
>> >Hi Gustavo,
>> >On 2024-05-15 at 14:35:55 -0300, Gustavo Sousa wrote:
>> >> Quoting Kamil Konieczny (2024-05-15 14:10:55-03:00)
>> >> >Hi Gustavo,
>> >> >On 2024-05-09 at 12:24:29 -0300, Gustavo Sousa wrote:
>> >> >> For development purposes, sometimes it is useful to have a way of
>> >> >> running custom scripts at certain points of test executions. A
>> >> >> real-world example I bumped into recently is to collect information from
>> >> >> sysfs before and after running each entry of a testlist.
>> >> >> 
>> >> >> While it is possible for the user to handcraft a script that calls each
>> >> >> test with the correct actions before and after execution, we can provide
>> >> >> a better experience by adding built-in support for running hooks during
>> >> >> test execution.
>> >> >> 
>> >> >> That would be even better when adding the same kind of support for
>> >> >> igt_runner (which is done in an upcoming change), since the user can
>> >> >> also nicely resume with igt_resume with the hook already setup in case a
>> >> >> crash happens during execution of the test list.
>> >> >> 
>> >> >> As such provide implement support for hooks, integrate it into
>> >> >> igt_core and expose the functionality via --hook CLI option on test
>> >> >> executables.
>> >> >
>> >> >Hmm, why not just a pre-hook@ and post-hook@ in testlist itself?
>> >> >It will be easier to handle - just more parsing.
>> >> 
>> >> How would that work with respect to filters? The current proposal allows
>> >> something filtering the events to be tracked. For example, one can use
>> >> `--hook "pre-test,pre-dyn-subtest:echo hello"` to run the command only
>> >> before test binary starts and before each dynamic subtest.
>> >> 
>> >> Also, there are cases where a testlist is not really used. Examples are
>> >> calling a test binary directly or calling igt_runner without
>> >> --test-list. So, while I believe we could consider support for
>> >> describing hooks in testlist, I do not think that would be a substitute
>> >> for the --hook option.
>> >> 
>> >> On a personal note, my current use case for hooks is more towards
>> >> debugging, so for me it is more convenient to have a --hook option than
>> >> having to make a copy of a testlist only to add the hook instructions
>> >> there.
>> >> 
>> >> --
>> >> Gustavo Sousa
>> >> 
>> >> >
>> >> >Added Petri to cc.
>> >> >
>> >> >Regards,
>> >> >Kamil
>> >> >
>> >
>> >Ok, that makes sense, I will look into your patches later (maybe next week).
>> 
>> Thanks!
>> 
>> >In meantime please look into GitLab failure here:
>> >https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/58540606
>> 
>> Yeah, already tried to take a look at it a few days ago [1], but as I
>> mentioned there, I'm not sure how I can get more info (e.g. logs) on the CI
>> failure. The test works fine for me locally. I wonder if it is possible
>> to setup the container locally so that I get the same test envionment.
>> 
>> [1] https://lore.kernel.org/all/171527425512.5772.1654105508452706087@gjsousa-mobl2/
>> 
>> --
>> Gustavo Sousa
>> 
>
>Command used is in line 29:
>meson test -C build --num-processes ${FDO_CI_CONCURRENT:-4}
>
>and fail is:
>13/30 lib igt_hook_integration FAIL 0.72 s (killed by signal 11 SIGSEGV)
>
>I tested this locally after building (so binaries are already there)
>using 'meson test -C build' and it fails:
>killed by signal 7 SIGBUS

I'm getting success result on my side:

   $ sudo meson test -C build
   (...extra lines omitted...)
    11/424 lib igt_hook_integration  OK  0.22s
   (...extra lines omitted...)
   Ok:                 420
   Expected Fail:      4
   Fail:               0
   Unexpected Pass:    0
   Skipped:            0
   Timeout:            0

So I wonder if there are environment differences causing the test to
succeed on my side and fail in other places. Is it possible to get the
image of the container used to run the tests in CI?

--
Gustavo Sousa

>
>Regards,
>Kamil
>
>> >
>> >Regards,
>> >Kamil
>> >