[PATCH i-g-t v3 0/7] benchmarks/kms_fb_stress: Test all possible combinations of formats

[Arthur Grillo]

On 20/03/24 12:42, Pekka Paalanen wrote:
> On Thu, 07 Mar 2024 16:41:02 -0300
> Arthur Grillo <arthurgrillo at riseup.net> wrote:
> 
>> After some discussion[1], I changed how to tackle the problem.
>>
>> With the upcoming addition of YUV format to VKMS[2] it would be great to
>> test the their performance. So, test all possible combinations of
>> formats.
> 
> Hi Arthur,
> 
> what do you gain by testing all possible combinations of formats on
> planes?
> 
> I would have thought that if you can pick and choose a format, then you
> don't need to test combinations. Each format alone (but as plane source
> and writeback destination separately) would exercise all the VKMS code
> there is for that format, given that VKMS uses the intermediate pixel
> format for all blending etc. operations.

Yeah, that does make a lot more sense. I think I was overcomplicating
things, sorry XD. About "but as plane source and writeback destination
separately", I didn't quite understand that part, could you talk more
about it?

> 
> 
> What will the standard benchmark results printout look like?
> 
> I imagine it should be a simple table that people could put in their
> commit message, something like this:
> 
> kms_fb_stress test results, plane source:
> 
>             before    after    difference
>   ARGB8888   3.6 s    3.7 s       +2.7 %
>   ABGR8888   3.6 s    3.8 s       +5.6 %
>   YUV420    15.5 s    9.9 s        -36 %
>   YUV410         -    8.8 s          -
> 
> and another table for writeback destination, if that's interesting.
> 
> But that requires both before and after runs, so one would need to save
> results in a file, and use a tool to generate the table. Well, maybe
> something to consider in the future. One could also get even fancier
> with statistical analysis, but I guess this would be plenty already.

Interesting ideas, maybe, for now, just the format and the time it took
run would suffice? What do you think?

Best Regards,
~Arthur Grillo

> 
> 
> Thanks,
> pq
> 
>> Also, to focus in an combination subset, add command-line options to
>> arbitrarily select the format of each plane and writeback.
>>
>> - PATCH #1: Just a non-related assert that I found needed while developing
>> - PATCH #2: Log the KMS structure
>> - PATCH #3: Moving code, should be noop
>> - PATCH #4 & #5: Fix a couple of bugs found while developing patch #6
>> - PATCH #6: Test all possible combinations
>> - PATCH #7: Add command-line options
>>
>> [1]: https://lore.kernel.org/all/6979cd2e-2b00-4dc4-8e41-66b435cf7ea8@riseup.net/
>> [2]: https://lore.kernel.org/all/20240304-yuv-v4-11-76beac8e9793@bootlin.com/
>>
>> Signed-off-by: Arthur Grillo <arthurgrillo at riseup.net>
>> ---
>> Changes in v3:
>> - Make the help message and commit message more clear on how the
>>   command-line options work (Louis Chauvet)
>> - Link to v2: https://lore.kernel.org/r/20240305-kms_fb_stress-dev-v2-0-78888a38e563@riseup.net
>>
>> Changes in v2:
>> - Change how the command-line options work, now they pin the specific
>>   format and let the other change.
>> - Add a patch to test all the possible formats combination
>> - Add a couple of patches to fix some bugs
>> - Move the KMS logging patch to the beginning of the series
>> - Move the default_kms definition to the static struct data_t variable
>> - Link to v1: https://lore.kernel.org/r/20240226-kms_fb_stress-dev-v1-0-1c14942b1244@riseup.net
>>
>> ---
>> Arthur Grillo (7):
>>       benchmarks/kms_fb_stress: Assert that we have an supported pipe
>>       benchmarks/kms_fb_stress: Log the KMS structure
>>       benchmarks/kms_fb_stress: Move the stress procedure to its own function
>>       benchmarks/kms_fb_stress: Free resources on the stress procedure
>>       benchmarks/kms_fb_stress: Don't paint the FB's if the format is DRM_FORMAT_RGB565
>>       benchmarks/kms_fb_stress: Test every possible format combinations
>>       benchmarks/kms_fb_stress: Add command line options to pin the planes or writeback formats
>>
>>  benchmarks/kms_fb_stress.c | 342 ++++++++++++++++++++++++++++++++++++---------
>>  1 file changed, 273 insertions(+), 69 deletions(-)
>> ---
>> base-commit: 32369affc1fdb8194c4e7274d1d4dfd74f4aae88
>> change-id: 20240226-kms_fb_stress-dev-c93ae176f901
>>
>> Best regards,
>