[Mesa-dev] [PATCH v3] st/vdpau: change the order in which filters are applied(v3)

[Christian König]

Am 16.08.2016 um 19:50 schrieb Nayan Deshmukh:
> Hi Christian,
>
>
>
> On Tue, Aug 16, 2016 at 11:02 PM, Christian König
> <christian.koenig at amd.com> wrote:
>> Am 16.08.2016 um 19:29 schrieb Nayan Deshmukh:
>>> Hi Andy,
>>>
>>> Thanks for testing.
>>>
>>> On Tue, Aug 16, 2016 at 9:14 PM, Andy Furniss <adf.lists at gmail.com> wrote:
>>>> Nayan Deshmukh wrote:
>>>>> Hi Andy,
>>>>
>>>>> You are right :( It messes up chroma but only in case of sharpen at
>>>>> least for me.
>>>>
>>>> Retested and denoise is also affected for me.
>>>>
>>>>> I tried some changes and the chroma effects are fixed if we apply the
>>>>> sharpness
>>>>> filter to only to the first surface instead of all the surfaces of the
>>>>> buffer.
>>>>> Can you verify this?
>>>>
>>>> If you mean changing surfaces[i] to surfaces[0] below, then it is
>>>> better, but there are still artifacts/chroma errors.
>>>>
>>>> In addition to changing as above, changing sampler_views[0] seems
>>>> to fix denoise and negative sharpen, but gives a new sort of
>>>> artifact for positive sharpen.
>>> Sorry for the misleading language. What I meant was that the filter should
>>> only
>>> be applied to first resource i.e. use only first sampler_view and
>>> surface. As you
>>> replaced i by 0 the filter gets applied multiple times. I tried doing
>>> that and I am
>>> experiencing same problems i.e. artifacts with positive sharpen.
>>>
>>> So can you please it try it again by replacing VL_MAX_SURFACES with 1 in
>>> the
>>> for loop. It should probably fix the problems.
>>
>> Well that would work around the problem, but not fix it.
>>
>> This way you only apply the filters to the luma channel and not all the
>> other ones.
>>
>> The problem is possible that the chroma channels are subsampled and you
>> don't use the correct resolution in the filters for them.
>>
> Yes that's it.  This is what the problem is. But to resolve it we need
> to initialize
> the filters with different resolution. So how can this be coded. I
> will need a bit
> guidance on this one (as I need on most of the things :) ).

You probably need to create multiple instance of the filters, one for 
the original video width/height, then one for width/2 and original 
height and one for width/2 and height/2.

Then check the chroma format of the video buffer and apply accordingly.

Regards,
Christian.

>
> I have this habit of fixing of patches with workarounds, need to change this
> approach.
>
> Regards,
> Nayan.
>> Regards,
>> Christian.
>>
>>
>>> Regards,
>>> Nayan.
>>>>
>>>>>>>> +   for(i = 0; i < VL_MAX_SURFACES; ++i) {
>>>>>>>> +      if(sampler_views[i] != NULL && surfaces[i] != NULL) {
>>>>>>>> +         if (vmixer->noise_reduction.filter)
>>>>>>>> +            vl_median_filter_render(vmixer->noise_reduction.filter,
>>>>>>>> +                                    sampler_views[i], surfaces[i]);
>>>>>>>> +
>>>>>>>> +         if (vmixer->sharpness.filter)
>>>>>>>> +            vl_matrix_filter_render(vmixer->sharpness.filter,
>>>>>>>> +                                    sampler_views[i], surfaces[i]);
>>>>