[Mesa-dev] [PATCH v3] st/vdpau: change the order in which filters are applied(v3)

[Christian König]

Am 16.08.2016 um 19:29 schrieb Nayan Deshmukh:
> Hi Andy,
>
> Thanks for testing.
>
> On Tue, Aug 16, 2016 at 9:14 PM, Andy Furniss <adf.lists at gmail.com> wrote:
>> Nayan Deshmukh wrote:
>>> Hi Andy,
>>
>>> You are right :( It messes up chroma but only in case of sharpen at
>>> least for me.
>>
>> Retested and denoise is also affected for me.
>>
>>> I tried some changes and the chroma effects are fixed if we apply the
>>> sharpness
>>> filter to only to the first surface instead of all the surfaces of the
>>> buffer.
>>> Can you verify this?
>>
>> If you mean changing surfaces[i] to surfaces[0] below, then it is
>> better, but there are still artifacts/chroma errors.
>>
>> In addition to changing as above, changing sampler_views[0] seems
>> to fix denoise and negative sharpen, but gives a new sort of
>> artifact for positive sharpen.
> Sorry for the misleading language. What I meant was that the filter should only
> be applied to first resource i.e. use only first sampler_view and
> surface. As you
> replaced i by 0 the filter gets applied multiple times. I tried doing
> that and I am
> experiencing same problems i.e. artifacts with positive sharpen.
>
> So can you please it try it again by replacing VL_MAX_SURFACES with 1 in the
> for loop. It should probably fix the problems.

Well that would work around the problem, but not fix it.

This way you only apply the filters to the luma channel and not all the 
other ones.

The problem is possible that the chroma channels are subsampled and you 
don't use the correct resolution in the filters for them.

Regards,
Christian.

>
> Regards,
> Nayan.
>>
>>>>>> +   for(i = 0; i < VL_MAX_SURFACES; ++i) {
>>>>>> +      if(sampler_views[i] != NULL && surfaces[i] != NULL) {
>>>>>> +         if (vmixer->noise_reduction.filter)
>>>>>> +            vl_median_filter_render(vmixer->noise_reduction.filter,
>>>>>> +                                    sampler_views[i], surfaces[i]);
>>>>>> +
>>>>>> +         if (vmixer->sharpness.filter)
>>>>>> +            vl_matrix_filter_render(vmixer->sharpness.filter,
>>>>>> +                                    sampler_views[i], surfaces[i]);
>>