Fragment Shader of FP16 produces incorrect image drawing.

Mon Nov 20 11:30:37 UTC 2017

Am 20.11.2017 09:08, schrieb Pekka Paalanen:
> On Sat, 18 Nov 2017 20:42:44 +0900
> YoungJun Jo<dtoartist at gmail.com>  wrote:
>
>> Hi pq,
>>
>> Thank you for your response.
>>
>>> However, weston_matrix *is* using float instead of double. Is that the
>>> problem?
>> No. according to your analysis, it does not seem to be the cause of the this
>> problem.
>>
>>> During animations and any transformations that apply scaling, it is
>>> expected that the result has some sampling artifacts. The gl-renderer
>>> also has some logic to switch between GL_NEAREST and GL_LINEAR
>>> interpolation. What kind of animation is it that triggers the problem
>>> for you?
>> When I trace it in gl-renderer, the problem occurs in GL_LINEAR filter.
>> After changing the logic to GL_NEAREST instead of the GL_LINEAR,
>> the incorrect drawing of the triangle has disappeared.
>> It seems that there is a problem when interpolation is performed
>> because of fp16 limitation.
>>
>>> In the attached picture, I see not only the triangle you speak of, but
>>> also a color difference at the topmost horizontal bar. Could you share
>>> the original full-resolution captures in the web instead of an
>>> attachment?
>> The error of the color difference seems to be in the form of a triangle.
>> Please refer to the following link for the captured image.
>> cycleline-error.png
>> <https://www.dropbox.com/s/k0u0bh8vn9vybee/cycleline-error.png?dl=0>
>>
> Hi,
>
> that is a very interesting image indeed. Is this really the captured
> image with the rendering problem?
>
> If I look at it at 100% or 700% scaling in firefox, I cannot see
> anything wrong in it.
>
> However, if I look at it at 300% and scroll it sideways, I do see the
> strange triangle of the top row, and I also see another vertical "edge"
> on the same row somewhat past the midpoint. I don't see them if the
> image stationary.
>
> If I look at it at 200% and squint, I can vaguely see the triangle and
> the "edge".
>
> I also tried 'xmag' of the area where I do see the triangle, and the
> magnification does not have it.
>
> Therefore, I don't think this is a misrendering but some other effect
> in the display signal, the monitor device, or even human vision.
>
> I don't know what to do about it.
>
>
> Thanks,
> pq
>
>> Regards,
>> yj
>>
>> 2017-11-17 18:56 GMT+09:00 Pekka Paalanen<ppaalanen at gmail.com>:
>>
>>> On Fri, 17 Nov 2017 17:30:47 +0900
>>> YoungJun Jo<dtoartist at gmail.com>  wrote:
>>>
>>>> Dear all,
>>>>
>>>> In the weston environment, there is a problem when displaying a specific
>>>> image using the GPU(Fragment Shader(FS) only supports FP16).
>>>> In fact, I think it's not common case to use a GPU with FS of FP16
>>>> constraints in a desktop environment, so I was worried about asking
>>>> question.
>>>>
>>>> The specific image which the error occurs is an image with black vertical
>>>> lines and white vertical lines arranged at 1 pixel intervals.
>>>> I modified the weston-image code to display fullscreen to better observe
>>>> where the error occurred.
>>>>
>>>> When I display the image on the screen, I get a triangle-shaped wrong
>>>> drawing during the time of the fade animation in the top-right part.
>>>> (Please refer to attachment 'cycleline-error.png')
>>>>
>>>> First, I contacted the GPU vendor about this issue and got the following
>>>> answer:
>>>> ---
>>>> I'm pretty sure the issue is that your texture coordinates are being
>>>> converted to fp16 because arithmetic is being performed on them before
>>> they
>>>> are used. For fp16 values above the 0.5 limit on the texture coordinate
>>> the
>>>> fp16 "increment" between values is 2^-11, or about 0.000488. For a
>>> texture
>>>> which is 1024 pixels wide this give an sample point accuracy of only +-
>>>> half a pixel width which is insufficient for accurate sampling
>>> (especially
>>>> when using GL_LINEAR filtering).
>>>> Short answer: don't do maths on texture coordinates in the fragment
>>> shader.
>>>> ---
>>>> Based on the above answer, I would like to confirm whether the exception
>>>> handling is appropriate for animation in the condition that FS is FP16 in
>>>> weston's gl-renderer.
>>>> In addition, I have found one way to prevent such incorrect rendering.
>>>> When i remove the *weston_matrix_init(&animation->transform.matrix)* in
>>>> *weston_view_animation_create()* function;, The error does not occur and
>>>> the animation effect is shown.
>>>> In Android(I do not know if it is appropriate to compare), since there is
>>>> no matrix operation during animation, wrong drawing does not occur.
>>>> So I would like to know whether the matrix operation in weston animation
>>> is
>>>> necessary or can be removed.
>>>> Or is there another good way to avoid the above precision errors?
>>>>
>>>> My debugging and analysis may be wrong, so I would appreciate any
>>> feedback.
>>>
>>> Hi,
>>>
>>> weston's vertex or fragment shaders do not do any computations on the
>>> texture coordinates, so I am confused about saying it's about the
>>> shaders.
>>>
>>> Commenting out a call to weston_matrix_init() does not prevent the
>>> matrix from being used. It will only cause the matrix to be used
>>> uninitialized.
>>>
>>> However, weston_matrix *is* using float instead of double. Is that the
>>> problem?
>>>
>>> During animations and any transformations that apply scaling, it is
>>> expected that the result has some sampling artifacts. The gl-renderer
>>> also has some logic to switch between GL_NEAREST and GL_LINEAR
>>> interpolation. What kind of animation is it that triggers the problem
>>> for you?
>>>
>>> In the attached picture, I see not only the triangle you speak of, but
>>> also a color difference at the topmost horizontal bar. Could you share
>>> the original full-resolution captures in the web instead of an
>>> attachment?
>>>
>>>
>>> Thanks,
>>> pq
>>>

Hello

I also looked at the image in a Firefox webbrowser but got different 
effects:
I got no triangle in any scale.
When I scale down then in one scale only the 4th bar got a white or grey 
between the black lines become a little green and in the next smaller 
scale the same happens only with the 5th bar.
When I scale up then some areas where the black and white lines touch 
each other I get a little green, red, or blue. These are the areas where 
a grey color is mixed.

So it seems to be that it is an effect of the display raster, but it 
could also be an illusion that tricks out the human vision when the eyes 
move over a specific line with the right size of the black and white 
lines respectively pattern.

Best Regards
Christian Stroetmann