Fragment Shader of FP16 produces incorrect image drawing.

[YoungJun Jo]

Dear all,

In the weston environment, there is a problem when displaying a specific
image using the GPU(Fragment Shader(FS) only supports FP16).
In fact, I think it's not common case to use a GPU with FS of FP16
constraints in a desktop environment, so I was worried about asking
question.

The specific image which the error occurs is an image with black vertical
lines and white vertical lines arranged at 1 pixel intervals.
I modified the weston-image code to display fullscreen to better observe
where the error occurred.

When I display the image on the screen, I get a triangle-shaped wrong
drawing during the time of the fade animation in the top-right part.
(Please refer to attachment 'cycleline-error.png')

First, I contacted the GPU vendor about this issue and got the following
answer:
---
I'm pretty sure the issue is that your texture coordinates are being
converted to fp16 because arithmetic is being performed on them before they
are used. For fp16 values above the 0.5 limit on the texture coordinate the
fp16 "increment" between values is 2^-11, or about 0.000488. For a texture
which is 1024 pixels wide this give an sample point accuracy of only +-
half a pixel width which is insufficient for accurate sampling (especially
when using GL_LINEAR filtering).
Short answer: don't do maths on texture coordinates in the fragment shader.
---
Based on the above answer, I would like to confirm whether the exception
handling is appropriate for animation in the condition that FS is FP16 in
weston's gl-renderer.
In addition, I have found one way to prevent such incorrect rendering.
When i remove the *weston_matrix_init(&animation->transform.matrix)* in
*weston_view_animation_create()* function;, The error does not occur and
the animation effect is shown.
In Android(I do not know if it is appropriate to compare), since there is
no matrix operation during animation, wrong drawing does not occur.
So I would like to know whether the matrix operation in weston animation is
necessary or can be removed.
Or is there another good way to avoid the above precision errors?

My debugging and analysis may be wrong, so I would appreciate any feedback.

Regards,
yj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20171117/eed51272/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cycleline-error.png
Type: image/png
Size: 288118 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20171117/eed51272/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cycleline-error.png
Type: image/png
Size: 288118 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20171117/eed51272/attachment-0003.png>