Fragment Shader of FP16 produces incorrect image drawing.

[Pekka Paalanen]

On Fri, 17 Nov 2017 17:30:47 +0900
YoungJun Jo <dtoartist at gmail.com> wrote:

> Dear all,
> 
> In the weston environment, there is a problem when displaying a specific
> image using the GPU(Fragment Shader(FS) only supports FP16).
> In fact, I think it's not common case to use a GPU with FS of FP16
> constraints in a desktop environment, so I was worried about asking
> question.
> 
> The specific image which the error occurs is an image with black vertical
> lines and white vertical lines arranged at 1 pixel intervals.
> I modified the weston-image code to display fullscreen to better observe
> where the error occurred.
> 
> When I display the image on the screen, I get a triangle-shaped wrong
> drawing during the time of the fade animation in the top-right part.
> (Please refer to attachment 'cycleline-error.png')
> 
> First, I contacted the GPU vendor about this issue and got the following
> answer:
> ---
> I'm pretty sure the issue is that your texture coordinates are being
> converted to fp16 because arithmetic is being performed on them before they
> are used. For fp16 values above the 0.5 limit on the texture coordinate the
> fp16 "increment" between values is 2^-11, or about 0.000488. For a texture
> which is 1024 pixels wide this give an sample point accuracy of only +-
> half a pixel width which is insufficient for accurate sampling (especially
> when using GL_LINEAR filtering).
> Short answer: don't do maths on texture coordinates in the fragment shader.
> ---
> Based on the above answer, I would like to confirm whether the exception
> handling is appropriate for animation in the condition that FS is FP16 in
> weston's gl-renderer.
> In addition, I have found one way to prevent such incorrect rendering.
> When i remove the *weston_matrix_init(&animation->transform.matrix)* in
> *weston_view_animation_create()* function;, The error does not occur and
> the animation effect is shown.
> In Android(I do not know if it is appropriate to compare), since there is
> no matrix operation during animation, wrong drawing does not occur.
> So I would like to know whether the matrix operation in weston animation is
> necessary or can be removed.
> Or is there another good way to avoid the above precision errors?
> 
> My debugging and analysis may be wrong, so I would appreciate any feedback.

Hi,

weston's vertex or fragment shaders do not do any computations on the
texture coordinates, so I am confused about saying it's about the
shaders.

Commenting out a call to weston_matrix_init() does not prevent the
matrix from being used. It will only cause the matrix to be used
uninitialized.

However, weston_matrix *is* using float instead of double. Is that the
problem?

During animations and any transformations that apply scaling, it is
expected that the result has some sampling artifacts. The gl-renderer
also has some logic to switch between GL_NEAREST and GL_LINEAR
interpolation. What kind of animation is it that triggers the problem
for you?

In the attached picture, I see not only the triangle you speak of, but
also a color difference at the topmost horizontal bar. Could you share
the original full-resolution captures in the web instead of an
attachment?


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20171117/d94797b2/attachment.sig>