Fragment Shader of FP16 produces incorrect image drawing.

Mon Nov 20 08:08:16 UTC 2017

On Sat, 18 Nov 2017 20:42:44 +0900
YoungJun Jo <dtoartist at gmail.com> wrote:

> Hi pq,
> 
> Thank you for your response.
> 
> > However, weston_matrix *is* using float instead of double. Is that the
> > problem?  
> 
> No. according to your analysis, it does not seem to be the cause of the this
> problem.
> 
> > During animations and any transformations that apply scaling, it is
> > expected that the result has some sampling artifacts. The gl-renderer
> > also has some logic to switch between GL_NEAREST and GL_LINEAR
> > interpolation. What kind of animation is it that triggers the problem
> > for you?  
> 
> When I trace it in gl-renderer, the problem occurs in GL_LINEAR filter.
> After changing the logic to GL_NEAREST instead of the GL_LINEAR,
> the incorrect drawing of the triangle has disappeared.
> It seems that there is a problem when interpolation is performed
> because of fp16 limitation.
> 
> > In the attached picture, I see not only the triangle you speak of, but
> > also a color difference at the topmost horizontal bar. Could you share
> > the original full-resolution captures in the web instead of an
> > attachment?  
> 
> The error of the color difference seems to be in the form of a triangle.
> Please refer to the following link for the captured image.
> cycleline-error.png
> <https://www.dropbox.com/s/k0u0bh8vn9vybee/cycleline-error.png?dl=0>
> 

Hi,

that is a very interesting image indeed. Is this really the captured
image with the rendering problem?

If I look at it at 100% or 700% scaling in firefox, I cannot see
anything wrong in it.

However, if I look at it at 300% and scroll it sideways, I do see the
strange triangle of the top row, and I also see another vertical "edge"
on the same row somewhat past the midpoint. I don't see them if the
image stationary.

If I look at it at 200% and squint, I can vaguely see the triangle and
the "edge".

I also tried 'xmag' of the area where I do see the triangle, and the
magnification does not have it.

Therefore, I don't think this is a misrendering but some other effect
in the display signal, the monitor device, or even human vision.

I don't know what to do about it.

Thanks,
pq

> Regards,
> yj
> 
> 2017-11-17 18:56 GMT+09:00 Pekka Paalanen <ppaalanen at gmail.com>:
> 
> > On Fri, 17 Nov 2017 17:30:47 +0900
> > YoungJun Jo <dtoartist at gmail.com> wrote:
> >  
> > > Dear all,
> > >
> > > In the weston environment, there is a problem when displaying a specific
> > > image using the GPU(Fragment Shader(FS) only supports FP16).
> > > In fact, I think it's not common case to use a GPU with FS of FP16
> > > constraints in a desktop environment, so I was worried about asking
> > > question.
> > >
> > > The specific image which the error occurs is an image with black vertical
> > > lines and white vertical lines arranged at 1 pixel intervals.
> > > I modified the weston-image code to display fullscreen to better observe
> > > where the error occurred.
> > >
> > > When I display the image on the screen, I get a triangle-shaped wrong
> > > drawing during the time of the fade animation in the top-right part.
> > > (Please refer to attachment 'cycleline-error.png')
> > >
> > > First, I contacted the GPU vendor about this issue and got the following
> > > answer:
> > > ---
> > > I'm pretty sure the issue is that your texture coordinates are being
> > > converted to fp16 because arithmetic is being performed on them before  
> > they  
> > > are used. For fp16 values above the 0.5 limit on the texture coordinate  
> > the  
> > > fp16 "increment" between values is 2^-11, or about 0.000488. For a  
> > texture  
> > > which is 1024 pixels wide this give an sample point accuracy of only +-
> > > half a pixel width which is insufficient for accurate sampling  
> > (especially  
> > > when using GL_LINEAR filtering).
> > > Short answer: don't do maths on texture coordinates in the fragment  
> > shader.  
> > > ---
> > > Based on the above answer, I would like to confirm whether the exception
> > > handling is appropriate for animation in the condition that FS is FP16 in
> > > weston's gl-renderer.
> > > In addition, I have found one way to prevent such incorrect rendering.
> > > When i remove the *weston_matrix_init(&animation->transform.matrix)* in
> > > *weston_view_animation_create()* function;, The error does not occur and
> > > the animation effect is shown.
> > > In Android(I do not know if it is appropriate to compare), since there is
> > > no matrix operation during animation, wrong drawing does not occur.
> > > So I would like to know whether the matrix operation in weston animation  
> > is  
> > > necessary or can be removed.
> > > Or is there another good way to avoid the above precision errors?
> > >
> > > My debugging and analysis may be wrong, so I would appreciate any  
> > feedback.
> >
> > Hi,
> >
> > weston's vertex or fragment shaders do not do any computations on the
> > texture coordinates, so I am confused about saying it's about the
> > shaders.
> >
> > Commenting out a call to weston_matrix_init() does not prevent the
> > matrix from being used. It will only cause the matrix to be used
> > uninitialized.
> >
> > However, weston_matrix *is* using float instead of double. Is that the
> > problem?
> >
> > During animations and any transformations that apply scaling, it is
> > expected that the result has some sampling artifacts. The gl-renderer
> > also has some logic to switch between GL_NEAREST and GL_LINEAR
> > interpolation. What kind of animation is it that triggers the problem
> > for you?
> >
> > In the attached picture, I see not only the triangle you speak of, but
> > also a color difference at the topmost horizontal bar. Could you share
> > the original full-resolution captures in the web instead of an
> > attachment?
> >
> >
> > Thanks,
> > pq
> >  

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20171120/8e7a9757/attachment-0001.sig>