[Mesa-dev] [PATCH] fix minor error in YUV2RGB matrix used in shader

[Kenneth Graunke]

On Friday, April 28, 2017 3:02:01 PM PDT Eric Anholt wrote:
> Johnson Lin <johnson.lin at intel.com> writes:
> 
> > The matrix used for YCbCr to RGB is listed in Wiki https://en.wikipedia.org/wiki/YCbCr;
> > There is minor error in the matrix constant: 0.0625=16/256 should be 16.0/255,
> >  and 0.5=128.0/256 should be 128.0/255.
> > Note that conversion from a 0-255 byte number to 0-1.0 float is to divide by 255
> >  instead of 256. That's we get 255=1.0f.
> > By the constant change we can see the CSC result is bit aligned with
> > Wiki conversion result and FFMPeg result.
> > Otherwise in some situation, there will be one bit difference
> >
> > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100854
> > ---
> >  src/compiler/nir/nir_lower_tex.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/src/compiler/nir/nir_lower_tex.c b/src/compiler/nir/nir_lower_tex.c
> > index 352d1499bc8d..385739a56a71 100644
> > --- a/src/compiler/nir/nir_lower_tex.c
> > +++ b/src/compiler/nir/nir_lower_tex.c
> > @@ -244,9 +244,9 @@ convert_yuv_to_rgb(nir_builder *b, nir_tex_instr *tex,
> >     nir_ssa_def *yuv =
> >        nir_vec4(b,
> >                 nir_fmul(b, nir_imm_float(b, 1.16438356f),
> > -                        nir_fadd(b, y, nir_imm_float(b, -0.0625f))),
> > -               nir_channel(b, nir_fadd(b, u, nir_imm_float(b, -0.5f)), 0),
> > -               nir_channel(b, nir_fadd(b, v, nir_imm_float(b, -0.5f)), 0),
> > +                        nir_fadd(b, y, nir_imm_float(b, -0.0627451f))),
> > +               nir_channel(b, nir_fadd(b, u, nir_imm_float(b, -0.50196078431f)), 0),
> > +               nir_channel(b, nir_fadd(b, v, nir_imm_float(b, -0.50196078431f)), 0),
> >                 nir_imm_float(b, 0.0));
> 
> Could we use 16.0/255.0 and 128.0/255.0, instead of magic-looking
> numbers?  With that, it will be:
> 
> Reviewed-by: Eric Anholt <eric at anholt.net>
> 

Also, please start the commit title with "nir: "
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20170428/4e2232eb/attachment.sig>