[Mesa-dev] [PATCH] fix minor error in YUV2RGB matrix used in shader

[Eric Anholt]

"Kristian H. Kristensen" <krh at bitplanet.net> writes:

> Johnson Lin <johnson.lin at intel.com> writes:
>
>> The matrix used for YCbCr to RGB is listed in Wiki https://en.wikipedia.org/wiki/YCbCr;
>> There is minor error in the matrix constant: 0.0625=16/256 should be 16.0/255,
>>  and 0.5=128.0/256 should be 128.0/255.
>> Note that conversion from a 0-255 byte number to 0-1.0 float is to divide by 255
>>  instead of 256. That's we get 255=1.0f.
>> By the constant change we can see the CSC result is bit aligned with
>> Wiki conversion result and FFMPeg result.
>> Otherwise in some situation, there will be one bit difference
>
> Thanks for fixing this.  Try to wrap the the commit message so it fits
> in 80 columns, eg:
>
> The matrix used for YCbCr to RGB is listed in:
>
>   https://en.wikipedia.org/wiki/YCbCr;
>
> There is minor error in the matrix constant: 0.0625=16/256 should be
> 16.0/255, and 0.5=128.0/256 should be 128.0/255.  Note that conversion
> from a 0-255 byte number to 0-1.0 float is to divide by 255 instead of
> 256. That's we get 255=1.0f.
>
> By the constant change we can see the CSC result is bit aligned with
> Wiki conversion result and FFMPeg result.  Otherwise in some situation,
> there will be one bit difference

Proposed rewording:

    nir/lower_tex: Fix minor error in YUV color conversion matrix
    
    The matrix used for YCbCr to RGB is listed in:
    
        https://en.wikipedia.org/wiki/YCbCr
    
    There was an error in converting the offsets from integers to unorm
    values: 0.0625=16/256 should be 16.0/255,and 0.5=128.0/256 should be
    128.0/255.  With this fix, the CSC result is bit aligned with wikipedia's
    conversion result and FFMPeg's result.
    
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100854
    Reviewed-by: Eric Anholt <eric at anholt.net>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20170502/96dc360b/attachment-0001.sig>