[Mesa-stable] [Mesa-dev] [PATCH v2 2/2] mesa: _mesa_format_convert should be endian agnostic

[Oded Gabbay]

On Fri, Aug 7, 2015 at 10:30 PM, Matt Turner <mattst88 at gmail.com> wrote:
> On Fri, Aug 7, 2015 at 12:24 PM, Oded Gabbay <oded.gabbay at gmail.com> wrote:
>> This patch fixes a bug that is manifested in the read path of mesa when
>> running on big-endian machines. The effects can be seen when running
>> piglit sanity test and/or taking a screen capture.
>>
>> The bug is caused when _mesa_format_convert receives src_format as
>> mesa_format, which it thens changes to mesa_array_format. During this
>> change, it checks for endianness and swaps the bytes accordingly.
>> However, because the bytes are _already_ swapped in the memory itself
>> (being written there by llvmpipe/softpipe/sw rast), and src_format
>> value matches the _actual_ contents of the memory, the result of the
>> read is wrong.
>>
>> Therefore, because other layers/functions, such as llvm or
>> read_rgba_pixels() takes care whether we are dealing with big-endian or
>> little-endian, _mesa_format_convert should be endian agnostic to avoid
>> duplicate swapping of bytes.
>>
>> btw, it is also mentioned in the comment of the function that it doesn't
>> handle byte-swapping, so the original code contradicts the documentation.
>>
>> Signed-off-by: Oded Gabbay <oded.gabbay at gmail.com>
>> CC: "10.5 10.6" <mesa-stable at lists.freedesktop.org>
>> Signed-off-by: Oded Gabbay <oded.gabbay at gmail.com>
>
> Multiple S-o-b. Whoops. Just fix before committing. :)
Matt, thanks for catching that :)
Oded