[Mesa-dev] [PATCH] st/vdpau: fix YCbCr down/up-loads for buffers larger than requested

[Andy Furniss]

Andy Furniss wrote:
> Christian König wrote:
>> When the video buffer turns out to be larger than
>> requested by the application we shouldn't upload
>> or download more data into / from it original requested.
>>
>> Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=39309
>
> Hi
>
> It doesn't fix decoding (for my 9600) - which that bug is mainly about.
>
> It does avoid the crashing with s/w decode that r300 had because npot
> wasn't enabled and the one case I found that mplayer would crash on r600
> mentioned in the bug is also fixed.
>
> Unfortunately it's not quite right, there's an artifact on r300 s/w
> decode = thin green lines right and bottom.
> R600 with the patch doesn't show it as much, but 1080 stuff does have
> one at the bottom.

Playing some more - enabling npot for r300 will make it the same as r600 
with the patch.

Never tried 422 -vo on r300 before, but it fails whatever combination of 
npot/patch is used -

VO: [vdpau] 704x576 => 704x576 Packed YUY2
r300: Ooops. Got unsupported format r8g8_r8b8_unorm in 
r300_create_sampler_view_custom.
r300_state.c:1497:r300_create_sampler_view_custom: Assertion `hwformat 
!= ~0' failed.
Trace/breakpoint trap

The next test may be a regression but I can't be sure - just would be 
surprised if I didn't test HD decode as well as SD in the past.

Anyway this is with or without patch. SD decode as before renders junk, 
but HD is now throwing errors.

r300: Implementation error: Render targets are too big in 
r300_set_framebuffer_state, refusing to bind framebuffer state!
radeon: The kernel rejected CS, see dmesg for more information.
Then lots of -
r300: Implementation error: Render targets are too big in 
r300_set_framebuffer_state, refusing to bind framebuffer state!

from dmesg -

[drm] Initialized radeon 2.15.0 20080528 for 0000:02:00.0 on minor 0
[drm:r100_cs_track_check] *ERROR* [drm] No buffer for z buffer !
[drm:radeon_cs_ib_chunk] *ERROR* Invalid command stream !

If I enable npot -

diff --git a/src/gallium/drivers/r300/r300_screen.c 
b/src/gallium/drivers/r300/r300_screen.c
index da47fa2..758c690 100644
--- a/src/gallium/drivers/r300/r300_screen.c
+++ b/src/gallium/drivers/r300/r300_screen.c
@@ -325,7 +325,7 @@ static int r300_get_video_param(struct pipe_screen 
*screen,
        case PIPE_VIDEO_CAP_SUPPORTED:
           return vl_profile_supported(screen, profile);
        case PIPE_VIDEO_CAP_NPOT_TEXTURES:
-         return 0;
+         return 1;
        case PIPE_VIDEO_CAP_MAX_WIDTH:
        case PIPE_VIDEO_CAP_MAX_HEIGHT:
           return vl_video_buffer_max_size(screen);

Then the errors go away (it still renders junk).