[Bug 39309] vdpau decodes noise on rv350

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=39309

--- Comment #9 from Andy Furniss <lists at andyfurniss.entadsl.com> 2012-03-12 17:29:18 PDT ---
(In reply to comment #2)

> Testing just -vo vdpau with sw decode it works but crashes if the window is
> resized this happened before as well as today.

After playing with gdb, valgrind and VDPAU_TRACE=1 I think I know why this is
happening now.

Mplayer is asking for video surfaces of a certain size but not checking what
size was allocated as the spec says it should. Because npot textures are not
enabled  for r300 video, this makes it more noticeable than r600 which for SD
will allocate buffers with lines = requested.

This is not the case for HD, mplayer asking for 1920x1080 will get 1088 - but
by luck or design it/ffmpeg seem to use buffers big enough. I have found an
exception to this - raw yuv, so I can now crash r600 as well as r300.

Enabling npot for r300 makes it almost always work just as r600 does. Is there
a reason it's disabled? It seems to be enabled for 3D but not video.

This is all with s/w decode - enabling npot doesn't help the decode problems.

Valgrind does throw r300 errors for decode, example below, but I don't think
that fixing them will help as xvmc has none but still decodes junk.

==17289== Invalid write of size 1
==17289==    at 0x5E4DB86: r300_dsa_inject_stencilref (r300_state.c:664)
==17289==    by 0x5EB553D: blitter_restore_fragment_states (u_blitter.c:427)
==17289==    by 0x5EB78A6: util_blitter_copy_texture_view (u_blitter.c:1060)
==17289==    by 0x5E4074C: r300_resource_copy_region (r300_blit.c:580)
==17289==    by 0x5E3EB42: r300_texture_transfer_destroy (r300_transfer.c:71)
==17289==    by 0x5EC4A8A: u_transfer_destroy_vtbl (u_resource.c:41)
==17289==    by 0x5EEE288: vl_zscan_layout (vl_zscan.c:411)
==17289==    by 0x5EE1EAD: vl_create_mpeg12_decoder (vl_mpeg12_decoder.c:857)
==17289==    by 0x5EDF353: vl_create_decoder (vl_decoder.c:73)
==17289==    by 0x5E36575: vlVdpDecoderCreate (decode.c:92)
==17289==    by 0x80F94AC: create_vdp_decoder (vo_vdpau.c:598)
==17289==    by 0x80FB0DD: control (vo_vdpau.c:1115)
==17289==  Address 0x5c13220 is 40 bytes inside a block of size 184 free'd
==17289==    at 0x4022BD8: free (vg_replace_malloc.c:427)
==17289==    by 0x5E4DEDC: r300_delete_dsa_state (r300_state.c:692)
==17289==    by 0x5EE2750: vl_mpeg12_destroy (vl_mpeg12_decoder.c:413)
==17289==    by 0x5E36382: vlVdpDecoderDestroy (decode.c:137)
==17289==    by 0x80F944B: create_vdp_decoder (vo_vdpau.c:574)
==17289==    by 0x80FB0DD: control (vo_vdpau.c:1115)
==17289==    by 0x81771E3: query_format (vf_vo.c:145)
==17289==    by 0x8148B8B: mpcodecs_config_vo (vd.c:195)
==17289==    by 0x82185D8: init_vo (vd_ffmpeg.c:511)
==17289==    by 0x821A0C6: get_format (vd_ffmpeg.c:944)
==17289==    by 0x857B3F9: mpeg_get_pixelformat (mpeg12.c:1224)
==17289==    by 0x8580D81: decode_chunks (mpeg12.c:1326)

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.