[Bug 102006] gstreamer vaapih264enc segfault

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=102006

--- Comment #2 from Julien Isorce <julien.isorce at gmail.com> ---
Comment on attachment 133180
  --> https://bugs.freedesktop.org/attachment.cgi?id=133180
proposed patch by Julien Isorce

commit 91d93aa62162f98d6377e5c796b63faa263f2c18
Author: Julien Isorce <julien.isorce at gmail.com>
Date:   Tue Jul 25 15:31:28 2017 +0100

    st/va: change frame_idx from array to hash table

    The picture_id was assumed to be a frame number so in 0-31.
    But the vaapi client gstreamer-vaapi uses the surfaces handles
    as identifier which are unsigned int.

    This bug can happen when using a lot of vaapi surfaces within
    the same process. Indeed Mesa/st/va increments a counter for the
    surface ID: mesa/util/u_handle_table.c::handle_table_add which
    starts from 0 and incremented by 1 at each call.
    So creating more than 32 surfaces was a problem.

    The following bug contains a test that reproduces the problem
    by running a couple of vaapih264enc in the same process. The
    above also explains why there was no pb when running them in
    separated processes.

    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=102006
    Signed-off-by: Julien Isorce <jisorce at oblong.com>
    Tested-by: Tomas Rataj <rataj28 at gmail.com>
    Acked-by: Christian König <christian.koenig at amd.com>
    Reviewed-and-tested-by: Boyuan Zhang <Boyuan.Zhang at amd.com>

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/dri-devel/attachments/20170814/e9576e74/attachment.html>