[Bug 754120] avdec_hevc: Segfault in hevc decode with glimagesink and AVX2 CPU feature

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=754120

--- Comment #7 from Tim-Philipp Müller <t.i.m at zen.co.uk> ---
Waiting for sree to confirm.

I don't fully understand why this is required though, it might just be
coincidence that this fixes it. I didn't find anything that says that 128-bit
alignment is required for vinserti128.

It crashes here:

ff_hevc_transform_add32_8_avx2 () at libavcodec/x86/hevc_res_add.asm:186
186        TR_ADD_SSE_16_32_8   0, r0,      r0+r2

which is defined as:

%macro TR_ADD_SSE_16_32_8 3
    mova             xm2, [r1+%1   ]
    mova             xm6, [r1+%1+16]
%if cpuflag(avx2)
    vinserti128       m2, m2, [r1+%1+32], 1
    vinserti128       m6, m6, [r1+%1+48], 1
%endif
%if cpuflag(avx)
    psubw             m1, m0, m2
    psubw             m5, m0, m6
%else
    mova              m1, m0
    mova              m5, m0
    psubw             m1, m2
    psubw             m5, m6
%endif
    packuswb          m2, m6
    packuswb          m1, m5

    mova             xm4, [r1+%1+mmsize*2   ]
    mova             xm6, [r1+%1+mmsize*2+16]
%if cpuflag(avx2)
    vinserti128       m4, m4, [r1+%1+96 ], 1
    vinserti128       m6, m6, [r1+%1+112], 1
%endif
%if cpuflag(avx)
    psubw             m3, m0, m4
    psubw             m5, m0, m6
%else
    mova              m3, m0
    mova              m5, m0
    psubw             m3, m4
    psubw             m5, m6
%endif
    packuswb          m4, m6
    packuswb          m3, m5

    paddusb           m2, [%2]
    paddusb           m4, [%3]
    psubusb           m2, m1
    psubusb           m4, m3
    mova            [%2], m2
    mova            [%3], m4
%endmacro

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.