[Libva] Weird memory problem around vaGetDisplay
Ratin
ratin3 at gmail.com
Sun Oct 25 10:11:29 PDT 2015
Thanks for your suggestion.
Yes it gave me some headache, not as much as some other ones related to
OpenGL and Xlib but it was significant, I had to write custom code to
experiment, also consult ASM code for latest Intel broadwell
processors(although any 64 bit processor opcodes would do), debug in
assembly etc. -Wall is actually not a good idea since it does throw a lot
of unimportant warnings since code base is large. I have fine tuned the
list of warnings that I believe absolute necessary, most of them come from
-Wall and some additional ones that -Wall doesn't cover.
Regards,
Ratin
On Sun, Oct 25, 2015 at 9:51 AM, Emil Velikov <emil.l.velikov at gmail.com>
wrote:
> Hi Ratin,
>
> I would strongly recommend going through the warning and fixing the
> (your and/or libva) code. I realise how tedious this might be,
> although it will save you a ton of time/headaches in the long run.
> Same goes for -Wextra ;-)
>
> Regards,
> Emil
>
> On 23 October 2015 at 19:15, Ratin <ratin3 at gmail.com> wrote:
> > Hi Emil, I only had basic warnings enabled (-Wpedantic -Wformat etc)
> since I
> > get tons of non-severe warnings. I added -Wall and I do get "implicit
> > declaration of function 'vaGetDisplay'
> [-Wimplicit-function-declaration]". I
> > will keep this enabled and perhaps redirect compiler output to a file and
> > grep for serious warnings.
> >
> > Thanks
> >
> > Ratin
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Oct 22, 2015 at 1:49 AM, Emil Velikov <emil.l.velikov at gmail.com>
> > wrote:
> >>
> >> Hi Ratin,
> >>
> >> Out of curiosity - when you say libva_x11.h you meant
> >> /usr/include/va/va_x11.h, is that correct ? Upon closer look, the
> >> header seems to be the only place where the function is declared, thus
> >> the compiler should have thrown a warning. Do you have those disabled
> >> by any chance ?
> >>
> >> Cheers,
> >> Emil
> >>
> >>
> >> On 22 October 2015 at 01:47, Ratin <ratin3 at gmail.com> wrote:
> >> > Found the culprit, it was missing header (libva_x11.h). If anybody
> >> > interested to know, this caused the compiler to think the function
> will
> >> > return a 32 bit pointer, so the compiler inserted a cltq statement
> into
> >> > the
> >> > assembly code which was flipping the upper 32 bits to all 1s.
> >> >
> >> > On Wed, Oct 21, 2015 at 12:15 PM, Ratin <ratin3 at gmail.com> wrote:
> >> >>
> >> >> # uname -a
> >> >> Linux nuc1 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7
> 15:32:41
> >> >> UTC
> >> >> 2015 x86_64 x86_64 x86_64 GNU/Linux
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Oct 21, 2015 at 12:13 PM, Ratin <ratin3 at gmail.com> wrote:
> >> >>>
> >> >>> Breakpoint 1, vaGetDisplay (native_dpy=0x7fffe4312870) at
> va_x11.c:154
> >> >>> 154 {
> >> >>> (gdb) list
> >> >>> 149
> >> >>> 150
> >> >>> 151 VADisplay vaGetDisplay (
> >> >>> 152 Display *native_dpy /* implementation specific */
> >> >>> 153 )
> >> >>> 154 {
> >> >>> 155 VADisplay dpy = NULL;
> >> >>> 156 VADisplayContextP pDisplayContext;
> >> >>> 157
> >> >>> ;;;
> >> >>> 179 pDisplayContext->opaque = NULL;
> >> >>> 177 pDisplayContext->vaDestroy =
> >> >>> va_DisplayContextDestroy;
> >> >>> 178 pDisplayContext->vaGetDriverName =
> >> >>> va_DisplayContextGetDriverName;
> >> >>> 181 dpy =
> >> >>> (VADisplay)pDisplayContext;
> >> >>> 195 }
> >> >>> (gdb) p dpy
> >> >>> $2 = (VADisplay) 0x7fffe42d3160
> >> >>> (gdb) n <========================== stack unroll happens here
> >> >>> (gdb) list
> >> >>> 935 pVAAPICNTX->vaDisplay = (VADisplay) <==== upper
> level
> >> >>> code vaGetDisplay(pVAAPICNTX->mX11Display);
> >> >>> 936 pVAAPICNTX->voWindow = pRendererParams->mWindow;
> >> >>> (gdb) p pVAAPICNTX->vaDisplay
> >> >>> $3 = (void *) 0xffffffffe42d3160
> >> >>>
> >> >>>
> >> >>> pVAAPICNTX is malloced struct ptr and one of the element of it is
> the
> >> >>> vaDisplay declated like this :
> >> >>>
> >> >>> typedef struct _VAAPICNTX
> >> >>> {
> >> >>> uint32_t VAContextID;
> >> >>> Window voWindow;
> >> >>> int voWidth;
> >> >>> int voHeight;
> >> >>> ;;;
> >> >>> ;;;
> >> >>> VADisplay vaDisplay;
> >> >>>
> >> >>> ;;;
> >> >>>
> >> >>> ;;;
> >> >>>
> >> >>> }VAAPICNTX, *PVAAPICNTX;
> >> >>>
> >> >>>
> >> >>> As you can see, dpy above around line 195 in vaGetDisplay function
> is
> >> >>> shown as 0x7fffe42d3160 but once the program counter comes out of
> that
> >> >>> function and stack rolling happens, and context switches to upper
> >> >>> level
> >> >>> code, the assigned value has 0xffffffffe42d3160 (sign extended vs
> >> >>> unsigned). What might cause this ? Next instance when I call
> >> >>> vaInitialize
> >> >>> with this vaDisplay, it craches:
> >> >>>
> >> >>> Program received signal SIGSEGV, Segmentation fault.
> >> >>> vaDisplayIsValid (dpy=0xffffffffe42d3450) at va.c:102
> >> >>> 102 return pDisplayContext && (pDisplayContext->vadpy_magic
> ==
> >> >>> VA_DISPLAY_MAGIC) && pDisplayContext->vaIsValid(pDisplayContext);
> >> >>>
> >> >>>
> >> >>>
> >> >>> Any help/pointer would be appreciated. Thanks
> >> >>>
> >> >>>
> >> >>>
> >> >>> gcc --version
> >> >>> gcc (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4
> >> >>> Copyright (C) 2013 Free Software Foundation, Inc.
> >> >>> This is free software; see the source for copying conditions. There
> >> >>> is
> >> >>> NO
> >> >>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> >> >>> PURPOSE.
> >> >>>
> >> >>> ls -la libc-2.19.so
> >> >>> -rwxr-xr-x 1 root root 1845024 Dec 4 2014 libc-2.19.so
> >> >>>
> >> >>> ldd --version
> >> >>> ldd (Ubuntu EGLIBC 2.19-0ubuntu6.5) 2.19
> >> >>>
> >> >>>
> >> >>> Platform:
> >> >>>
> >> >>> Ratin
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Libva mailing list
> >> > Libva at lists.freedesktop.org
> >> > http://lists.freedesktop.org/mailman/listinfo/libva
> >> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/libva/attachments/20151025/1fe4eaad/attachment.html>
More information about the Libva
mailing list