claims of python unit test un-debugability considered somewhat exaggerated

[David Malcolm]

On Tue, 2013-04-09 at 11:22 +0100, Michael Meeks wrote:
> Hi guys,
> 
> On Mon, 2013-04-08 at 11:32 -0600, Tom Tromey wrote:
> > CCing David Malcolm.
> > 
> > Michael> (gdb) py-bt
> > 
> > I just wanted to mention here that Phil Muldoon is working on a "frame
> > filter" patch series that is basically "pretty printing for stack
> > frames".  With this in place you'll no longer need a special command --
> > with appropriate support from Python, you'll be able to see interpreted
> > frames interleaved with the low-level C frames.
> 
> 	Oh - nice ! :-)
> 
> > This series is nearing final review now and should appear in gdb 7.7,
> > later this year.
> 
> 	Excellent news indeed.
> 
> > I think we're planning to rewrite the existing "py-bt" code into frame
> > filter form ourselves.
> 
> 	So - in which case this significantly reduces the problems of using
> python for debugging; I assume that with py-bt - there are still issues
> with interleaving two tools for printing parts of backtraces when we
> call several times to and fro between C++ and python.
> 
> > Eventually we'd like to have more full support for "mixed" interpreter
> > and C debugging.  However this is a ways off.
> 
> 	For me, clearly seeing the python line numbers is the key rather than
> the interpreter details - we treat python itself as working black-box -
> and the script itself as the problem ;-)

Aha.  Sorry; I came into this discussion halfway though.

I wrote the Python gdb hooks for CPython itself (i.e. "py-bt" and
others).  You can see documentation for them here:

http://docs.python.org/devguide/gdb.html#gdb-7-and-later

In particular:
  (gdb) py-list
should give you python source code for the current python frame.
	

> 
> 	So - re-examining my requirements:
> 
> >     + debug-ability - when the test fails - and the best tests are
> >       written to fail regularly ;-) we need to be able -very-simply-
> >       to get a backtrace we can work with - without a ton of
> >       training, reading a dozen wiki pages, poking for obscure
> >       symbols, etc.
> 
> 	Michael - as/when a python unit test fails - can we make the magic
> environment variables that are listed include some debugging
> instructions that include the "py-bt" detail etc. so that everything
> needed is in the failure message ? when we're in the automatic trace
> collection mode (I forget what env. var that is) do we dump the python
> pieces as well ? Also - can we distribute / include / install the magic
> that makes 'py-bt' work - not sure it works for me (on openSUSE - at
> least a facile start gdb, type py-bt fails in spades ;-)

On Fedora we package the python-gdb.py code so it appears in the
python-debuginfo rpm and is in the correct path so that it gets
autoloaded by gdb when libpython2.7.so is debugged.

Specifically (quoting Fedora's python3.spec):

  # We install a collection of hooks for gdb that make it easier to debug
  # executables linked against libpython3* (such as /usr/bin/python3 itself)
  #
  # These hooks are implemented in Python itself (though they are for the version
  # of python that gdb is linked with, in this case Python 2.7)
  #
  # gdb-archer looks for them in the same path as the ELF file, with a -gdb.py suffix.
  # We put them in the debuginfo package by installing them to e.g.:
  #  /usr/lib/debug/usr/lib/libpython3.2.so.1.0.debug-gdb.py
  #
  # See https://fedoraproject.org/wiki/Features/EasierPythonDebugging for more
  # information
  #
  # Copy up the gdb hooks into place; the python file will be autoloaded by gdb
  # when visiting libpython.so, provided that the python file is installed to the
  # same path as the library (or its .debug file) plus a "-gdb.py" suffix, e.g:
  #  /usr/lib/debug/usr/lib64/libpython3.2.so.1.0.debug-gdb.py
  # (note that the debug path is /usr/lib/debug for both 32/64 bit)
  #
  # Initially I tried:
  #  /usr/lib/libpython3.1.so.1.0-gdb.py
  # but doing so generated noise when ldconfig was rerun (rhbz:562980)
  #
%if 0%{?with_gdb_hooks}
  DirHoldingGdbPy=%{_prefix}/lib/debug/%{_libdir}
  PathOfGdbPy=$DirHoldingGdbPy/$PyInstSoName.debug-gdb.py

  mkdir -p %{buildroot}$DirHoldingGdbPy
  cp Tools/gdb/libpython.py %{buildroot}$PathOfGdbPy
%endif # with_gdb_hooks


> >        + that backtrace should not have big, opaque chunks that
> >         are either empty (cf. Java) or point to random / irrelevant
> >         pieces of C/C++ interpreter code - Python / StarBasic &
> >         others. It should allow interactive inspection of variables
> >         up and down the stack.
> 
> 	It seems that this is getting towards working on modern Linux systems.
> Do we get variable inspection sorted out there as well ?

py-locals
py-print

> >       + reliability and performance: new unit tests should be small,
> >         fast, non-duplicative (ie. re-using existing code &
> >         frameworks)
> 
> 	I guess re-using the existing C++ test code / bootstrapping framework
> mostly avoids duplication here; and either way unit tests are (sadly)
> one of the more duplicative pieces of code we have.

The issue you may run into is that, at sufficiently high optimization
levels on x86_64, gdb's python hooks can have trouble locating the
PyFrameObject* within the C locals of the relevant C frame, and without
that, the "py-" commands can't work.  I've tried to make them fail
gracefully.

> >       + ideally - the unit tests should run -in-the-same-process-
> >         which significantly helps with the above performance,
> >         debugging, reliability and more.
> 
> 	And this is now fixed; so - David & Michael - thanks for working on
> this - looks like a good solution with a brighter future :-)

Hope the above is helpful
Dave