Difference between revisions of "Python/Debugging"

From SambaWiki
(add some debugging notes)
 
(Add note about using PYTHONMALLOC=malloc)
 
(3 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
Debugging Python code
 
Debugging Python code
  +
== Debugging Python code ==
=====================
 
   
 
A commonly used trick is to import the Python debugger at some point where you would like to do debugging. Just add something like this:
 
A commonly used trick is to import the Python debugger at some point where you would like to do debugging. Just add something like this:
   
  +
<pre>
{{{
 
 
import pdb
 
import pdb
 
pdb.set_trace()
 
pdb.set_trace()
  +
</pre>
}}}
 
   
 
and Python will drop into the Python debugger, allowing you to inspect variables, execute code, step into/next/continue, etc. The commands are pretty similar to those of gdb.
 
and Python will drop into the Python debugger, allowing you to inspect variables, execute code, step into/next/continue, etc. The commands are pretty similar to those of gdb.
   
Debugging Python extensions
+
== Debugging Python extensions ==
===========================
 
   
 
As Python extensions are written in C, it is usually not sufficient to rely on pdb when debugging extensions.
 
As Python extensions are written in C, it is usually not sufficient to rely on pdb when debugging extensions.
   
GDB
+
=== GDB ===
----
 
   
The following gdb macros (to be placed in ~/.gdbinit) allow you to easily inspect the reference count of python objects, their string representation as used in Python and the python stack from within gdb.
+
There are some useful gdb macros distributed with Python that allow you to easily inspect the reference count of python objects, their string representation as used in Python and the python stack from within gdb. On Debian/Ubuntu systems the file with macros can be found in /usr/share/doc/pythonX.Y/gdbinit. Copy this file to ~/.gdbinit to use it.
   
  +
See the file itself for the new commands that it provides and an explanation of their use.
{{{
 
# -*- ksh -*-
 
#
 
# If you use the GNU debugger gdb to debug the Python C runtime, you
 
# might find some of the following commands useful. Copy this to your
 
# ~/.gdbinit file and it'll get loaded into gdb automatically when you
 
# start it up. Then, at the gdb prompt you can do things like:
 
#
 
# (gdb) pyo apyobjectptr
 
# <module 'foobar' (built-in)>
 
# refcounts: 1
 
# address : 84a7a2c
 
# $1 = void
 
# (gdb)
 
   
  +
=== Valgrind ===
# Prints a representation of the object to stderr, along with the
 
# number of reference counts it current has and the hex address the
 
# object is allocated at. The argument must be a PyObject*
 
define pyo
 
print _PyObject_Dump($arg0)
 
end
 
   
  +
Running Python with PYTHONMALLOC=malloc will have it use malloc() as its memory allocator, avoiding most warnings emitted by Valgrind. There is also a suppressions file for Python's memory manager (pymalloc). It is part of the Python source tree, and distributed inside of the Python package in some distributions (Debian/Ubuntu put it in the "python" package, in /usr/lib/valgrind/python.supp). With this suppressions file enabled you *should* have no valgrind warnings from Python with any of the Samba modules loaded. See also /usr/share/doc/python2.4/README.valgrind for a more detailed explanation.
# Prints a representation of the object to stderr, along with the
 
# number of reference counts it current has and the hex address the
 
# object is allocated at. The argument must be a PyGC_Head*
 
define pyg
 
print _PyGC_Dump($arg0)
 
end
 
   
  +
Example usage:
# print the local variables of the current frame
 
define pylocals
 
set $_i = 0
 
while $_i < f->f_nlocals
 
if f->f_localsplus + $_i != 0
 
set $_names = co->co_varnames
 
set $_name = PyString_AsString(PyTuple_GetItem($_names, $_i))
 
printf "%s:\n", $_name
 
# side effect of calling _PyObject_Dump is to dump the object's
 
# info - assigning just prevents gdb from printing the
 
# NULL return value
 
set $_val = _PyObject_Dump(f->f_localsplus[$_i])
 
end
 
set $_i = $_i + 1
 
end
 
end
 
   
  +
<pre>
# A rewrite of the Python interpreter's line number calculator in GDB's
 
  +
valgrind --suppressions=/usr/lib/valgrind/python.supp python foo.py
# command language
 
  +
</pre>
define lineno
 
set $__continue = 1
 
set $__co = f->f_code
 
set $__lasti = f->f_lasti
 
set $__sz = ((PyStringObject *)$__co->co_lnotab)->ob_size/2
 
set $__p = (unsigned char *)((PyStringObject *)$__co->co_lnotab)->ob_sval
 
set $__li = $__co->co_firstlineno
 
set $__ad = 0
 
while ($__sz-1 >= 0 && $__continue)
 
set $__sz = $__sz - 1
 
set $__ad = $__ad + *$__p
 
set $__p = $__p + 1
 
if ($__ad > $__lasti)
 
set $__continue = 0
 
end
 
set $__li = $__li + *$__p
 
set $__p = $__p + 1
 
end
 
printf "%d", $__li
 
end
 
   
# print the current frame - verbose
 
define pyframev
 
pyframe
 
pylocals
 
end
 
   
  +
== Pyflakes ==
define pyframe
 
set $__fn = (char *)((PyStringObject *)co->co_filename)->ob_sval
 
set $__n = (char *)((PyStringObject *)co->co_name)->ob_sval
 
printf "%s (", $__fn
 
lineno
 
printf "): %s\n", $__n
 
### Uncomment these lines when using from within Emacs/XEmacs so it will
 
### automatically track/display the current Python source line
 
# printf "%c%c%s:", 032, 032, $__fn
 
# lineno
 
# printf ":1\n"
 
end
 
   
  +
Pyflakes is a simple static code checker for Python. It has some nice integration for emacs and vim:
### Use these at your own risk. It appears that a bug in gdb causes it
 
### to crash in certain circumstances.
 
   
  +
* emacs: http://plope.com/Members/chrism/flymake-mode
#define up
 
  +
* vim: http://www.vim.org/scripts/script.php?script_id=2441
# up-silently 1
 
# printframe
 
#end
 
 
#define down
 
# down-silently 1
 
# printframe
 
#end
 
 
define printframe
 
if $pc > PyEval_EvalFrameEx && $pc < PyEval_EvalCodeEx
 
pyframe
 
else
 
frame
 
end
 
end
 
 
# Here's a somewhat fragile way to print the entire Python stack from gdb.
 
# It's fragile because the tests for the value of $pc depend on the layout
 
# of specific functions in the C source code.
 
 
# Explanation of while and if tests: We want to pop up the stack until we
 
# land in Py_Main (this is probably an incorrect assumption in an embedded
 
# interpreter, but the test can be extended by an interested party). If
 
# Py_Main <= $pc <= Py_GetArgcArv is true, $pc is in Py_Main(), so the while
 
# tests succeeds as long as it's not true. In a similar fashion the if
 
# statement tests to see if we are in PyEval_EvalFrame().
 
 
# print the entire Python call stack
 
define pystack
 
while $pc < Py_Main || $pc > Py_GetArgcArgv
 
if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx
 
pyframe
 
end
 
up-silently 1
 
end
 
select-frame 0
 
end
 
 
# print the entire Python call stack - verbose mode
 
define pystackv
 
while $pc < Py_Main || $pc > Py_GetArgcArgv
 
if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx
 
pyframev
 
end
 
up-silently 1
 
end
 
select-frame 0
 
end
 
}}}
 
 
Valgrind
 
--------
 
 
There is a suppressions file for Python's memory manager (pymalloc). It is part of the Python source tree, and distributed inside of the Python package in some distributions (Debian/Ubuntu put it in the "python" package, in /usr/lib/valgrind/python.supp). With this suppressions file enabled you *should* have no valgrind warnings from Python with any of the Samba modules loaded. See also /usr/share/doc/python2.4/README.valgrind for a more detailed explanation.
 
 
Example usage:
 
 
{{{
 
valgrind --suppressions=/usr/lib/valgrind/python.supp python foo.py
 
}}}
 

Latest revision as of 23:38, 9 September 2021

Debugging Python code

Debugging Python code

A commonly used trick is to import the Python debugger at some point where you would like to do debugging. Just add something like this:

import pdb
pdb.set_trace()

and Python will drop into the Python debugger, allowing you to inspect variables, execute code, step into/next/continue, etc. The commands are pretty similar to those of gdb.

Debugging Python extensions

As Python extensions are written in C, it is usually not sufficient to rely on pdb when debugging extensions.

GDB

There are some useful gdb macros distributed with Python that allow you to easily inspect the reference count of python objects, their string representation as used in Python and the python stack from within gdb. On Debian/Ubuntu systems the file with macros can be found in /usr/share/doc/pythonX.Y/gdbinit. Copy this file to ~/.gdbinit to use it.

See the file itself for the new commands that it provides and an explanation of their use.

Valgrind

Running Python with PYTHONMALLOC=malloc will have it use malloc() as its memory allocator, avoiding most warnings emitted by Valgrind. There is also a suppressions file for Python's memory manager (pymalloc). It is part of the Python source tree, and distributed inside of the Python package in some distributions (Debian/Ubuntu put it in the "python" package, in /usr/lib/valgrind/python.supp). With this suppressions file enabled you *should* have no valgrind warnings from Python with any of the Samba modules loaded. See also /usr/share/doc/python2.4/README.valgrind for a more detailed explanation.

Example usage:

valgrind --suppressions=/usr/lib/valgrind/python.supp python foo.py


Pyflakes

Pyflakes is a simple static code checker for Python. It has some nice integration for emacs and vim: