[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

On Wed, Apr 10, 2019 at 1:50 PM Steve Dower <steve.dower at> wrote:
> On 10Apr2019 1227, Nathaniel Smith wrote:
> > On Wed, Apr 10, 2019, 04:04 Victor Stinner <vstinner at
> > <mailto:vstinner at>> wrote:
> >     I don't think that I ever used sys.getobjects(), whereas many projects
> >     use gc.get_objects() which is also available in release builds (not
> >     only in debug builds).
> >
> >
> > Can anyone explain what pydebug builds are... for? Confession: I've
> > never used them myself, and don't know why I would want to.
> >
> > (I have to assume that most of Steve's Windows downloads are from folks
> > who thought they were downloading a python debugger.)
> They're for debugging :)
> In general, debug builds are meant for faster inner-loop development.
> They generally do incremental builds properly and much faster by
> omitting most optimisations, which also enables source mapping to be
> more accurate when debugging. Assertions are typically enabled so that
> you are notified when a precondition is first identified rather than
> when it causes the crash (compiling these out later means you don't pay
> a runtime cost once you've got the inputs correct - generally these are
> used for developer-controlled values, rather than user-provided ones).
> So the idea is that you can quickly edit, build, debug, fix your code in
> a debug configuration, and then use a release configuration for the
> actual released build. Full release builds may take 2-3x longer than
> full debug builds, given the extra effort they make at optimisation, and
> very often can't do minimal incremental builds at all (so they may be
> 10-100x slower if you only modified one source file). But because the
> builds behave functionally equivalently, you can iterate with the faster
> configuration and get more done.

Sure, I'm familiar with the idea of debug and optimization settings in
compilers. I build python with custom -g and -O flags all the time. (I
do it by setting OPT when running configure.) It's also handy that
many Linux distros these days let you install debug metadata for all
the binaries they ship ? I've used that when debugging third-party
extension modules, to get a better idea of what was happening when a
backtrace passes through libpython. But --with-pydebug is a whole
other thing beyond that, that changes the ABI, has its own wheel tags,
requires special cases in packages that use ctypes to access PyObject*
internals, and appears to be almost entirely undocumented.

It sounds like --with-pydebug has accumulated a big grab bag of
unrelated features, mostly stuff that was useful at some point for
some CPython dev trying to debug CPython itself? It's clearly not
designed with end users as the primary audience, given that no-one
knows what it actually does and that it makes third-party extensions
really awkward to run. If that's right then I think Victor's plan of
to sort through what it's actually doing makes a lot of sense,
especially if we can remove the ABI breaking stuff, since that causes
a disproportionate amount of trouble.

> The reason we ship debug Python binaries is because debug builds use a
> different C Runtime, so if you do a debug build of an extension module
> you're working on it won't actually work with a non-debug build of CPython.

...But this is an important point. I'd forgotten that MSVC has a habit
of changing the entire C runtime when you turn on the compiler's
debugging mode. (On Linux, we generally don't bother rebuilding the C
runtime unless you're actually debugging the C runtime, and anyway if
you do want to switch to a debug version of the C runtime, it's ABI
compatible so your program binaries don't have to be rebuilt.)

Is it true that if the interpreter is built against ucrtd.lib, and an
extension module is built against ucrt.lib, then they'll have
incompatible ABIs and not work together? And that this detail is part
of what's been glommed together into the "d" flag in the soabi tag on

Is it possible for the Windows installer to include PDB files (/Zi
/DEBUG) to allow debuggers to understand the regular release
executable? (That's what I would have expected to get if I checked a
box labeled "Download debug binaries".)


Nathaniel J. Smith --