[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build
On 12Apr2019 1819, Nathaniel Smith wrote:
> On Fri, Apr 12, 2019 at 5:05 PM Steve Dower <steve.dower at python.org> wrote:
>> On 12Apr.2019 1643, Nathaniel Smith wrote:
>>> On Thu, Apr 11, 2019 at 8:26 AM Steve Dower <steve.dower at python.org> wrote:
>> The very first question I asked was whether this would let us converge
>> the ABIs, and the answer was "no".
>> Otherwise I'd have said go for it, despite the C runtime issues.
> I don't see that in the thread... just Victor saying he isn't sure
> whether there might be other ABI incompatibilities lurking that he
> hasn't found yet. Did I miss something?
"I don't know" means we can't say the APIs are converged, which is a no.
I don't think you missed anything, but just read it through a different
> I'm mostly interested in this because of the possibility of converging
> the ABIs. If you think that the C runtime thing isn't a blocker for
> that, then that's useful information. Though obviously we still need
> to figure out whether there are any other blockers :-).
> Do you happen to have a list of places where the C API leaks details
> of the underlying CRT?
> (I'm mostly curious because whenever I've looked my conclusion was
> essentially: "Well....... I don't see any places that are *definitely*
> broken, so maybe mixing CRTs is fine? but I have zero confidence that
> I caught everything, so probably better to play it safe?". At least on
> py3 ? I know the py2 C API was definitely broken if you mixed CRTs,
> because of the exposed FILE*.)
Not since the discussions about migrating to VS 2015, but a few off the
top of my head:
* file descriptors
* stream buffers
* thread locals
* exception [handler] state (yes, there are exceptions used within the
CRT, and they occasionally intentionally leak out past the C code)
* atexit handlers
* internal callbacks (mostly debug handlers, but since we're talking
I'm pretty sure if I did some digging I'd be able to figure out which of
these come from vcruntime140.dll vs ucrtbase.dll, and then come up with
some far-too-clever linker options to make some of these more
consistent, but there's no complete solution other than making sure
you've got a complete debug or complete release build.
>> For the most part, disabling optimizations in your own extension but
>> using the non-debug ABI is sufficient, and if you're having to deal with
>> other people's packages then maybe you don't have any choice (though I
>> do know of people who have built debug versions of numpy before - turns
>> out Windows developers are often just as capable as non-Windows
>> developers when it comes to building things ;)
> I'm not sure why you think I was implying otherwise? I'm sorry if you
> thought I was attacking your users or something. I did say that I
> thought most users downloading the debug builds were probably confused
> about what they were actually getting, but I didn't mean because they
> were stupid Windows users, I meant because the debug builds are so
> confusing that even folks on the Python core team are confused about
> what they're actually getting.
"Our users", please :)
In my experience, Windows developers just treat debug and release builds
as part of the normal development process. The only confusion I've seen
has been related to CPython's not-quite-Windows-ish approach to debug
builds, and in practically every case it's been enough to explain
"release CPython uses a different CRT to your debug extension, but once
you align those it'll be fine".
I definitely *do not* want to force or encourage package developers to
release debug ABI versions of their prebuilt packages. But at the same
time I don't want to remove the benefits that debug builds currently
Basically, I'm happy with the status quo, and the users I talk to are
happy with it. So I'd rather not worry about optimising debug builds for
speed or memory usage. (It's a question of direction more than anything
else, and until we get some official statement of direction then I'll
keep advocating a direction based on my experiences ;) )