git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Python-Dev] Have a big machine and spare time? Here's a possible Python bug.


There's a Stackoverflow report[1] I suspect is worth looking into, but
it requires far more RAM (over 80GB) than I have).  The OP whittled it
down to a reasonably brief & straightforward pure Python 3 program.
It builds a ternary search tree, with perhaps a billion nodes.  The
problem is that it "takes forever" for Python to reclaim the tree
storage when it goes out of scope (the author waited at least hours).

Alas, the OP said it takes about 45 minutes to build the tree, and the
problem goes away if the tree is materially smaller.  So it takes a
long time just to try once.  With a tree about 10x smaller, for me it
takes about 45 seconds for Python to reclaim the tree storage.

The tree is deep enough that the "trashcan" may be implicated, and the
node objects are small enough that obmalloc carves them out of a
(relatively) great many arenas.  Those are two ways in which Python
_may_ be to blame.  The sheer number of objects involved may be
provoking edge cases we normally never see.

But, for a start, it would be good to know if anyone else can actually
reproduce the problem.

[1] https://stackoverflow.com/questions/56228799/python-hangs-indefinitely-trying-to-delete-deeply-recursive-object