[Python-Dev] PEP 590 discussion
On 3/30/19 11:36 PM, Jeroen Demeyer wrote:
> On 2019-03-30 17:30, Mark Shannon wrote:
>> 2. The claim that PEP 580 allows "certain optimizations because other
>> code can make assumptions" is flawed. In general, the caller cannot make
>> assumptions about the callee or vice-versa. Python is a dynamic language.
> PEP 580 is meant for extension classes, not Python classes. Extension
> classes are not dynamic. When you implement tp_call in a given way, the
> user cannot change it. So if a class implements the C call protocol or
> the vectorcall protocol, callers can make assumptions about what that
>> PEP 579 is mainly a list of supposed flaws with the
>> 'builtin_function_or_method' class.
>> The general thrust of PEP 579 seems to be that builtin-functions and
>> builtin-methods should be more flexible and extensible than they are. I
>> don't agree. If you want different behaviour, then use a different
>> object. Don't try an cram all this extra behaviour into a pre-existing
> I think that there is a misunderstanding here. I fully agree with the
> "use a different object" solution. This isn't a new solution: it's
> already possible to implement those different objects (Cython does it).
> It's just that this solution comes at a performance cost and that's what
> we want to avoid.
It does seem like there is some misunderstanding.
PEP 580 defines a CCall structure, which includes the function pointer,
flags, "self" and "parent". Like the current implementation, it has
various METH_ flags for various C signatures. When called, the info from
CCall is matched up (in relatively complex ways) to what the C function
PEP 590 only adds the "vectorcall". It does away with flags and only has
one C signatures, which is designed to fit all the existing ones, and is
well optimized. Storing the "self"/"parent", and making sure they're
passed to the C function is the responsibility of the callable object.
There's an optimization for "self" (offsetting using
PY_VECTORCALL_ARGUMENTS_OFFSET), and any supporting info can be provided
as part of "self".
>> I'll reiterate that PEP 590 is more general than PEP 580 and that once
>> the callable's code has access to the callable object (as both PEPs
>> allow) then anything is possible. You can't can get more extensible than
Anything is possible, but if one of the possibilities becomes common and
useful, PEP 590 would make it hard to optimize for it.
Python has grown many "METH_*" signatures over the years as we found
more things that need to be passed to callables. Why would
"METH_VECTORCALL" be the last? If it won't (if you think about it as one
more way to call functions), then dedicating a tp_* slot to it sounds
In one of the ways to call C functions in PEP 580, the function gets
- the arguments,
- "self", the object
- the class that the method was found in (which is not necessarily
I still have to read the details, but when combined with
LOAD_METHOD/CALL_METHOD optimization (avoiding creation of a "bound
method" object), it seems impossible to do this efficiently with just
the callable's code and callable's object.
> I would argue the opposite: PEP 590 defines a fixed protocol that is not
> easy to extend. PEP 580 on the other hand uses a new data structure
> PyCCallDef which could easily be extended in the future (this will
> intentionally never be part of the stable ABI, so we can do that).
> I have also argued before that the generality of PEP 590 is a bad thing
> rather than a good thing: by defining a more rigid protocol as in PEP
> 580, more optimizations are possible.
>> PEP 580 has the same limitation for the same reasons. The limitation is
>> necessary for correctness if an object supports calls via `__call__` and
>> through another calling convention.
> I don't think that this limitation is needed in either PEP. As I
> explained at the top of this email, it can easily be solved by not using
> the protocol for Python classes. What is wrong with my proposal in PEP
> 580: https://www.python.org/dev/peps/pep-0580/#inheritance
I'll add Jeroen's notes from the review of the proposed PEP 590
The statement "PEP 580 is specifically targetted at function-like
objects, and doesn't support other callables like classes, partial
functions, or proxies" is factually false. The motivation for PEP 580 is
certainly function/method-like objects but it's a general protocol that
every class can implement. For certain classes, it may not be easy or
desirable to do that but it's always possible.
Given that `PY_METHOD_DESCRIPTOR` is a flag for tp_flags, shouldn't it
be called `Py_TPFLAGS_METHOD_DESCRIPTOR` or something?
Py_TPFLAGS_HAVE_VECTOR_CALL should be Py_TPFLAGS_HAVE_VECTORCALL, to be
consistent with tp_vectorcall_offset and other uses of "vectorcall" (not
And mine, so far:
I'm not clear on the constness of the "args" array.
If it is mutable (PyObject **), you can't, for example, directly pass a
tuple's storage (or any other array that could be used in the call).
If it is not (PyObject * const *), you can't insert the "self" argument in.
The reference implementations seems to be inconsistent here. What's the