Allow spies to replicate function signatures for introspection.

Review Request #9327 — Created Oct. 25, 2017 and submitted

Information

kgb
master
fe3e42c...

Reviewers

kgb

Attempting to use functions like inspect.getargspec() with function or
method spies would result in all sors of failures, depending on what the
spy was representing. Method spies didn't have any of the attributes
needed for introspection (like func_defaults) and didn't identify as a
method (preventing inspect.ismethod() from working). Function spies
had the attributes, but the signature of the forwarding function didn't
match that of the original function, negating the usefulness of
getargspec().

This change fixes all that by better impersonating the functions being
spied on.

When spying on a method, a FunctionSpy instance will report its class
as types.MethodType, allowing isinstance(spy, types.MethodType)
(and, by extension, inspect.ismethod()) to work. This is the case for
Python 2.x and 3.x. It's not really important for FunctionSpy to
report its true class, so this is completely fine to do.

Method spies also forward any attribute accesses not otherwise handled
by the spy up to the original function, allowing func_defaults and
others to work.

These two things combined allow getargspec() to work for method spies.
Function spies, though, are a whole different beast.

For function spies, we override the bytecode and some other data for the
function, since we can't replace the function's instance itself. In the
past, the new method just accepted *args, **kwargs and passed those
on, but this meant that the function's signature wouldn't show any
specific argument names, breaking introspection.

To fix that, we now dynamically build the function definition using
exec() and inspect.formatargspec(). The resulting definition exactly
matches the positional and keyword arguments of the function being spied
on (with the exception that the keyword arguments' default values are
set to a special value used to identify provided vs. default keyword
arguments -- this does not break introspection as getargspec() reads
the original function's func_defaults).

However, the forwarding function still needs to call the original
function in exactly the same way that it was called, in order for
argument tracking to continue to work on the spy. To do this, we grab
the caller's frame from the current frame and inspect the bytecode of
the call (which is available to us in the frame data). The values of a
function call's opcode are the number of positional and keyword
arguments provided. The positional argument data allows us to determine
how many of the positional arguments in the function signature need to
be passed to the original function. We then check all keyword arguments
to see if any were provided and pass those as well. Finally, if the
signature takes *args and/or **kwargs, those are also passed.

To make this work, the forwarding function needs to build the call using
another exec(), meaning we have two layers of nested exec()s. Which
is excellent.

This allows both types of spies to accurately represent the function
signatures and faithfully forward on the signature of the call, allowing
introspection to fully work.

Note that it's now more important that function signatures for fake
functions match the original function. That is, a fake function expected
to be called with 3 positional arguments should retain the same names
and should include *args and **kwargs if they exist in the original
function.

All kgb unit tests pass, including new tests that pass arguments in
different orders and run getargspec.

Ran the test suites of several large projects that use kgb. All tests
pass, including those that previously failed due to getargspec use.

david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to master (a024f3d)