How is frame evaluation used in pydevd?

First some background in frame evaluation:

Since Python 3.6, CPython has a mechanism which allows clients to override how it evaluates frames. This is done by changing PyThreadState.interp.eval_frame to a different C-function (the default being _PyEval_EvalFrameDefault). See: pydevd_frame_evaluator.pyx#L370 in pydevd (note that Cython is used there).

Note that this affects the Python runtime globally, whereas the regular Python tracing function -- set through sys.settrace() -- affects only the current thread (so, some of the caches for frame evaluation in pydevd are thread-local due to that).

How is this used in the debugger?

Well, the debugger doesn't really want to change how Python code is executed, but, there's another interesting side effect of the frame evaluation: it's possible to change the bytecode of the frame right before it's evaluated and CPython will interpret that bytecode instead of the original bytecode of the frame.

So, this works the following way: the frame evaluation function receives a PyFrameObject*, and at that point, the debugger checks the frame for existing breakpoints, if it has a breakpoint, it'll create a new code object which has a programmatic breakpoint (pydevd_frame_evaluator.pyx#L234) and change PyFrameObject.f_code to point to the new code object (pydevd_frame_evaluator.pyx#L358) -- when it reaches the programmatic breakpoint (pydevd_frame_tracing.py#L34), the regular (trace-based) debugger will kick in at that frame. Until that breakpoint is reached, frames are executed at full speed.

But if it runs at full speed, why is my program still running slower when using pydevd with frame evaluation?

Well, frames are executed at full speed, but, the debugger still adds some overhead at function calls (when it decides whether to add the programmatic breakpoint) and it also needs to add an almost no-op trace (pydevd_frame_evaluator.pyx#L95) function to sys.settrace -- which makes function calls slower too (this is needed because otherwise the debugger is not able to switch to the regular tracing by just changing the frame.f_trace as frame.f_trace is only checked when a tracing function is set for some thread through sys.settrace()). There are also some cases where it can't completely skip tracing for a frame even if it doesn't have a breakpoint (for instance, when it needs to break on caught exceptions or if it's stepping in the debugger).

It's interesting to note that even the regular (tracing) debugger on pydevd can run frames at full speed (it evaluates all frames and if a frame doesn't have a breakpoint the tracing for that frame will be skipped), the difference is that if a frame does have a breakpoint, that frame can run at full speed until it reaches the breakpoint in the frame eval mode, whereas in the regular mode each new line tracing event would need to be individually checked for a breakpoint.

If it just changes the bytecode, why use frame eval at all, can't you just change the bytecode of objects at a custom import hook? (which could have the benefit of avoiding the performance penalty of checking the frame on each new frame call)

There are 2 main reasons for that: the 1st is that breakpoints can change and when they change the frame evaluation would need to be completely shut down and only the tracing debugger would be valid from that point onwards (whereas right now, if breakpoints change, the tracing debugger kicks in for all the frames that were currently running but the frame evaluation debugger can still be used for new frames). The 2nd is that it can be hard to consistently do that if not just before frame evaluation (because user code can also change a method code and there are a number of corner cases to change the bytecode for live objects -- think of a function inside a function or decorated functions).

Note that this means that the debugger could probably get away with something simpler than frame evaluation and could potentially be applicable to other Python implementations (say, a different callback just before the frame is evaluated which allows to change the frame code... unfortunately it can't currently be done through the "call" event received by the trace function set by sys.settrace because at that point the frame is already being evaluated with the current code and at that point, even if it's changed, Python won't pick up that change).

That's it, hope you enjoyed pydevd using frame evaluation for debugging purposes 101 ;)