The latest release of PyDev (12.0.0) is now available and it brings a really nice speed improvement for those who are already in Python 3.12! -- If you're a LiClipse user, it's now available in LiClipse 11.
The PyDev Debugger now uses sys.monitoring, which enables faster debugging (on my tests it can be up to 15 times faster than the version using sys.setttrace, depending on the use case -- kudos to Mark Shannon for PEP 669 😉).
It took me a while to cover the many scenarios that pydevd deals with, but the good thing is that most of the infrastructure available in pydevd didn't need changes (the debugger already had all the concepts needed as it already tried to trace only the frames which were actually needed, which definitely helped a lot as this is now pretty central in how to deal with the tracing using sys.monitoring)
Given that it's now out, I'll talk about how it works and some of the caveats when using it.
So, the first thing to note is that PEP 669 defines a bunch of callbacks which are now related to the code object (and not the frame as happened with sys.settrace) and when inside one of those callbacks the debugger can react to decide what should happen.
Some things that could happen could be pausing due to a breakpoint or a step instruction or deciding that the given code should not be traced again (by returning a DISABLE).
The reason it becomes faster than the previous approach is that the DISABLE is then considered by the Python interpreter which then bakes that DISABLE into the code when it's executing (up until sys.monitoring.restart_events() is called again). This does come with a big caveat though: if there are multiple threads running the program and one of those threads returns a DISABLE instruction then the related callback will actually be disabled for all the threads. Note that if DISABLE is not returned, the speed ends up being close to what was available with sys.settrace (which wasn't all that bad in pydevd already, but definitely a step down from what is now available).
This means that the debugger is really much faster when going for a breakpoint (because it can DISABLE many of the tracing instructions), but after a breakpoint is hit, if one thread is doing a step operation, then the speed reverts back to being close to the sys.settrace variant (the debugger can still DISABLE tracing for places it knows the user never wants to stop, but it cannot DISABLE the tracing for any code in any thread where the user may want to stop, because the thread which is stepping could be affected by a DISABLE in another thread which is not stepping, or at least the stepping must be considered for all threads even if a given thread hasn't really stopped and is not stepping).
Also, it's worth to mention that sys.monitoring has finer grained events vs. the ones available in sys.settrace, which is good as for instance, it's possible to get events from exceptions separate from events related to entering/returning from a function or lines, which helps the debugger in tracing only what's actually needed (the tracing debugger had to do lots of gymnastics to create a separate tracer just for exceptions when there would be no need to trace a given scope for lines, so, the new code ends up being both simpler and faster).
Note however that in pydevd I've done some other improvements and stepping should be more responsive even when using sys.settrace with older versions of Python!
p.s.: the attach to process is still not available for Python 3.12.
Note: back in the day other players which make use of pydevd such as JetBrains and Microsoft did step up to finance it, but at this point the only support it has is from backers in the community.
So, if you enjoy using pydevd please consider becoming a supporter... I've actually just setup sponsoring through GitHub sponsorships (https://github.com/sponsors/fabioz). Any takes on becoming one of the first backers there 😉?
