So, for instance, if you have a frame (which you could get from a traceback, sys._getframe().f_back, etc), you could get its locals with frame.f_locals, but changing the frame.f_locals (which gives you a dictionary) wouldn't apply the results back to the frame.
This is mostly due to how CPython works: frame.f_locals actually creates a dictionary using PyFrame_FastToLocals, but changes to the dictionary aren't applied back.
Some years ago I had found a way to make it work (see: http://bugs.python.org/issue1654367) through a CPython function: PyFrame_FastToLocals, but up until recently, I thought it needed a modified version of CPython in order to work, now, recently I discovered ctypes can access a lot from the python api (through ctypes.pythonapi):
So, after changing frame.f_locals, it's possible to use ctypes to call PyFrame_LocalsToFast doing:
import ctypes
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame), ctypes.c_int(0))
A note: the second parameter (which may be 0 or 1) defines whether we want to erase variables removed from the dict (which would require 1) or not.
So, the PyDev debugger now incorporates this utility so that if you're running in CPython, it will properly change the variable in a scope when you change a variable :)
Note that this isn't compatible with other Python implementations (this is not something the language dictates how it should work -- probably the ideal would be making frame.f_locals writable).
Thank you for posting this! It's half a decade later but I was able to use your information to build a better breakpoint() hook for modern python.
ReplyDeleteThe key turned out to be calling LocalsToFast() before *each* update to a variable:
```
import ctypes
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
for k, v in f_locals.items():
frame.f_locals[k] = v
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
```
I found that doing it in the other order gave bizarre results. For example, if in my frame `a=1; b=2; c=3` then
```
frame.f_locals['a'] = 17
import ctypes
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
frame.f_locals['b'] = 29
import ctypes
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
frame.f_locals['c'] = 84
```
would produce `a=17; b=2; c=3`
Hi @kousu, if you want you can take a look at: https://github.com/fabioz/PyDev.Debugger/blob/master/_pydevd_bundle/pydevd_save_locals.py (after some discussions on this, Pypy added support for that too but under a different API and that code also handles Pypy).
ReplyDeleteThank you! I used this snippet to update generators on-the-fly in this little side project: https://github.com/alfiopuglisi/pipeline
ReplyDelete