Configuring Kivy on PyDev

Kivy installation on PyDev is a bit more involved because it requires some environment variables to be set in order to work properly, so, I thought I'd give a step-by-step on where exactly to configure that on PyDev (especially now that LiClipse: http://brainwy.github.io/liclipse/ adds support to the Kivy Language).

Note: The details below use paths based on windows, but it should be similar in other platforms.

1. Download/extract Kivy (for this example D:\bin\Kivy-1.7.2-w32\)

2. Add interpreter in Window > Preferences > Pydev > Interpreters > Python Interpreter (point to: D:\bin\Kivy-1.7.2-w32\Python\python.exe).

Note: For PyDev 3.3.3 onwards, the easier way to go there is doing: Ctrl+3 and writing 'Python interpreter' to open that preferences page (and the same thing can be used to go to a view or even activate some action).

3. Add the Kivy directory to the PYTHONPATH for this interpreter (in the same Python Interpreter page > libraries > add folder > D:\bin\Kivy-1.7.2-w32\kivy)

4. Add 'kivy' to the 'forced builtins' (again in that same page > forced builtins).

5. Add the needed environment variables (in that same page > environment):

GST_REGISTRY = D:\bin\Kivy-1.7.2-w32\gstreamer\registry.bin

GST_PLUGIN_PATH = D:\bin\Kivy-1.7.2-w32\gstreamer\lib\gstreamer-0.10

PATH = D:\bin\Kivy-1.7.2-w32;D:\bin\Kivy-1.7.2-w32\Python;D:\bin\Kivy-1.7.2-w32\gstreamer\bin;D:\bin\Kivy-1.7.2-w32\MinGW\bin;%PATH%

Alternatively, instead of adding those manually to the environment, open a cmd.exe, execute D:\bin\Kivy-1.7.2-w32\kivyenv.bat and then start Eclipse (but then you have to remember to do that manually every time -- or add it to the system environment variables -- note that you have to remember to update it if you move it or upgrade kivy).

After that, it should be possible to go to the pydev package explorer, expand the interpreter node in the tree > system libs > examples, open some example main.py and open it, then, with the editor opened used F9 to run the example (you may have to select which project should be used to get the information on the PYTHONPATH to be used as it's running as an external file).

Changing the locals of a frame (frame.f_locals) and persisting results (with ctypes)

Up until now I didn't know of a proper way to change the locals of a frame (out of the normal execution flow in Python), so, when in the PyDev debugger changing a variable wouldn't always work.

So, for instance, if you have a frame (which you could get from a traceback, sys._getframe().f_back, etc), you could get its locals with frame.f_locals, but changing the frame.f_locals (which gives you a dictionary) wouldn't apply the results back to the frame.

This is mostly due to how CPython works: frame.f_locals actually creates a dictionary using PyFrame_FastToLocals, but changes to the dictionary aren't applied back.

Some years ago I had found a way to make it work (see: http://bugs.python.org/issue1654367) through a CPython function: PyFrame_FastToLocals, but up until recently, I thought it needed a modified version of CPython in order to work, now, recently I discovered ctypes can access a lot from the python api (through ctypes.pythonapi):

So, after changing frame.f_locals, it's possible to use ctypes to call PyFrame_LocalsToFast doing:

import ctypes

ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame), ctypes.c_int(0))

 

A note: the second parameter (which may be 0 or 1) defines whether we want to erase variables removed from the dict (which would require 1) or not.

So,  the PyDev debugger now incorporates this utility so that if you're running in CPython, it will properly change the variable in a scope when you change a variable :)

Note that this isn't compatible with other Python implementations (this is not something the language dictates how it should work -- probably the ideal would be making frame.f_locals writable).

New PyDev release: improved indexing, Kivy support on LiClipse, etc.

PyDev 3.3.3 is now released.

For this release, there are lots of enhancements in many areas (full details on http://pydev.org).

My favorite one is that PyDev will now index completions from compiled modules (i.e.: .pyd files or entries in the 'forced builtins' -- http://pydev.org/manual_101_interpreter.html has more details on what are forced builtins). This was a long due request but it required the many other incremental changes from recent releases to be feasible.

With that, the context-insensitive code completion (the one that'll automatically add an import for the token) will work for libraries such as PyQt, itertools, etc.

Another which is really nice is that in the debugger, changing a local variable works properly (until now, this did work sometimes, but usually it didn't, as it had shortcomings depending on what variable was changed and how it was put in the scope and whether it was the top frame). Now this works even when assigning a local in the Debug Console!

Also in the debugger, now it's possible to mark some functions that you want to ignore in the debugger with a comment: #@DontTrace and when stepping in the debugger will ignore those: this is a huge time saver when debugging to ignore paths which are just scaffolding (and many times get in the way during a debug session).

As for LiClipse, the Kivy language is now supported -- it has syntax highlighting, code-completion, outline and many other goodies you'd expect. Besides this, in the latest release (0.9.7) LiClipse users can benefit from mark occurrences in the created editors and the theming now applies to EGit views too (more details on http://brainwy.github.io/liclipse).

Now, this is just a brief and incomplete summary of the changes. Personally, I think that PyDev improved on so many things in this release that it's a must have update if you're a PyDev/LiClipse user (especially performance-wise) -- I almost thought about making it a version 4.0 coming from 3.2, but I just couldn't skip doing a 3.3.3 version :)

Profiling a method on Python

This week I needed to do a profile session and usually on Python I just use the profile module and dump the output stats (of cProfile) in textual mode (which is usually enough to find out about the problem).

Now, this week I had to do some profiling which demanded a bit more, so, researching a bit, it seems that graphviz (http://www.graphviz.org/) can be used to plot the results of the profile session output with the help of gprof2dot (http://gprof2dot.jrfonseca.googlecode.com/git/gprof2dot.py).

So, using the code below (gist from https://gist.github.com/fabioz/8314370), it's possible to profile a function and have a graphical (.svg) output with the results of the profile (besides the usual text output, which I usually save temporarily during a profile session to compare the results from subsequent optimizations).

Hopefully the docstring explains how to use it properly (as well as its dependencies):




Note that it relies on having a .svg viewer installed (I had Inkscape: http://www.inkscape.org/ installed, so, I'm just using it, but there may be better .svg viewers around).

Happy profiling!

PyDev 3.2.0 released

The new PyDev release is out.

For those using 3.1.0, the upgrade is recommended as this version had some compile issues which made Ctrl+1 to create classes/methods fail on some situations.

The focus on this build was mostly on bugfixing and performance/memory enhancements, so, if you're interested in getting a faster PyDev, the upgrade is recommended :)

Python stackless debugger integration is also much improved.

In the debugger, there's also a new feature, where running the debugger with catch caught exceptions turned on will now show a view with the caught exception stack (indicating the current frame), and it's possible to ignore that same exception afterwards by right-clicking the stack in that view and choosing the related option (note that handling caught exceptions in the debugger may be enabled in the debug perspective > pydev > manage exception breakpoints).

See: http://pydev.org for more details on the release.

p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)

p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).