This week I needed to do a profile session and usually on Python I just use the profile module and dump the output stats (of cProfile) in textual mode (which is usually enough to find out about the problem).
Now, this week I had to do some profiling which demanded a bit more, so, researching a bit, it seems that graphviz (http://www.graphviz.org/) can be used to plot the results of the profile session output with the help of gprof2dot (http://gprof2dot.jrfonseca.googlecode.com/git/gprof2dot.py).
So, using the code below (gist from https://gist.github.com/fabioz/8314370), it's possible to profile a function and have a graphical (.svg) output with the results of the profile (besides the usual text output, which I usually save temporarily during a profile session to compare the results from subsequent optimizations).
Hopefully the docstring explains how to use it properly (as well as its dependencies):
Note that it relies on having a .svg viewer installed (I had Inkscape: http://www.inkscape.org/ installed, so, I'm just using it, but there may be better .svg viewers around).
Happy profiling!
Wednesday, January 08, 2014
Monday, December 30, 2013
PyDev 3.2.0 released
The new PyDev release is out.
For those using 3.1.0, the upgrade is recommended as this version had some compile issues which made Ctrl+1 to create classes/methods fail on some situations.
The focus on this build was mostly on bugfixing and performance/memory enhancements, so, if you're interested in getting a faster PyDev, the upgrade is recommended :)
Python stackless debugger integration is also much improved.
In the debugger, there's also a new feature, where running the debugger with catch caught exceptions turned on will now show a view with the caught exception stack (indicating the current frame), and it's possible to ignore that same exception afterwards by right-clicking the stack in that view and choosing the related option (note that handling caught exceptions in the debugger may be enabled in the debug perspective > pydev > manage exception breakpoints).
See: http://pydev.org for more details on the release.
p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)
p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).
For those using 3.1.0, the upgrade is recommended as this version had some compile issues which made Ctrl+1 to create classes/methods fail on some situations.
The focus on this build was mostly on bugfixing and performance/memory enhancements, so, if you're interested in getting a faster PyDev, the upgrade is recommended :)
Python stackless debugger integration is also much improved.
In the debugger, there's also a new feature, where running the debugger with catch caught exceptions turned on will now show a view with the caught exception stack (indicating the current frame), and it's possible to ignore that same exception afterwards by right-clicking the stack in that view and choosing the related option (note that handling caught exceptions in the debugger may be enabled in the debug perspective > pydev > manage exception breakpoints).
See: http://pydev.org for more details on the release.
p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)
p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).
Thursday, December 12, 2013
PyDev 3.1.0 released
PyDev 3.1.0 is now already available for download.
The debugger got a lot of attention in this release:
One of my favorite features is the code reloading. This means that when you change a file in the PyDev editor and there's a debug session going on, PyDev will automatically reload that module.
There are some caveats on doing that:
1. It'll try to reload the objects code in place (without changing references), but the code you're currently using in a frame in the current stack cannot be changed, so, you'll have to get out of the function and then back to see the changes (the set next line feature in the debugger which allows you to set the next line to be executed in the current scope can be very helpful for that).
2. Not everything can be cleanly reloaded (see: https://github.com/fabioz/Pydev/blob/development/plugins/org.python.pydev/pysrc/pydevd_reload.py for the limitations on xreload).
Still, it's very nice that you can now do the reload and it does work well on many situations ;)
Another nice thing is that it's now possible to inspect which are the referrers of a given variable by right-clicking it on the variables or expressions view and selecting 'get referrers'.
One last thing I'd like to mention is that Stackless Python is now supported in the debugger, which will show the tasklets as if they were threads in the stack view.
Besides the debugger, the rename refactoring also got some love, so, it's now possible to rename a module/package having all references updated as well. Also, the find references (Ctrl+Shift+G) had some bugfixes and should be more reliable now.
There are still a number of other nice things, so, check the release details on http://pydev.org for more information.
p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)
p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).
The debugger got a lot of attention in this release:
One of my favorite features is the code reloading. This means that when you change a file in the PyDev editor and there's a debug session going on, PyDev will automatically reload that module.
There are some caveats on doing that:
1. It'll try to reload the objects code in place (without changing references), but the code you're currently using in a frame in the current stack cannot be changed, so, you'll have to get out of the function and then back to see the changes (the set next line feature in the debugger which allows you to set the next line to be executed in the current scope can be very helpful for that).
2. Not everything can be cleanly reloaded (see: https://github.com/fabioz/Pydev/blob/development/plugins/org.python.pydev/pysrc/pydevd_reload.py for the limitations on xreload).
Still, it's very nice that you can now do the reload and it does work well on many situations ;)
Another nice thing is that it's now possible to inspect which are the referrers of a given variable by right-clicking it on the variables or expressions view and selecting 'get referrers'.
One last thing I'd like to mention is that Stackless Python is now supported in the debugger, which will show the tasklets as if they were threads in the stack view.
Besides the debugger, the rename refactoring also got some love, so, it's now possible to rename a module/package having all references updated as well. Also, the find references (Ctrl+Shift+G) had some bugfixes and should be more reliable now.
There are still a number of other nice things, so, check the release details on http://pydev.org for more information.
p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)
p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).
Thursday, November 07, 2013
PyDev 3.0!
PyDev 3.0 has just been released.
This is the first version which breaks compatibility with older Eclipse and Java versions, so, in order to run it requires Eclipse 3.7 onwards and Java 7 (for older versions, keep using PyDev 2.x) -- check LiClipse (http://brainwy.github.io/liclipse/), if you want a hassle free install where things should 'just work'.
This release has 2 main improvements: usability improvements related to configuring an interpreter and several enhancements in the interactive console.
On the interpreter configuration side, there were several improvements, but I believe the major change is that when the files related to the interpreter are changed in the filesystem (i.e.: when pip-installing a library for instance), PyDev will automatically get that the interpreter configuration needs to be updated (previously a manual step was required in order to add that path to the PyDev interpreter configuration). This was the most voted issue on the tracker and I believe one of the sources of greater frustration for PyDev users -- and it's finally solved! (which for me is the cherry in this release and warrants the newly-acquired 3.0 status!)
As for the interactive console, it has many noteworthy enhancements (and more enhancements are expected in the upcoming versions):
1. The interactive console can now execute in debug mode. By enabling this feature in the preferences, it's possible to place breakpoints in code, execute some code in the interactive console and stop at that breakpoint (which is really nice).
2. User Module Deleter (UMD): By enabling UMD in the preferences (i.e.: using runfile instead of execfile to run a python module), dependencies imported when executing some code are reimported (so, changes in files are automatically gotten without requiring manual reload() calls).
3. IPython 1.0 is now supported. Also, %edit will open the file in the PyDev editor and %gui customizes the gui event loop integration (i.e.: %gui wx to enable wxPython backend).
More details on the interactive console can be seen at: http://pydev.org/manual_adv_interactive_console.html
Other features have also been added, including better handling of numpy arrays in the debugger, right-clicking a variable in the debugger > pretty print to pretty-print it, persistent history in the interactive console -- besides some important fixes.
The last thing is that LiClipse (http://brainwy.github.io/liclipse/) has also been released with the latest changes in PyDev 3.0 and now also supports Mako Templates and Dart (besides the languages it already supported, such as Django Templates, Javascript, HTML, C/C++, etc.)
This is the first version which breaks compatibility with older Eclipse and Java versions, so, in order to run it requires Eclipse 3.7 onwards and Java 7 (for older versions, keep using PyDev 2.x) -- check LiClipse (http://brainwy.github.io/liclipse/), if you want a hassle free install where things should 'just work'.
This release has 2 main improvements: usability improvements related to configuring an interpreter and several enhancements in the interactive console.
On the interpreter configuration side, there were several improvements, but I believe the major change is that when the files related to the interpreter are changed in the filesystem (i.e.: when pip-installing a library for instance), PyDev will automatically get that the interpreter configuration needs to be updated (previously a manual step was required in order to add that path to the PyDev interpreter configuration). This was the most voted issue on the tracker and I believe one of the sources of greater frustration for PyDev users -- and it's finally solved! (which for me is the cherry in this release and warrants the newly-acquired 3.0 status!)
As for the interactive console, it has many noteworthy enhancements (and more enhancements are expected in the upcoming versions):
1. The interactive console can now execute in debug mode. By enabling this feature in the preferences, it's possible to place breakpoints in code, execute some code in the interactive console and stop at that breakpoint (which is really nice).
2. User Module Deleter (UMD): By enabling UMD in the preferences (i.e.: using runfile instead of execfile to run a python module), dependencies imported when executing some code are reimported (so, changes in files are automatically gotten without requiring manual reload() calls).
3. IPython 1.0 is now supported. Also, %edit will open the file in the PyDev editor and %gui customizes the gui event loop integration (i.e.: %gui wx to enable wxPython backend).
More details on the interactive console can be seen at: http://pydev.org/manual_adv_interactive_console.html
Other features have also been added, including better handling of numpy arrays in the debugger, right-clicking a variable in the debugger > pretty print to pretty-print it, persistent history in the interactive console -- besides some important fixes.
The last thing is that LiClipse (http://brainwy.github.io/liclipse/) has also been released with the latest changes in PyDev 3.0 and now also supports Mako Templates and Dart (besides the languages it already supported, such as Django Templates, Javascript, HTML, C/C++, etc.)
Thursday, October 03, 2013
First LiClipse public release
The crowdfunding at the start of this year (http://igg.me/at/liclipse) had 2 targets, one was keeping the PyDev development going (which is going strong), and the other was creating a base to complement PyDev when dealing with other languages and providing some enhancements in the form of another plugin (LiClipse) which I thought were crucial to keep the Eclipse ecosystem healthy (and in return PyDev too).
Well, today I've just made the first LiClipse public release (it was in a closed Alpha so far). For me it's now the easiest way to get started on PyDev (as LiClipse bundles PyDev and has native standalone installers for Windows/Mac OS/Linux), besides providing support for common languages which often go with Python development (such as HTML, JavaScript, CoffeScript, Django Templates, RST, etc.), improved theming support and multi-edition (which may seem a minor thing, but it does make a number of workflows much more straightforward to me, so, I can hardly think about developing without it anymore).
More details on: http://liclipse.blogspot.com.br/2013/10/first-liclipse-public-release.html and its homepage: http://brainwy.github.io/liclipse/
Enjoy!
Well, today I've just made the first LiClipse public release (it was in a closed Alpha so far). For me it's now the easiest way to get started on PyDev (as LiClipse bundles PyDev and has native standalone installers for Windows/Mac OS/Linux), besides providing support for common languages which often go with Python development (such as HTML, JavaScript, CoffeScript, Django Templates, RST, etc.), improved theming support and multi-edition (which may seem a minor thing, but it does make a number of workflows much more straightforward to me, so, I can hardly think about developing without it anymore).
More details on: http://liclipse.blogspot.com.br/2013/10/first-liclipse-public-release.html and its homepage: http://brainwy.github.io/liclipse/
Enjoy!
Subscribe to:
Posts (Atom)