PyDev 2.3.0 released

PyDev 2.3.0 is out now.

See http://pydev.org for details :)

Preparing for PyDev 2.3.0

The major changes in this version will be the introduction of a pep8 analysis (it must be enabled in window > preferences > PyDev > Editor > Code Analysis > pep8) and scope selection/deselection (Shift+Alt+Up/Down).

The internal Jython version that PyDev uses was bumped from 2.1 to 2.2.1 to support pep8.py -- Jython 2.5 was discarded as an option because it became too big for PyDev (the PyDev distribution with Jython 2.2.1 has 7MB, if it went to Jython 2.5, it'd be around 14MB, so, I think it's not worth making the download twice as big just for this upgrade).

Also, as a plus, as Jython 2.2.1 is still based on the same grammar structure (JavaCC) that PyDev uses, so, I was able to give it a boost and provide some performance improvements that had been done to the PyDev grammar (Jython 2.5 uses Antlr, which seems to be one of the main reasons it became so big).

As a nice side-effect of this change, the PyDev startup seems to have improved somewhat (Jython 2.2.1 seems to start up faster than Jython 2.1), and now the Jython sources are included in the PyDev project (and not only a jython.jar), so, I can fine tune Jython itself for some PyDev use-cases (as it seems PyDev won't go for Jython 2.5 anyways).

There are also some critical bug-fixes in this release (an out of memory error when dealing with PyOpenGL, one cache issue which could make the pythonpath become wrong in memory and one deadlock were fixed).

So, anyone interested should get the current nightly build to see if all is working properly as an official release should be issued later this week (the current version should be the one released unless some critical issue is found in the meanwhile).

Making Eclipse look better on Ubuntu

The steps below are things to remember when I do a new Ubuntu install (so, I may revisit this when/if I reinstall Ubuntu again).

1. Tab bars in Eclipse are too big when run in Ubuntu:

There are 2 things for this:

1.1: Install Extended VS Presentation (update site: http://andrei.gmxhome.de/eclipse/)

1.2: Make menu bar smaller

Short version

gedit ~/.gtkrc-2.0

paste:

style "gtkcompact" {
GtkButton::default_border={0,0,0,0}
GtkButton::default_outside_border={0,0,0,0}
GtkButtonBox::child_min_width=0
GtkButtonBox::child_min_heigth=0
GtkButtonBox::child_internal_pad_x=0
GtkButtonBox::child_internal_pad_y=0
GtkMenu::vertical-padding=1
GtkMenuBar::internal_padding=0
GtkMenuItem::horizontal_padding=4
GtkToolbar::internal-padding=0
GtkToolbar::space-size=0
GtkOptionMenu::indicator_size=0
GtkOptionMenu::indicator_spacing=0
GtkPaned::handle_size=4
GtkRange::trough_border=0
GtkRange::stepper_spacing=0
GtkScale::value_spacing=0
GtkScrolledWindow::scrollbar_spacing=0
GtkTreeView::vertical-separator=0
GtkTreeView::horizontal-separator=0
GtkTreeView::fixed-height-mode=TRUE
GtkWidget::focus_padding=0
}
class "GtkWidget" style "gtkcompact"

Full explanation in http://ubuntuforums.org/showthread.php?t=1465712


2. Window buttons are not in the right:

Short version:
Alt+F2, gconf-editor, apps/metacity/general, button_layout, menu:minimize,maximize,close

Full version: http://www.howtogeek.com/howto/13535/move-window-buttons-back-to-the-right-in-ubuntu-10.04/


3. Set proper theme:

System > preferences > appearance (get online theme): Clearlooks Compact (customize colors: selected items: #969696 (150,150,150))


4. Set fonts

Copy Windows fonts from C:\Windows\Fonts to ~/.fonts (tahoma and consolas) -- yes, I really like those fonts :)

System > preferences > appearance: use tahoma 8 for app/doc/desktop, sans bold 8 for window and consolas 9 for monofont (in windows I use consolas 10, but it seems Linux will render the fonts a bit bigger, so, a smaller size is needed).

Rendering: subpixel smoothing.


By request, a picture of how it looks (click to see it full):

PyDev 2.2.4 released

PyDev 2.2.4 is now out.

Major changes include:

• Cython support

• Better handling for external files (especially the ones under a folder/zip file in the PYTHONPATH)

• Modules/packages appear in the globals browser (Ctrl+Shift+T)

• Some improvements in the debugger

Along with a bunch of other things (more details on the release may be gotten on http://pydev.org )

PyDev & Cython

The current PyDev nightly build has integrated Cython support :)

Instructions to get the nightly are available at: http://pydev.org/download.html

This is something I wanted to do quite some time already but hadn't found a good way to support Cython (mostly because the grammar support seems to be undocumented and all manually done).

So, I gave up for now on a full blown Cython parser and settled on parsing what needs to shown in the outline: classes, functions and Cython specifics: cdefs and ctypedefs -- without actually trying to do anything fancy with it (mostly showing things in the outline). It's still missing the actual scoping of things, so, items will appear flat in the outline -- but that should be fixed soon.

So, this means that code-analysis will gather much less things (right now I think it'll only gather mixed indentation analysis and duplication of function/classes).

As for the code-completion, it should be working for things that are context insensitive (i.e.: builtins, templates and global tokens for the auto-import). Some more testing needs to be done for the context-sensitive code-completion (not sure how much more can be done without a full blown parser behind it).

Aside from those shortcomings, most editor features should be working (anyone interested, please test it and let me know if there's something missing -- although I've used Cython on a few occasions, I'm hardly an expert in the area, so, feedback is much appreciated -- this should appear in a final release in a short time).

Note that to use the Cython editor, just open any .pyx file with the regular PyDev editor and things should work properly (it uses the regular PyDev editor, just customizing the parsing process).

PyDev 2.2.3 released

The new version is out already.

Major in this release were: a bugfix related to zip files (in short, the PyDev indexes related to zips got corrupted, so, any attempt to use a zip file in the PYTHONPATH wouldn't work properly), some performance improvements and a bunch of bug-fixes.

IPython & PyDev

In the current nightly build (see: http://pydev.org/download.html to get it), IPython can be used as the backend for the PyDev interactive console.

IPython is a Python shell that provides many improvements over the standard Python shell (more details may be found at http://ipython.org/ ).

To use it, install IPython (0.10.x or 0.11) in your Python interpreter (if you use easy-install, just do easy_install ipython), make sure that the IPython paths are properly added in your interpreter configuration inside of PyDev ( http://pydev.org/manual_101_interpreter.html ) and start up the interactive console in PyDev ( http://pydev.org/manual_adv_interactive_console.html ). If all is properly configured, the interactive shell should use IPython as the backend (if it's not found, it'll fallback to the standard PyDev shell and warn that IPython could not be found).

Another improvement in the shell is that the output for a long running command will now be printed while it's executing, and not only after it returns.

I'm waiting for feedback to know if there's something missing in the integration before a final release is provided with the integration...

Pydev 2.2.1 released

This was mainly a bug-fix release, but there were a couple of niceties added there:

1. The Quick Outline (Ctrl+O on editor) can now show the information from parents in the hierarchy (press Ctrl+O to toggle).

2. The extract local refactoring has an option to replace duplicates.

The full details may be seen at http://pydev.org

PyDev 2.2.0 released

It's out already.

This release is the first to add support for Eclipse 3.7 (which has just been released too).

The major feature added was the exception breakpoints handling (so, it's now possible to stop the debugger on caught or uncaught exceptions -- the UI for it may be accessed at the menu Run > Manage Python Exception Breakpoints).

The debugger should also be a bit faster (and some bugs related to multithreaded debugging were fixed).

For those using Django, some utilities were added so that it's possible to easily do a (remote) debug session with auto-reload enabled (see details at: http://pydev.org/manual_adv_remote_debugger.html#django-remote-debugging-with-auto-reload)

And as usual, a bunch of minor bugs were also fixed.

Override/implements templates on PyDev

One thing I miss when working on Python is a way to let people know when some method is overridden (and having it raise an error if it's not)... I'm thinking about adding a decoration for that in PyDev (and probably a way to navigate to the super with that implementation), so, that'll work when using PyDev, but still, many times I look to the source code outside of PyDev (i.e.: review board, github, etc), and to sort that out, what I'm currently using is the following:

1. Create overrides (for when a method is overridden from a subclass) and implements (to indicate that some method conforms to an interface, even if not actually overriding a method) decorators as follows:



2. Create templates (in PyDev) as follows:

Name: overrides
Description: @overrides
Pattern: @overrides(${superclass}.${next_class_or_method})

Name: implements
Description: @implements
@implements(${interface_class}.${next_class_or_method})


3. Use it in your code:



To use it, just start typing 'overrides' before a 'def method' and apply the completion (and it should complete the class/method properly on overrides -- for the implements, the name of the implementing class will still be needed later on).

PyDev 2.1.0 released

PyDev 2.1.0 has just been released, with a few noteworthy changes:

The first change that's noteworthy is that only files opened in an editor will be code-analyzed... Until now, any file changed in the workspace would be analyzed for coding errors, which is a process that is usually fast, but there are 2 problems there:

1. if lots of files changed at once, it could still be costly to make that analysis (and on really big files, that could be a problem too)

2. there's no dependency analysis to know if a file should be re-analyzed when a given file changes, so, the actual state of warnings/errors could be misleading

Still, as sometimes it's useful to do the analysis of many files at once, it's now possible to right-click a folder in the PyDev package explorer and ask for a full code-analysis to be done (note that the re-indexing of internal information still must be done for any file changed, but this process is pretty fast).

Another nice feature added is a code-completion to override methods, so, after typing "def ", a code-completion may be requested to override a given method.

And for TDD lovers, there's now an option to relaunch the last launch in the PyUnit view whenever a file changes :)

The full details on the release may be seen at: http://pydev.org/

PyDev: Paying down technical debt...

Yeap, pay time :)

The next release is taking a bit more than usual because I'm taking time to pay some of the technical debt that PyDev accumulated over the years...

Not that this will also reflect in a bunch of things that will be noticed externally (e.g.: faster build times) and there are some features added too, such as an override methods code-completion...

The current nightly is already stable (although there are some more changes that are still planned), so, anyone curious is welcome to grab the latest nightly and take it for a spin...

Profiling slow downs on Eclipse

Java has this very nice utility that ships with the JDK called Java VisualVM which can provide many insights about what may be happening inside a java vm.

The steps for verifying are:

1. Have the actual java JDK installed (not only the jre) -- preferably the latest java 6.

2. Start Eclipse with -Dcom.sun.management.jmxremote vmarg (edit the eclipse.ini file and add a line -Dcom.sun.management.jmxremote after the line -vmargs -- at least for Eclipse 3.6.x -- and restart it)

3. Go on with your life until you get to something that's slow.

4. When you get up to that point, start up the Java VisualVM: it's an executable usually located at: JAVA_HOME\bin\jvisualvm.exe

5. Double click the java process running Eclipse (may appear as <Unknown Application>)

6. Go to the 'monitor' tab and check if the Heap/PermGen is too close to the max size (if it is, probably the -Xmx argument must be raised in eclipse.ini -- if you feel it shouldn't be taking up so much memory -- i.e.: there's a memory leak, click the 'heap dump', which may be a big file, zip it, upload it somewhere and point to it in a bug report).

7. If the memory seems fine, the next step is seeing up the threads running and creating a thread dump and right after it, go to the sampler, click on cpu to get sample data and add that data to a bug report (and attach both the thread dump and the sampler info to a bug report).

And hopefully, that's enough information to know what may be wrong :)

PyDev 2.0

First the latest news: PyDev 2.0 is released.

The main features introduced are:

* TDD actions similar to those available in the Eclipse Java Development Tools (i.e.: create class, method, field, module with Ctrl+1 quick fixes);

* Code-coverage support has improved a lot (note that it relies on having the coverage module from http://nedbatchelder.com/code/coverage/ installed).

The full release notes may be found at: http://pydev.org

Click the image below to see a short video showing these additions -- among a few other things that can be handy for those aiming to be productive using PyDev...



Unfortunately I couldn't be at PyCon to represent PyDev this year -- where there was a panel on Python IDEs -- so, at least I got to do a video with the presentation I'd do there :)

Another nice thing is that PyDev won the “Best Developer Tool” award from the Eclipse Foundation!

PyDev is a finalist in the Eclipse Community Awards

Actually, this is something that's already old news (announced more than a week ago) but I couldn't manage to get time to post about it sooner, as I'm pretty busy coding for the next PyDev release ;)

PyDev has been named one of the finalists for the Best Developer Tool in the Eclipse Community Awards (the other finalist in this category is Diver).

So, just wanted to share the good news for PyDev :)

Git on the command line

[Update]: When dealing with multiple repositories I now use what's described in: http://pydev.blogspot.com.br/2012/07/git-multiple-repos.html

Yes, sometimes I still use the command line for git (even with EGit getting better).

This is an update to a previous post about git (with msysgit).

[Update] October 31st, 2011: Improved gitdd command a bit
[Update] February 6th, 2012: Adedd command to preview incoming changes on a branch

1. Configure running shell

Usually I use TCC/LE as a shell, so, there's an alias to start the git bash shell from it (tcc allows editing the a 4START.BAT which will run when it's started, so, the configuration may be added there):

alias gitshell=path_to_git/bin/sh.exe --login -i

Other aliases:

alias git1=git log --graph --oneline --decorate

Shows the log with nice graph.

1a Use pageant/plink so that the password does not need to be entered all the time

Download pageant/plink, set environment variable GIT_SSH to the plink location, start pageant and load the private ssh key with pageant.

1b Global git configurations

git config --global user.name "fabioz"
git config --global core.autocrlf input

2. Fix history behavior

I find it annoying that the default history completion behavior (when up/down is used) is always getting the previous command instead of completing based on what's already there. To fix that, create a .inputrc file in your user home directory and set the following contents to it:

## By default up/down are bound to previous-history
## and next-history respectively. The following does the
## same but gives the extra functionality where if you
## type any text (or more accurately, if there is any text
## between the start of the line and the cursor),
## the subset of the history starting with that text
## is searched (like 4dos for e.g.).
## Note to get rid of a line just Ctrl-C
"\e[B": history-search-forward
"\e[A": history-search-backward

$if Bash
## F10 toggles mc on and off
## Note Ctrl-o toggles panes on and off in mc
"\e[21~": "mc\C-M"

##do history expansion when space entered
Space: magic-space
$endif

## Include system wide settings which are ignored
## by default if one has their own .inputrc
$include /etc/inputrc


3. Diff changes

Using mu-repo (https://github.com/fabioz/mu-repo/) with WinMerge.

3a. WinMerge tree view

I also always like to see things as a tree in WinMerge (Menu view > Mode: Alt+V M) and also expanded (Menu view > Expand All Subfolders: Alt+V X). It'll store the configuration to show as tree, but unfortunately it needs to be expanded all the times when doing a compare (i.e.: no auto-expand).

4. Show things in a compact way

git config format.pretty "%h %ct %ad %Cgreen%aN%Creset %s"
git config log.date short

#Make git log show comments wrapped.
git config --global core.pager 'less -r'

5. The commands used to get the contents of a pull request (create local branch, merge, get dev, merge with dev):

git checkout -b PullFromBranch-dev development
git pull https://github.com/pull_from_user/Pydev.git PullFromBranch
git checkout development
git merge PullFromBranch-dev --no-commit --no-ff

Then, to accept the merge do a commit or to reject it do:
git merge --abort
or
git merge --reset


6. When creating a feature in a branch:

git checkout -b FeatureBranch-dev development
git checkout development
git pull FeatureBranch-dev --no-ff


7. Preview incoming changes:

Create a bash file with the contents below (usually just hardcoding the ${current-branch} to the development branch). The command below will create 2 branches from the current branch, update one of those, diff it, compare with external tool and revert changes again.

echo Saving local changes just in case... git stash apply may be done later!
git stash
git checkout -b preview-branch
git checkout -b preview-branch2
git pull origin ${current_branch}
git checkout preview-branch
git merge --no-ff --no-commit preview-branch2
gitdd
git reset --hard
git checkout ${current_branch}
git branch -D preview-branch
git branch -D preview-branch2


8. Common commands

git status
git commit -a -m "Message"
git push origin master
git checkout
git shortlog -4 -w
git log -n 6
git log -n 3 --format=full
git commit --amend (change last commit message)
git show (see what happened on a commit)

And some of the commands I had to discover how to use in the msysgit bash are:

Alt+Space+E+K: mark contents for copy
Insert: paste contents
Alt+Backspace: erase until whitespace


Doing a merge:

Steps: create a new branch based on the current development, pull the merge request into that branch, then checkout the development branch, merge it there without commiting (to do a proper review) and then accept or reject the merge.

git checkout -b branch_to_do_merge_dev development
git pull https://github.com/user/Pydev.git jonahkichwacoders:branch_to_do_merge
git checkout development
git merge branch_to_do_merge_dev --no-commit --no-ff

Then, to accept the merge do a commit to reject it do:
git merge --abort
or
git merge --reset

Pydev 1.6.5 released

Pydev 1.6.5 is already available for download.

The major features added in this released where:

1. Code generation for classes and methods

* It's now possible to select a given token and press Shift+Alt+S C to create a class for it or Shift+Alt+S M to create a method

* In some undefined variable, do Ctrl+1 and select create method or create class.

Note that this is an initial implementation of the idea, aimed as those that use a TDD (Test Driven Development) approach, so, one can create the test first and generate the classes/methods later on from using shortcuts or quick-fixes (which is
something that those using JDT -- Java Development Tools -- in Eclipse should be already familiar with). This feature should be already usable on a number of situations but it's still far from being 100% complete.

2. Syntax highlighting is now available for (), [], {} and operators.

Aside from those, a number of bug-fixes were also made available (particularly nasty was a bug that prevented the interactive console from working with some of the Python 3.x versions).

Appcelerator, Aptana & Pydev

Well, as publicized already, Aptana was acquired by Appcelerator (http://www.appcelerator.com/appcelerator-acquires-aptana/).

So, I thought I'd write a bit about it...

First thing to note is that (as the rest of the team) I've joined Appcelerator, and the plan is to keep Pydev going forward (as it has been going since I had joined Aptana). However, unlike the rest of the team, I'm still based on Brazil :)

I think that the only difference in focus is that now, aside from the support for Django and Google App Engine, Titanium (which is Appcelerator's platform for developing desktop and mobile applications) will also have some special attention, although I still don't have more details on what exactly this means...

All in all, I think this was a good thing, as one of the things that Aptana was looking for was something to support (as the plans were always about giving Aptana Studio for free and making money on something else), and Appcelerator will have a nice improvement in terms of a better coding environment :)

Pydev 1.6.4 released

Pydev 1.6.4 is now available.

The major change is the new unittest support, which was previously commented at: http://pydev.blogspot.com/2010/12/improved-unittest-support-in-pydev.html

Now, there were some other changes since that post that are worth mentioning too:

The default Pydev test runner can now distribute tests among multiple processes, which can make testing much faster (if you have multiple CPUs available). Although note that if the tests weren't properly designed they may break. One example would be the case when 2 tests end up writing/reading to the same file.

Note that there's also an option to run all the tests from a given module in the same process (instead of separating them 'randomly' among processes).

See: http://pydev.org/manual_adv_pyunit.html for details on the flags to be passed to enable running in multiple processes and basic instructions on how to use the new unittest integration.

Also important is that a major bug was fixed -- in this bug, existing interpreters could end up being corrupted when a new interpreter was added (so, if you are using multiple interpreters you're strongly advised to upgrade) -- note: if there seems to be some issue with corruption of the internal pydev indexes, ctrl+2 --reindex can be used (while in the pydev editor) to ask for the pydev indexes to be rebuilt (which would be the case if the globals browser is not working properly or the pydev code analysis was manually stopped for some reason) it shouldn't be needed, but, if the need arises, it's there :)