Pydev and SoC 2006

It's official: Pydev will have a student working on Google's Summer of Code!!

Sean Handley will be working mainly on some debugger features and in the code-completion engine, doing some long time requested features.

I'll be mentoring him, so, we'll see how that goes -- I'll keep you informed ;-)

Sean will be working on a branch so that we can manage to work both without stepping on each other's toes, and I'll synch that branch to the HEAD from time to time. I know it'll probably be more work, but I hope it will be easier to manage things this way.

It was also pretty nice to see that pydev had a good support from the PSF (otherwise it would surely not get a project approved at the SoC -- and even better: it was among the top of those approved projects, which is quite an achievement given the number of good proposals submitted).

So, that's it... let the work begin ;-)

p.s. It's something very strange to me, being a part of the SUMMER of Code, since it's damn cold (at least for me) here in the south of Brazil! ;-)

Pydev Release 1.0.8

Pydev 1.0.8 has ben just released... A major bug triggered this release (that's why it's been issued in less than one day from the previous release).

Mainly, if you had a file that had a docstring at the global level with an empty line, it could get to a loop when adding a new-line to the document. This has been fixed and is already available for download.

Also, 2 other minors have been done for Pydev Extensions (but they surely would not be worth a release for them).

-- Fabio

Pydev release 1.0.7

Ok, the new release is out already.

This release focused mainly on bug-fixing and optimizations, so it had only few features added. This is actually a sort of release candidate for 1.1, that will be a major release, and will be the last release to support Eclipse 3.1 (after 1.1, releases will support Eclipse 3.2).

In pydev the only feature added was a contribution by Don Tailor with the Jython Scripting Engine (if you would like to create your own, you can take a look at http://www.fabioz.com/pydev/manual_articles_scripting.html)

And in pydev extensions, a mark-occurrences and rename-occurrences was added (as well as some options to make remote debugging more attractive).

As always, you can check details on http://www.fabioz.com/pydev and http://pydev.sourceforge.net/

Cheers,

Fabio

Pydev release 1.0.7

Ok, the new release is out already.

This release focused mainly on bug-fixing and optimizations, so it had only few features added. This is actually a sort of release candidate for 1.1, that will be a major release, and will be the last release to support Eclipse 3.1 (after 1.1, releases will support Eclipse 3.2).

In pydev the only feature added was a contribution by Don Tailor with the Jython Scripting Engine (if you would like to create your own, you can take a look at http://www.fabioz.com/pydev/manual_articles_scripting.html)

And in pydev extensions, a mark-occurrences and rename-occurrences was added (as well as some options to make remote debugging more attractive).

As always, you can check details on http://pydev.sourceforge.net/

Cheers,

Fabio

Pydev package explorer

No, currently no plans for it, but there are some good news...

One of the major reason pydev has no navigator is that eclipse still does not provide a way to do it easily, you'd have to create it from ground up, with the TreeViewer, etc.

Now, it appears that version 3.2 will have a commom base for creating it...

So, I decided to google for "eclipse Common Navigator" and got some nice results, which I'm posting to remind me of checking them once the Eclipse 3.2 final is released (and based on this results, I believe that the package explorer will be available shortly after that final release of Eclipse 3.2).

A good example on how to use this api can be found at:
http://scribbledideas.blogspot.com/2006/05/building-common-navigator-based-viewer.html

-- Fabio

Remote debugging getting smarter

There're 2 things that can get in the way when remote debugging some application (especially if it's in another computer):
- Not being able to find the source code
- Not being able to get the output

So, I've changed the debugger in Pydev to make it smarter to find some file if it does not have a 'direct' corresponding file (so, if in the server you have a file mod1.py in a path /usr/local/foo, you'll be able to specify that location in your computer -- say c:/projects/foo/mod1.py). Also, if it has only one file named that way in the workspace, it will open it directly (and if more are available, it will give you a choice over which file to open), and as a last resort, if it is not found in the worskpace, it will allow you to choose a file in the filesystem.

As for getting the output, 2 new parameters were introduced in the settrace function, so that you can specify: pydevd.settrace(stdoutToServer=True, stderrToServer = True). Through that, it will redirect all that goes in stdout and stderr to the debug server. Note that I've done it in such a way that it will still print it in the original stdout and stderr -- so that you don't loose your logging (if you have any).

This will be available for the Pydev Extensions release 1.0.7 (which should come out by monday or tuesday).

-- Fabio

Pydev Videos

I've done 2 videos about pydev. One is pretty close to the getting started tutorial, and explains how to configure pydev and use it to the point where you can debug some module and the other explains some 'power' features for pydev and pydev extensions such as code-completion, navigation utilities, code-analysis, quick-fixes...

Both have a little more than 6 minutes. They are hosted at showmedo.com and are recorded with voice (I guess you'll be able to tell that english is not my first language -- but still, I believe you should be able to understand it without any problems).

What's next?

Ok, I'm getting the new pydev release ready to go... There have been a number of bug-fixes, so 1.0.7 should be out soon.

I'm trying to get the 'mark-occurrences' feature out in the next release, but I'm not sure if I'll get it complete in time. Still, I might release it early (in a beta-state) if it turns out it'll take too long, as there have been many bug-fixes and speed enhancements that I think are already ready for the release, and it is not a critical thing, so, you can simply disable it if you want.

Another thing is that the sf cvs is (finally) up again, and, I've just synched the developers cvs with the last version (I'm not sure how long it'll take to sync with the non-developers cvs).

-- Fabio

Profiling Pydev

The first thing I profiled was the grammar used, since it is used among many places, I believe it to be the something that would have a good impact overall.

One of the things we end up doing when testing some stuff is embeeding some file -- say, an image -- into a .py file. Among those, we could get a 3-4 mb sized file, and pydev would take a long time to parse it, so, my first target was to speed that up.

Making 'general' changes thinking it will get better is usually a mistake, as you need a way to measure those impacts. I started then with a file with a couple of statements and a single 'huge' multiline string and made my target optimizing parsing that file faster.

After playing around a little in the parser I discovered that the actuall speed loss was not at the grammar itself, but at the Reader that should give the chars to the tokenizer. After looking at its code, you could see it allocateded lots of memory in the process, so, I decided to create another reader from scratch and with the help of some unit-tests, and the results were pretty impressive (for big files):

Parsing a huge .py file (3mb) it was taking about 4-5 minutes... now it only takes 2-3 seconds (yeah, the previous approach had an 'exponential' behaviour depending exclusively only on the size of some file, not to mention that it would make the garbage collector work a lot more).

This will be available for 1.0.7 -- But before I do release it, I'm still looking for other 'hotspots' to optimize ;-)

Fabio