The pydev debugger can now make use of psyco if it's available in the environment (it's actually highly recommended... in tests a 50% improvement in the debugger speed was observed... mainly, all the tracing goes through 2 functions, so, speeding it was pretty straightforward -- after discovering how to integrate it).
Now, this relation is kind of strange: the pydev debugger can make use of psyco, but the debugged code can't (because if it gets compiled, pydev can't trace it).
Until now, a Null() object was added as a psyco module in sys.modules so that clients didn't worry about it, but that had some nasty side effects, so, this version will also change this integration so that clients that rely on psyco don't have to change code: a stub module will replace the psyco module in sys.modules and will mimic it better -- and it'll be installed only after pydev finishes using it to get its optimizations.
Saturday, February 16, 2008
Saturday, February 02, 2008
Pydev 1.3.12: bug in outline
The release has been out for some days now, but there's one bug in that release I think is worth mentioning: the outline is not behaving as it should: the actions are not there and it doesn't link with the editor.
This is already fixed in the cvs and will be released next week in a bugfix release...
Nice tip: Always remember to check if a weak-reference is still alive before trying to access it -- this specific portion of the code hasn't been changed in a while, but after changing other things this bug started to show...
Ain't it amazing how software works? I wonder if this kind of thing will be a thing of the past at some point in the future...
Testing for common situations and expanding those tests as bugs are found seems to be the best alternative (currently) -- this is a good feature on dynamic languages: you're never left thinking that if it compiles it works... that's never true anyways -- but it would be nice if there was a better way to ensure things work :)
I guess code-analysis helps there, but it is surely not foolproof... and the number of false positives must also be taken into account: that's one of the reasons why 'self' access check has not been added to pydev extensions: the number of false positives in some sample programs was showing that it was too common to access attributes that the code-analysis couldn't get.
If you think in the dynamic world, code-analysis is also more limited (as there's less info to work on)... The pydev extensions code-analysis helps while you're programming in the same way that a compiler would help you catch stupid errors... and having it does not ensure the program will work anyways (although I think it makes life much more pleasant!... I can't even imagine myself programming in python without having those errors gotten by the code-analysis... the code-run cycle is much slower without it, mostly because the run part ends up acting as the code-analysis -- which should have happened in the code part).
And you know... after you know the reason for the bug it is usually easy fixing it. The hard part is always discovering the bug (which is what we try to tackle with tests, code-analysis, code-reviews... ... ... )!
This is already fixed in the cvs and will be released next week in a bugfix release...
Nice tip: Always remember to check if a weak-reference is still alive before trying to access it -- this specific portion of the code hasn't been changed in a while, but after changing other things this bug started to show...
Ain't it amazing how software works? I wonder if this kind of thing will be a thing of the past at some point in the future...
Testing for common situations and expanding those tests as bugs are found seems to be the best alternative (currently) -- this is a good feature on dynamic languages: you're never left thinking that if it compiles it works... that's never true anyways -- but it would be nice if there was a better way to ensure things work :)
I guess code-analysis helps there, but it is surely not foolproof... and the number of false positives must also be taken into account: that's one of the reasons why 'self' access check has not been added to pydev extensions: the number of false positives in some sample programs was showing that it was too common to access attributes that the code-analysis couldn't get.
If you think in the dynamic world, code-analysis is also more limited (as there's less info to work on)... The pydev extensions code-analysis helps while you're programming in the same way that a compiler would help you catch stupid errors... and having it does not ensure the program will work anyways (although I think it makes life much more pleasant!... I can't even imagine myself programming in python without having those errors gotten by the code-analysis... the code-run cycle is much slower without it, mostly because the run part ends up acting as the code-analysis -- which should have happened in the code part).
And you know... after you know the reason for the bug it is usually easy fixing it. The hard part is always discovering the bug (which is what we try to tackle with tests, code-analysis, code-reviews... ... ... )!
Tuesday, January 29, 2008
Pydev Release 1.3.12 and bug-reporting changes
Pydev 1.3.12 is out!
The major change is the speed-up in operations that require type-inference (e.g.: code-completion, code-analysis, etc.) and some bug-fixing!
Aside from that, from now on, bug reports will need a valid user registered at sourceforge... the main rationale for this is that most of the reports that come from anonymous users are usually not properly fixed because lots of times the 1st report misses the basic info to reproduce the bug, and if reporters can't be reached those end up being pretty much useless...
The major change is the speed-up in operations that require type-inference (e.g.: code-completion, code-analysis, etc.) and some bug-fixing!
Aside from that, from now on, bug reports will need a valid user registered at sourceforge... the main rationale for this is that most of the reports that come from anonymous users are usually not properly fixed because lots of times the 1st report misses the basic info to reproduce the bug, and if reporters can't be reached those end up being pretty much useless...
Friday, January 25, 2008
Eclipse Jobs API and Pydev Optimizations
1st thing to learn: Don't use UIJobs unless it's imperative that you have access to the UI... From 1.3.9 to 1.3.10 I was doing some refactorings to use the Eclipse Jobs API instead of raw Threads...
After taking a look at the core options (Job, UIJob and WorkbenchJob), the WorkbenchJob was chosen, as things shouldn't run if the workbench was shutting down (the docs say that it'll do checks to see if the Workbench is running, before actually running the Job -- as the Job in case: code-analysis is not suitable to run at shutdown, it seemed a nice choice)
But the thing is that the workbench job ends up running in the UI Thread -- and thus gives a little halt to the user every time the job runs... (and only after going after it in a bug report I found out about it).
On the good side, I took some more time to profile the code-analysis / code-completion (mainly when interacting with large files) and was able to make some nice improvements that will surely be noticeable... even on smaller cases:
- a cache that works for the duration of a type-analysis request to hold non-direct tokens available at a module was added
- a cache to hold scope information within a module was also created...
In my tests it gave a 40x improvement on a type-analysis request, as many traverses of the ASTs are not needed anymore with those caches (those will have a higher impact when a given module is imported a lot from different places).
All in all, optimizing Pydev is becoming more and more challenging :)
p.s.: Those improvements will be available on Pydev 1.3.12 (which hopefully will be released in the beginning of the next week).
After taking a look at the core options (Job, UIJob and WorkbenchJob), the WorkbenchJob was chosen, as things shouldn't run if the workbench was shutting down (the docs say that it'll do checks to see if the Workbench is running, before actually running the Job -- as the Job in case: code-analysis is not suitable to run at shutdown, it seemed a nice choice)
But the thing is that the workbench job ends up running in the UI Thread -- and thus gives a little halt to the user every time the job runs... (and only after going after it in a bug report I found out about it).
On the good side, I took some more time to profile the code-analysis / code-completion (mainly when interacting with large files) and was able to make some nice improvements that will surely be noticeable... even on smaller cases:
- a cache that works for the duration of a type-analysis request to hold non-direct tokens available at a module was added
- a cache to hold scope information within a module was also created...
In my tests it gave a 40x improvement on a type-analysis request, as many traverses of the ASTs are not needed anymore with those caches (those will have a higher impact when a given module is imported a lot from different places).
All in all, optimizing Pydev is becoming more and more challenging :)
p.s.: Those improvements will be available on Pydev 1.3.12 (which hopefully will be released in the beginning of the next week).
Tuesday, January 15, 2008
Pydev release: 1.3.11
This release fixed some problems on the 3.3 integration (such as the undo/redo limit not being respected and the new 'insert spaces for tabs' in the general preferences conflicting with the pydev options).
Some niceties, such as being able to define custom filters in the pydev package explorer, not collapsing things in the outline unless really needed and having spell checking (which still depends on JDT -- because the Eclipse platform will only have that when 3.4 is released) are also there...
Now, the actual highlights were on the jython side this time: Java projects can now be referenced and have code-completion and go-to-definition working (actually, the go-to-definition is only available in pydev extensions) and the jython debugger should be finally working... There's even a little story about it:
The debugger used to work strangely in jython -- almost randomly... After lots of investigation, it turned up that pydev had a thread that was started with a delay to keep checking events in the debugger. That thread tried to run untraced, and changed sys.settrace(None) -- at that time, all stopped working (easy to say that now, but I had to go, get jython, compile it and run the debugger in a 'mixed mode' with the JDT/pydev debugger to find out about that).
In the end, changing sys.settrace in jython didn't only change the current thread tracing, but for the whole system... Paul Jean had already submitted a patch (which was applied to the trunk yesterday) so that that threads can run untraced... meanwhile, the pydev debugger is running with all threads traced on jython (checking for the actual version, so, as soon as a new jython release comes out, it should already be working).
Enjoy ;-)
Some niceties, such as being able to define custom filters in the pydev package explorer, not collapsing things in the outline unless really needed and having spell checking (which still depends on JDT -- because the Eclipse platform will only have that when 3.4 is released) are also there...
Now, the actual highlights were on the jython side this time: Java projects can now be referenced and have code-completion and go-to-definition working (actually, the go-to-definition is only available in pydev extensions) and the jython debugger should be finally working... There's even a little story about it:
The debugger used to work strangely in jython -- almost randomly... After lots of investigation, it turned up that pydev had a thread that was started with a delay to keep checking events in the debugger. That thread tried to run untraced, and changed sys.settrace(None) -- at that time, all stopped working (easy to say that now, but I had to go, get jython, compile it and run the debugger in a 'mixed mode' with the JDT/pydev debugger to find out about that).
In the end, changing sys.settrace in jython didn't only change the current thread tracing, but for the whole system... Paul Jean had already submitted a patch (which was applied to the trunk yesterday) so that that threads can run untraced... meanwhile, the pydev debugger is running with all threads traced on jython (checking for the actual version, so, as soon as a new jython release comes out, it should already be working).
Enjoy ;-)
Subscribe to:
Posts (Atom)