Monday, December 30, 2013

PyDev 3.2.0 released

The new PyDev release is out.

For those using 3.1.0, the upgrade is recommended as this version had some compile issues which made Ctrl+1 to create classes/methods fail on some situations.

The focus on this build was mostly on bugfixing and performance/memory enhancements, so, if you're interested in getting a faster PyDev, the upgrade is recommended :)

Python stackless debugger integration is also much improved.

In the debugger, there's also a new feature, where running the debugger with catch caught exceptions turned on will now show a view with the caught exception stack (indicating the current frame), and it's possible to ignore that same exception afterwards by right-clicking the stack in that view and choosing the related option (note that handling caught exceptions in the debugger may be enabled in the debug perspective > pydev > manage exception breakpoints).

See: http://pydev.org for more details on the release.

p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)

p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).

Thursday, December 12, 2013

PyDev 3.1.0 released

PyDev 3.1.0 is now already available for download.

The debugger got a lot of attention in this release:

One of my favorite features is the code reloading. This means that when you change a file in the PyDev editor and there's a debug session going on, PyDev will automatically reload that module.

There are some caveats on doing that:

1. It'll try to reload the objects code in place (without changing references), but the code you're currently using in a frame in the current stack cannot be changed, so, you'll have to get out of the function and then back to see the changes (the set next line feature in the debugger which allows you to set the next line to be executed in the current scope can be very helpful for that).

2. Not everything can be cleanly reloaded (see: https://github.com/fabioz/Pydev/blob/development/plugins/org.python.pydev/pysrc/pydevd_reload.py for the limitations on xreload).

Still, it's very nice that you can now do the reload and it does work well on many situations ;)

Another nice thing is that it's now possible to inspect which are the referrers of a given variable by right-clicking it on the variables or expressions view and selecting 'get referrers'.

One last thing I'd like to mention is that Stackless Python is now supported in the debugger, which will show the tasklets as if they were threads in the stack view.

Besides the debugger, the rename refactoring also got some love, so, it's now possible to rename a module/package having all references updated as well. Also, the find references (Ctrl+Shift+G) had some bugfixes and should be more reliable now.

There are still a number of other nice things, so, check the release details on http://pydev.org for more information.

p.s.: Note that PyDev requires Eclipse 3.8 onwards and Java 7 (for older versions, keep using PyDev 2.x)

p.s.2: LiClipse (http://brainwy.github.io/liclipse/) is recommended for users that want a PyDev standalone with a hassle free install where things should 'just work' (also, by licensing LiClipse you directly support the development of PyDev).

Thursday, November 07, 2013

PyDev 3.0!

PyDev 3.0 has just been released.

This is the first version which breaks compatibility with older Eclipse and Java versions, so, in order to run it requires Eclipse 3.7 onwards and Java 7 (for older versions, keep using PyDev 2.x) -- check LiClipse (http://brainwy.github.io/liclipse/), if you want a hassle free install where things should 'just work'.

This release has 2 main improvements: usability improvements related to configuring an interpreter and several enhancements in the interactive console.

On the interpreter configuration side, there were several improvements, but I believe the major change is that when the files related to the interpreter are changed in the filesystem (i.e.: when pip-installing a library for instance), PyDev will automatically get that the interpreter configuration needs to be updated (previously a manual step was required in order to add that path to the PyDev interpreter configuration). This was the most voted issue on the tracker and I believe one of the sources of greater frustration for PyDev users -- and it's finally solved! (which for me is the cherry in this release and warrants the newly-acquired 3.0 status!)

As for the interactive console, it has many noteworthy enhancements (and more enhancements are expected in the upcoming versions):

1. The interactive console can now execute in debug mode. By enabling this feature in the preferences, it's possible to place breakpoints in code, execute some code in the interactive console and stop at that breakpoint (which is really nice).

2. User Module Deleter (UMD): By enabling UMD in the preferences (i.e.: using runfile instead of execfile to run a python module), dependencies imported when executing some code are reimported (so, changes in files are automatically gotten without requiring manual reload() calls).

3. IPython 1.0 is now supported. Also, %edit will open the file in the PyDev editor and %gui customizes the gui event loop integration (i.e.: %gui wx to enable wxPython backend).

More details on the interactive console can be seen at: http://pydev.org/manual_adv_interactive_console.html

Other features have also been added, including better handling of numpy arrays in the debugger, right-clicking a variable in the debugger > pretty print to pretty-print it, persistent history in the interactive console -- besides some important fixes.

The last thing is that LiClipse (http://brainwy.github.io/liclipse/) has also been released with the latest changes in PyDev 3.0 and now also supports Mako Templates and Dart (besides the languages it already supported, such as Django Templates, Javascript, HTML, C/C++, etc.)

Thursday, October 03, 2013

First LiClipse public release

The crowdfunding at the start of this year (http://igg.me/at/liclipse) had 2 targets, one was keeping the PyDev development going (which is going strong), and the other was creating a base to complement PyDev when dealing with other languages and providing some enhancements in the form of another plugin (LiClipse) which I thought were crucial to keep the Eclipse ecosystem healthy (and in return PyDev too).

Well, today I've just made the first LiClipse public release (it was in a closed Alpha so far). For me it's now the easiest way to get started on PyDev (as LiClipse bundles PyDev and has native standalone installers for Windows/Mac OS/Linux), besides providing support for common languages which often go with Python development (such as HTML, JavaScript, CoffeScript, Django Templates, RST, etc.), improved theming support and multi-edition (which may seem a minor thing, but it does make a number of workflows much more straightforward to me, so, I can hardly think about developing without it anymore).

More details on: http://liclipse.blogspot.com.br/2013/10/first-liclipse-public-release.html and its homepage: http://brainwy.github.io/liclipse/

Enjoy!

Thursday, September 05, 2013

PyDev 2.8.2 released. Plans for PyDev 3.

The latest PyDev release is out. Thank you to all supporters (https://sw-brainwy.rhcloud.com/supporters/PyDev/) which make it possible for me to keep working on PyDev!

Now, before giving more details on the release, let me outline the plans for PyDev 3:

Based on the feedback from my previous post, the plan will be moving forward with PyDev 3 to support only the latest Eclipse and Java versions (i.e.: Eclipse 3.8 and Eclipse 4.3 onwards and Java 7) . So, this release may mark the last release that supports older versions of Eclipse/Java. It's possible that there are more bugfix-only releases for the PyDev 2.x series, but only if some critical bug is found.

In practice: a development_pydev2 branch was created to provide for fixes that may be needed on PyDev 2, which is now in bugfix mode and the current development branch will now target PyDev 3.

Now related to the latest 2.8.2 release, lots of things came in, so, I'll try to keep the list short on what I think is nicest :)

  • Improved docstring support for sphinx/epydoc, so, you can have a different color to mark the related markup and code-completion can be requested to show sphinx markup on docstrings (as it already did for epydoc).
  • It's possible to use PEP-8 style imports (which is now the default on organize imports) and it's also possible to configure organize imports to remove unused imports -- personally, I think that's a bit dangerous, but it can be really handy sometimes :)
  • The project configuration got much nicer. As an example, it's possible to right click some folder to mark/unark it a source folder (i.e.: add/remove it from the PYTHONPATH).

There are many other changes too (see: http://pydev.org/ for details).

Wednesday, August 07, 2013

PyDev: poll about minimum Java/Eclipse versions to support

Ok, one issue that has been brought up a few times in the last month was the reasons for supporting older versions of Java and Eclipse in PyDev...

Java 7 has been around for a while already (and Java 6 EOL was about a year ago), so, I'm considering making newer versions of PyDev compatible with Java 7 and dropping support for older versions. If there are use cases where people that can't upgrade, I'd like to know about those.

Also, the latest PyDev can support up to Eclipse 3.2 (although some features might not be available if they were introduced on later versions of Eclipse, the reason for this was that Aptana Studio 2 was built on that version, but it's also no longer active, so, this may be a bad reason at this time) -- note that right now the codebase needs to be compiled with the latest Eclipse version, but in runtime it can support older versions of Eclipse.

So, I'm considering supporting only Eclipse 3.8 and 4.3, but I'd also like feedback to check if there are scenarios where this can be a problem...

Thursday, July 25, 2013

PyDev 2.8.0 released

I guess that the major feature added in PyDev 2.8.0 is that the type inference engine will now work with docstrings to find out about the types.

The page: http://pydev.org/manual_adv_type_hints.html has more details about that (note that the syntax is very broad, so, I'm waiting on reports if there are use-cases I did not cover properly). Also note that PyDev still can't provide code-completion when unpacking types -- such as a list(str) -- if you want that, please vote on https://sw-brainwy.rhcloud.com/tracker/PyDev/147 (as it's a request that's considerably hard to do, I'll probably only do it if there are many people interested).

The second is that the interactive console supports running with Qt and Gtk event loops (which was a patch by Edward Catmur). This allows users to interactively create a UI in those frameworks while actually seeing and manipulating the window.

-- Update: 2.8.1 has been released to remove the Gtk event loop because was not working properly.

Wx is still not supported properly because I wasn't able to make it work without redirect=False on the app, and if redirect=True it won't put the output in the interactive console (if there's someone knowledgeable with time to check it, org.python.pydev/pysrc/pydevconsole.py: WxMainLoop is the place to look at).

The last thing is that debugging should be working again on Google App Engine (which was broken after the latest updates on the dev_appserver.py) -- it was really a bummer not having the debugger there.

Besides that, there are many other niceties added in this release. See http://pydev.org for more details.

Now, unrelated to release features: I think that a nice side-effect of the plead to support for PyDev is that more people are contributing code to PyDev too (besides allowing me to spend more time to develop it myself), so, this release had much more pull requests merged than usual -- great times to be developing PyDev :)

Another nice thing is that for this release I was finally able to enable voting properly on the tracker and allow supporters to fund PyDev again (and get votes in the tracker or space in the homepage as a reward), so, for those that haven't been able to contribute before, https://sw-brainwy.rhcloud.com can be accessed to provide funds to keep it going so that it can remain viable on the long term... (as short term it's viable thanks to the previous Indiegogo crowdfunding).




Tuesday, May 28, 2013

PyDev 2.7.5 released

Just issued a new release. It's mostly a bugfix release, but there are some critical fixes there...

The major ones are:

  • A deadlock was fixed.
  • The icons are now properly in the outline (they moved places in the last release but the build file wasn't changed properly to include them).
  • When browsing the modules related to the interpreter in the PyDev Package Explorer, those were not properly opened.

And there are a couple of improvements released too (such as having the pyunit view output font use the console font, auto-formatting can be enabled just for files in the workspace, etc.).

Tuesday, May 14, 2013

PyDev crowdfunding finished

Ok, the PyDev crowdfunding has just finished -- and it reached its basic goal :).

The funding was divided in a series of perks, 610 people contributed and the final distribution for them (monetary-wise) was:


  • Altruist: 1.5% (Thank you)
  • Early adopter: 17.95% (LiClipse regular license -- no access to betas)
  • Believer: 11.92% (LiClipse regular license -- access to tracker and betas)
  • PyDev Knight: 23.48% (Get a vote in the PyDev tracker and contribute only to PyDev itself)
  • Oracle: 22.61% (LiClipse perpetual license)
  • Bronze Sponsor: 2.86% (10 regular licenses)
  • Silver Sponsor: 19.68% (10 perpetual licenses)


I must say that the funding changed quite a bit. Initially there was no PyDev Knight perk (which is a perk targeted only towards PyDev, whereas the others are targeted at PyDev and LiClipse). So, this was the result of listening to feedback and adjusting the campaign (also, initially I didn't put enough details on PyDev and added to much emphasis on LiClipse -- which is something adjusted during the campaign).


There is also one thing I failed to check properly before : Paypal fees are not the same everywhere. Here in Brazil it's 7.4% + 0.3US$ (whereas in US it's 2.9%+0.3US$). So, in the end, 12%  went to Paypal/Indiegogo, whereas I was planning something around 8% (and there are still taxes to pay, but this is something to be expected anyways, so, in the end the net result for me will be something as 73% -- but just because I have a company setup for that, had I done it as a person it'd be around 63% here in Brazil -- I just hope the government does use it's share properly -- grin).

I still think it was worth as a way for a one time funding as everything is already setup and to know if there's interest before starting it -- which I think is the main point of a crowdfunding campaign: discover if something is viable before delving into it (not that failing on a crowdfunding is always a sure sign that something won't work, but in this specific case, with a product that is already mature, and with an existing community, I think it's how I'd see it).

Also, during that process I've set up a new tracker at https://sw-brainwy.rhcloud.com for both PyDev and LiClipse. It's a tracker I had done for fun some time ago (and yes, I know it's one of the seven sins of a programmer), but as I'd have to customize any solution for what I want, I think it's nice. The idea is that anyone will be able to create tickets and comment on those, but for voting which tickets will be done next you'll have to become a PyDev Knight (I still haven't sent the access for existing PyDev Knights to vote, but I'll do it during the upcoming month -- as you can see, I still have some things to set up).

A note on the tracker: it's hosted on RedHat's OpenShift, and it's called Brainwy because that's the company I created to keep PyDev going :)

Mainly it's a Javascript-based tracker where it's easy to see the whole picture and move things around. If you're curious, you can use an offline version at https://sw-brainwy.rhcloud.com/demo/, because as a user in the PyDev/LiClipse trackers you won't really be able to move things around, just add ticket, comment on existing ones and vote (once you have a vote) -- you can use it as much as you want, but note that I don't plan on adding support to it or actually distributing it differently -- I've already enough supporting PyDev and LiClipse :)

All in all, it was definitely a good outcome and it was nice having so much support from so many people!

Thank you everyone!

Fabio

Tuesday, April 30, 2013

PyDev crowdfunding basic funding reached!

The initial target of the crowdfunding for PyDev has been reached. Thank you everyone who helped in reaching that target!

Now, please don't stop sharing and helping it get to the higher proposal levels (having more there will enable more things to be done).

Also, I've had some people ask me privately on how things will proceed afterwards... well, with this funding, I'll be able to work in PyDev for the upcoming year, so, during that time I'll create a structure so that people that like it can keep supporting it so that it can move forward year after year.

Also, I hope LiClipse itself turns out to be successful, as the idea is that earnings in LiClipse will also help in getting PyDev going on -- if things turn out pretty well, I'll probably be able to get more people to help there too :)

Tuesday, April 16, 2013

PyDev crowdfunding / LiClipse initially supported languages


Right now (30+ days elapsed on the funding), things are looking good, the PyDev/LiClipse funding already got to 70% of its target (with 354 supporters). Thank you everyone! Please keep sharing about it ;) (http://igg.me/at/liclipse)

On the PyDev side, the new tracker is still not up, but I'm working on it (so, please buffer your requests until the tracker is up... I'll provide proper links once that's done).

Regarding the initial languages to be supported in LiClipse, I'm sharing below the results of the poll so far (note: only languages with more than 15 answers are shown).

python 152
javascript 106
html 98
css 93
django templates 70
cython 53
c++ 47
c 39
jinja2 37
java 33
coffescript 27

On the total there were 50 languages voted. There was one I didn't even know about (the Ren'Py language http://www.renpy.org/ ).

I'll still leave the poll there until the end of the funding (vote on poll) to decide on the initial supported languages, but note that even if a language is not initially there, I'll put up proper docs and provide support so that anyone should be able to easily add a language there ;)

Thursday, April 04, 2013

PyDev / LiClipse funding (and initially supported languages)

Ok, 20 days on the crowdfunding elapsed and the funding is already at 57% (with 258 funders)! So, the funding is going pretty well so far. Thank you everyone!

Also, I'd like to give special thanks for the current Silver sponsors (http://holdenweb.com - http://kichwacoders.com) and the Bronze sponsor (http://wobe-systems.com).

Still seeking Gold and Platinum sponsors though... so, if there's a company that is a heavy user of PyDev -- or maybe the future LiClipse -- and would like some space in the PyDev / LiClipse homepage...

Also, I've put up a poll on google docs to help in deciding which will be the languages supported initially in LiClipse. You can vote at: LiClipse Languages Poll.

Note that I still have not put up a new tracker for PyDev itself (but I'm working on that).

Monday, March 25, 2013

Crowdfunding PyDev / LiClipse status

So far, the experience on the crowdfunding has been really positive. In 10 days of funding, there are already 168 funders and 41% of the basic funding goals have been reached!

Also, it was nice to gather feedback and talk to many PyDev users out there.

Regarding PyDev, reactions were pretty positive, but regarding LiClipse, they were initially a bit mixed.

On the dark theme, it was either a love or hate reaction, but just to note, although that's one of the major points of LiClipse, it will definitely not force you to use a dark theme -- it'll be an opt in, not opt out -- and I  expect that part to be the easy / fast part in the proposal :)

As for the languages support, initially it was a bit more neutral (with some ups and downs), and I think initially I did a bad job on explaining why I think that was needed, so, I've updated http://igg.me/at/liclipse to explain the 'why' part too :)

Note that as a result of talking to PyDev users, I did change the funding a bit too: there's now a special perk (named PyDev Knight) which empowers users who enjoy PyDev and only really care about a better Python support and not a dark theme or other languages (even if related to Python, such as Django Templates, Mako, etc).

So, I can only thank everyone so far for contributing and spreading the word on the funding!



Saturday, March 16, 2013

PyDev funding and LiClipse?

I got some feedback related to the creation of LiClipse and its relation with PyDev in the current crowdfunding proposal (http://igg.me/at/liclipse/), so, I'd like to explain how I believe things fit together.

1. So, is LiClipse a fork of Eclipse?

Definitely not. The idea behind LiClipse is having 2 things:
  • A way to theme Eclipse itself better and have some other UI improvements (I'm truly annoyed by not having what I'd consider a professional dark theme in Eclipse right now, so, I'd like to take those matters in my own hands).
  • An editor which should be able to support lots of languages out of the box. Think something closer to other all-purpose editors -- as opposed to IDEs -- such as Notepad++, Vi, TextMate, Sublime, etc. i.e.: the idea is supporting lots of languages out of the box, so, the idea is having it resembling formats such as ultraedit wordfiles or TextMate language files.
To be clear, the idea is not to replace the more advanced editors inside Eclipse for each language, but to provide a lightweight way to deal with any language -- usually you work with 1 or 2 main languages, for which you'll have the plugins you need, but sometimes, when you just want to open a file in a language  you work seldomly and may not need/want to install a bulkier plugin, LiClipse would be a good addition to your toolbox (LiClipse is a short for "Lightweight Eclipse" BTW), and for me, not having this is a major shortcoming of Eclipse itself (and it may be an alternative for people who want less features and more speed).

Also, that should be doable without having to create a fork of Eclipse (although some theming issues may need to be resolved at Eclipse itself -- but on those parts, things should be fixed at that level, not on LiClipse).

2. How does that relate to PyDev?

Well, PyDev is very tightly bound to the environment it works on (Eclipse), and I'd like to solve some of the issues I see in it to improve its ecosystem as a whole, which IMHO is something I see needed for PyDev and Eclipse itself to keep moving forward.

3. But won't that divert too many resources out of PyDev itself?

For this proposal, my plan is spending my time 50/50 (I don't think LiClipse is a major undertaking -- all the pieces are out there, it's mostly a matter of stitching them together), although during that time, yes, it'll divert some resources from PyDev, but as I think that having it is very important for PyDev itself (as well as other language), I see the issues being tackled as very serious shortcomings of Eclipse which hinder the adoption of Eclipse itself (thus affecting PyDev directly).

4. I still think LiClipse is not a good idea and would like to support only PyDev.

Please e-mail me with those thoughts. I really believe LiClipse is needed for PyDev to keep getting traction, but if you feel that's not the case, please, please share your thoughts with me.

Also, just to make things clear, if the funding at http://igg.me/at/liclipse/ does not succeed, I won't really be able to support PyDev itself anymore (personally, I really want it to succeed, but I see the funding as the community speaking, so, if after 10 years working on it the PyDev community doesn't see it as a worthy goal or doesn't trust me enough to support me on LiClipse while properly maintaining PyDev, well, I really need to hear it and move on -- as a note, until now, I see the funding as a huge success, getting to 10% in the first day, so, if everyone keeps helping a bit there, it'll be awesome :)

Thursday, March 14, 2013

Keeping PyDev alive through crowdfunding

Ok, I just started a crowdfunding project (at www.indiegogo.com) for the continued development of PyDev (and improvements on Eclipse overall).

In this post, I hope to shed some light on why this funding is needed.

First, a little bit of history just to give some context: I've been developing PyDev for more than 10 years already (wow, I just realized that as I started the campaign) and it's definitely a pretty successful project (if it's not the most used Python IDE, it's definitely among the top ones).

So, backtrack a few years...

PyDev started doing success and I created a commercial extension to enable me to work more time on PyDev itself. After some time, Aptana acquired it and I joined them. Unfortunately, Aptana itself didn't turn out very well: it focused on its main product (the IDE) without generating any revenue from it, while trying to make side projects worthy enough to cover for everything (but those side projects weren't successful enough for that).

Enter Appcelerator: it acquired Aptana out of the need to provide a deeper integration for its main product (which is Titanium). So, for some time, it did back up the PyDev development (along with the other languages), but in the end, their main product is Titanium and the tools around it, so, PyDev itself wasn't seen as relevant enough to be kept supported (to be clear, they're still hosting the homepage and downloads, but not backing the development itself). As such, in the end of the last year they stopped supporting its development.

So, that's where the project is at now: it's (IMHO) a pretty successful, but unable to generate revenue for its continued support. Given this scenario I decided to create a crowdfunding project to ask for the community to provide resources to make that happen.

In the funding, I expanded its reach a bit, on what I think are the main issues with the PyDev environment right now. So, the idea is focusing on a nicer dark UI, usability, speed and memory and providing a way to easily have other editors out of the box in a lightweight implementation (Python is pretty strong in the web, so, one of the weak points right now is actually not the Python editing itself, but related web languages, such as CoffeScript and JavaScript).

So, please help in funding (and sharing) at http://igg.me/at/liclipse to keep PyDev going strong!

Thursday, January 17, 2013

Interrupting a Python thread with signals

First off, in Python, as appears to be common knowledge, signals can only be received by the main thread, and usually you need to do synchronization work with Queues or something else to work with other threads, so, what I'll describe below is a different approach to the problem which in effect will give you a way of interrupting a thread in Python from anywhere.

First off, I'll show the code and explain it afterwards:

import time
import sys
import threading

class SigFinish(Exception):
    pass

def throw_signal_function(frame, event, arg):
    raise SigFinish()

def do_nothing_trace_function(frame, event, arg):
    # Note: each function called will actually call this function
    # so, take care, your program will run slower because of that.
    return None

def interrupt_thread(thread):
    for thread_id, frame in sys._current_frames().items():
        if thread_id == thread.ident:  # Note: Python 2.6 onwards
            set_trace_for_frame_and_parents(frame, throw_signal_function)

def set_trace_for_frame_and_parents(frame, trace_func):
    # Note: this only really works if there's a tracing function set in this
    # thread (i.e.: sys.settrace or threading.settrace must have set the
    # function before)
    while frame:
        if frame.f_trace is None:
            frame.f_trace = trace_func
        frame = frame.f_back
    del frame


class MyThread(threading.Thread):

    def run(self):
        # Note: this is important: we have to set the tracing function
        # when the thread is started (we could set threading.settrace
        # before starting this thread to do this externally)
        sys.settrace(do_nothing_trace_function)
        try:
            while True:
                time.sleep(.1)
        except SigFinish:
            sys.stderr.write('Finishing thread cleanly\n')


thread = MyThread()
thread.start()
time.sleep(.5)  # Wait a bit just to see it looping.

interrupt_thread(thread)
sys.stderr.write('Joining\n')
thread.join()  # Joining here: if we didn't interrupt the thread before, we'd be here forever.
sys.stderr.write('Finished\n')



So, there you have it: a hacky approach which will prevent your code from being properly debugged :)

-- note that sys.settrace could be changed for sys.setprofile to achieve the same result -- in which case your program could be debugged but not profiled (which may be a better approach but will make the signal be raised only on function calls, not on line calls).

To sum it up, we use the tracing mechanisms that's intended for debuggers (or profilers) to actually throw the signal for us. This means that we have to enable the tracing on that thread (which will make that thread to execute a bit slower) and set the tracer function to a function which will actually throw the signal.

IMO, it works in a hackish (but nice) way... not sure if I'd use that in a production environment, but, there you have it: interruptible threads in Python (yes, you still have the limitation of the GIL: a thread won't be interrupted while calling some atomic operation that doesn't release the GIL)... Now, if only there was a Python signal library better integrated with the Python interpreter so that it checked for signals itself (probably in a way close to the tracing function itself with less overhead as only a single additional check for a null variable would be needed -- and otherwise a signal would be thrown), this hack wouldn't be needed in the first place -- but I'll leave that to someone else reading this :)

Monday, January 07, 2013

Python to Javascript translation

I'm currently doing a project with a backend in Python and a client mostly in Javascript -- still not public, but if everything turns out right I'll post about it later on -- and there are some algorithms I'd like to reuse in both, but without going to a full Python in browser style (such as Pyjs or brython)... Ideally I'd like something like Coffescript (but with a Python-like syntax so that a subset of Python can be used for both Python and Javascript generation).

-- Sidenote: I decided I really have to grasp Javascript in order to find out how things work before trying an alternative language which compiles down to Javascript -- so far my experience has been nice. I found that I end up programming much more in a functional style and use classes in a very limited way (haven't felt the need for inheritance so far), although I feel that it's hard to find resources on what would be the proper way of doing things (but in the plus side, there are LOTS of javascript-related resources, so, I've hardly found myself stuck while learning it).

Anyways, I have some algorithms I've written in Python which I'd like to reuse in Javascript without having to use huge runtime with it (I'm restricting that to simple functions) and searching for a translator, I've found Pyvascript, which is a Python-like language which compiles to Javascript (so, there's only a subset of Python that can really be used, which is ok as I'm not after the ones that attempt to do full Python in Javascript and end up with a huge emulated runtime), but it does fit my needs properly with some minor changes (and now I have some code I can use in both Python and Javascript at the same time). Just to note, it doesn't seem like the project is active right now, but it does work fine for me :)

Wednesday, January 02, 2013

Python: get parent process id (pid) in windows

Below is code to monkey-patch the os module to provide a getppid() function to get the parent process id in windows using ctypes (note that on Python 3.2, os.getppid() already works and is available on windows, but if you're on an older version, this can be used as a workaround).

import os
if not hasattr(os, 'getppid'):
    import ctypes

    TH32CS_SNAPPROCESS = 0x02L
    CreateToolhelp32Snapshot = ctypes.windll.kernel32.CreateToolhelp32Snapshot
    GetCurrentProcessId = ctypes.windll.kernel32.GetCurrentProcessId

    MAX_PATH = 260

    _kernel32dll = ctypes.windll.Kernel32
    CloseHandle = _kernel32dll.CloseHandle

    class PROCESSENTRY32(ctypes.Structure):
        _fields_ = [
            ("dwSize", ctypes.c_ulong),
            ("cntUsage", ctypes.c_ulong),
            ("th32ProcessID", ctypes.c_ulong),
            ("th32DefaultHeapID", ctypes.c_int),
            ("th32ModuleID", ctypes.c_ulong),
            ("cntThreads", ctypes.c_ulong),
            ("th32ParentProcessID", ctypes.c_ulong),
            ("pcPriClassBase", ctypes.c_long),
            ("dwFlags", ctypes.c_ulong),

            ("szExeFile", ctypes.c_wchar * MAX_PATH)
        ]

    Process32First = _kernel32dll.Process32FirstW
    Process32Next = _kernel32dll.Process32NextW

    def getppid():
        '''
        :return: The pid of the parent of this process.
        '''
        pe = PROCESSENTRY32()
        pe.dwSize = ctypes.sizeof(PROCESSENTRY32)
        mypid = GetCurrentProcessId()
        snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0)

        result = 0
        try:
            have_record = Process32First(snapshot, ctypes.byref(pe))

            while have_record:
                if mypid == pe.th32ProcessID:
                    result = pe.th32ParentProcessID
                    break

                have_record = Process32Next(snapshot, ctypes.byref(pe))

        finally:
            CloseHandle(snapshot)

        return result

    os.getppid = getppid