Interrupting a Python thread with signals

First off, in Python, as appears to be common knowledge, signals can only be received by the main thread, and usually you need to do synchronization work with Queues or something else to work with other threads, so, what I'll describe below is a different approach to the problem which in effect will give you a way of interrupting a thread in Python from anywhere.

First off, I'll show the code and explain it afterwards:

import time
import sys
import threading

class SigFinish(Exception):
    pass

def throw_signal_function(frame, event, arg):
    raise SigFinish()

def do_nothing_trace_function(frame, event, arg):
    # Note: each function called will actually call this function
    # so, take care, your program will run slower because of that.
    return None

def interrupt_thread(thread):
    for thread_id, frame in sys._current_frames().items():
        if thread_id == thread.ident:  # Note: Python 2.6 onwards
            set_trace_for_frame_and_parents(frame, throw_signal_function)

def set_trace_for_frame_and_parents(frame, trace_func):
    # Note: this only really works if there's a tracing function set in this
    # thread (i.e.: sys.settrace or threading.settrace must have set the
    # function before)
    while frame:
        if frame.f_trace is None:
            frame.f_trace = trace_func
        frame = frame.f_back
    del frame


class MyThread(threading.Thread):

    def run(self):
        # Note: this is important: we have to set the tracing function
        # when the thread is started (we could set threading.settrace
        # before starting this thread to do this externally)
        sys.settrace(do_nothing_trace_function)
        try:
            while True:
                time.sleep(.1)
        except SigFinish:
            sys.stderr.write('Finishing thread cleanly\n')


thread = MyThread()
thread.start()
time.sleep(.5)  # Wait a bit just to see it looping.

interrupt_thread(thread)
sys.stderr.write('Joining\n')
thread.join()  # Joining here: if we didn't interrupt the thread before, we'd be here forever.
sys.stderr.write('Finished\n')

So, there you have it: a hacky approach which will prevent your code from being properly debugged :)

-- note that sys.settrace could be changed for sys.setprofile to achieve the same result -- in which case your program could be debugged but not profiled (which may be a better approach but will make the signal be raised only on function calls, not on line calls).

To sum it up, we use the tracing mechanisms that's intended for debuggers (or profilers) to actually throw the signal for us. This means that we have to enable the tracing on that thread (which will make that thread to execute a bit slower) and set the tracer function to a function which will actually throw the signal.

IMO, it works in a hackish (but nice) way... not sure if I'd use that in a production environment, but, there you have it: interruptible threads in Python (yes, you still have the limitation of the GIL: a thread won't be interrupted while calling some atomic operation that doesn't release the GIL)... Now, if only there was a Python signal library better integrated with the Python interpreter so that it checked for signals itself (probably in a way close to the tracing function itself with less overhead as only a single additional check for a null variable would be needed -- and otherwise a signal would be thrown), this hack wouldn't be needed in the first place -- but I'll leave that to someone else reading this :)