There are some important details about handling signals in a Python program that uses threads, especially if those threads perform tasks in an infinite loop. I realized it today while making some improvements to a script I use for system monitoring, as I ran into various problems with the proper handling of the
SIGINT signals, which should normally result in the termination of all the running threads. After a thorough read of the documentation and some research on the web, I finally made it work and thought it would be a good idea to write a post that points out these important details using a sample code snippet.
Set signal handlers in the main thread
The first most important thing to remember is that all signal handler functions must be set in the main thread, as this is the one that receives the signals.
def signal_handler_function(signum, frame): # ... def main_function(): signal.signal(signal.SIGTERM, signal_handler_function) # ...
Registering signal handlers within the thread objects is wrong and doesn’t work. The documentation of the signal module has a very informative note:
Some care must be taken if both signals and threads are used in the same program. The fundamental thing to remember in using signals and threads simultaneously is: always perform signal() operations in the main thread of execution. Any thread can perform an alarm(), getsignal(), pause(), setitimer() or getitimer(); only the main thread can set a new signal handler, and the main thread will be the only one to receive signals (this is enforced by the Python signal module, even if the underlying thread implementation supports sending signals to individual threads). This means that signals can’t be used as a means of inter-thread communication. Use locks instead.
Keep the main thread running
This is actually a crucial step, otherwise all signals sent to your program will be ignored. Adding an infinite loop using
time.sleep() after the threads have been started will do the trick:
thread_1.start() # ... thread_N.start() while True: time.sleep(0.5)
Note that simply calling the thread’s
.join() method is not going to work.
Example code snippet
Here is a basic Python program to demonstrate the functionality. The main thread starts two threads (jobs) that perform their task in an infinite loop. There is a registered handler for the
INT signals, which gives all running threads the opportunity to shut down cleanly. Note that a
Ctrl-C on your keyboard) is interpreted as a
SIGINT, so this is an easy way to terminate both the running threads and the main program.
For more information about how it works, please refer to the next section containing some useful remarks.
import time import threading import signal class Job(threading.Thread): def __init__(self): threading.Thread.__init__(self) # The shutdown_flag is a threading.Event object that # indicates whether the thread should be terminated. self.shutdown_flag = threading.Event() # ... Other thread setup code here ... def run(self): print('Thread #%s started' % self.ident) while not self.shutdown_flag.is_set(): # ... Job code here ... time.sleep(0.5) # ... Clean shutdown code here ... print('Thread #%s stopped' % self.ident) class ServiceExit(Exception): """ Custom exception which is used to trigger the clean exit of all running threads and the main program. """ pass def service_shutdown(signum, frame): print('Caught signal %d' % signum) raise ServiceExit def main(): # Register the signal handlers signal.signal(signal.SIGTERM, service_shutdown) signal.signal(signal.SIGINT, service_shutdown) print('Starting main program') # Start the job threads try: j1 = Job() j2 = Job() j1.start() j2.start() # Keep the main thread running, otherwise signals are ignored. while True: time.sleep(0.5) except ServiceExit: # Terminate the running threads. # Set the shutdown flag on each thread to trigger a clean shutdown of each thread. j1.shutdown_flag.set() j2.shutdown_flag.set() # Wait for the threads to close... j1.join() j2.join() print('Exiting main program') if __name__ == '__main__': main()
Run the above code and press
Ctrl-C to terminate it.
Below, there are some remarks which aim to help you better understand how the code snippet above works:
- As mentioned previously, each Job object performs its task in its own thread using an infinite loop. Each Job object has a
shutdown_flagattribute (threading.Event object). On each cycle, the status of the
shutdown_flagis checked. As long as the shutdown flag is not set, the threads continue doing their jobs. When set, the job threads shut down cleanly.
ServiceExitis a custom exception. When raised, it triggers the termination of the running job threads.
service_shutdownfunction is the signal handler. When a supported signal is received, this function raises the
- In the
service_shutdownfunction is registered as the handler for the
- Whenever a
SIGINTis received, the signal handler (
service_shutdownfunction) raises the
ServiceExitexception. When this happens, we handle the exception by setting the shutdown flag of each job thread, which leads to the clean shutdown of each running thread. When all job threads have stopped, the main thread exits cleanly as well.
Using signals to terminate or generally control a Python script, that does its work using threads running in a never-ending cycle, is very useful. Learning to do it right gives you the opportunity to easily create a single service that performs various tasks simultaneously, for instance system monitoring, and control it by sending signals externally from the system.
How to terminate running Python threads using signals by George Notaras is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright © 2016 - Some Rights Reserved