Using module 'subprocess' with timeout


Here's the Python code to run an arbitrary command returning its stdout data, or raise an exception on non-zero exit codes:

proc = subprocess.Popen(
    stderr=subprocess.STDOUT,  # Merge stdout and stderr

communicate is used to wait for the process to exit:

stdoutdata, stderrdata = proc.communicate()

The subprocess module does not support timeout--ability to kill a process running for more than X number of seconds--therefore, communicate may take forever to run.

What is the simplest way to implement timeouts in a Python program meant to run on Windows and Linux?

11/1/2015 12:18:26 AM

Accepted Answer

In Python 3.3+:

from subprocess import STDOUT, check_output

output = check_output(cmd, stderr=STDOUT, timeout=seconds)

output is a byte string that contains command's merged stdout, stderr data.

check_output raises CalledProcessError on non-zero exit status as specified in the question's text unlike proc.communicate() method.

I've removed shell=True because it is often used unnecessarily. You can always add it back if cmd indeed requires it. If you add shell=True i.e., if the child process spawns its own descendants; check_output() can return much later than the timeout indicates, see Subprocess timeout failure.

The timeout feature is available on Python 2.x via the subprocess32 backport of the 3.2+ subprocess module.

8/20/2019 2:47:36 PM

I don't know much about the low level details; but, given that in python 2.6 the API offers the ability to wait for threads and terminate processes, what about running the process in a separate thread?

import subprocess, threading

class Command(object):
    def __init__(self, cmd):
        self.cmd = cmd
        self.process = None

    def run(self, timeout):
        def target():
            print 'Thread started'
            self.process = subprocess.Popen(self.cmd, shell=True)
            print 'Thread finished'

        thread = threading.Thread(target=target)

        if thread.is_alive():
            print 'Terminating process'
        print self.process.returncode

command = Command("echo 'Process started'; sleep 2; echo 'Process finished'")

The output of this snippet in my machine is:

Thread started
Process started
Process finished
Thread finished
Thread started
Process started
Terminating process
Thread finished

where it can be seen that, in the first execution, the process finished correctly (return code 0), while the in the second one the process was terminated (return code -15).

I haven't tested in windows; but, aside from updating the example command, I think it should work since I haven't found in the documentation anything that says that thread.join or process.terminate is not supported.

Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow