I am trying to do some plotting in parallel to finish large batch jobs quicker. To this end, I start a thread for each plot I plan on making.
I had hoped that each thread would finish its plotting and close itself (as I understand it, Python closes threads when they get through all the statements in run()). Below is some code that shows this behavior.
If the line that creates a figure is commented out, it runs as expected. Another plausibly helpful tidbit is that it also runs as expected when you only spawn one thread.
import matplotlib.pyplot as plt import time import Queue import threading def TapHistplots(): ## for item in ['str1']: # # it behaves as expected if the line above is used instead of the one below for item in ['str1','str2']: otheritem = 1 TapHistQueue.put((item, otheritem)) makeTapHist().start() class makeTapHist(threading.Thread): def run(self): item, otheritem = TapHistQueue.get() fig = FigureQueue.get() FigureQueue.put(fig+1) print item+':'+str(fig)+'\n', time.sleep(1.3) plt.figure(fig) # comment out this line and it behaves as expected plt.close(fig) TapHistQueue = Queue.Queue(0) FigureQueue = Queue.Queue(0) def main(): start = time.time() """Code in here runs only when this module is run directly""" FigureQueue.put(1) TapHistplots() while threading.activeCount()>1: time.sleep(1) print 'waiting on %d threads\n' % (threading.activeCount()-1), print '%ds elapsed' % (time.time()-start) if __name__ == '__main__': main()
Any help is duly appreciated.
Why not just use multiprocessing? As far as I can tell from your description, threading won't help you much, anyway...
Matplotlib already threads so that you can display and interact with multiple figures at once. If you want to speed up batch processing on a multicore machine, you're going to need multiprocessing regardless.
As a basic example (Warning: This will create 20 small .png files in whatever directory you run it in!)
import multiprocessing import matplotlib.pyplot as plt import numpy as np def main(): pool = multiprocessing.Pool() num_figs = 20 input = zip(np.random.randint(10,1000,num_figs), range(num_figs)) pool.map(plot, input) def plot(args): num, i = args fig = plt.figure() data = np.random.randn(num).cumsum() plt.plot(data) plt.title('Plot of a %i-element brownian noise sequence' % num) fig.savefig('temp_fig_%02i.png' % i) main()