|
Title: pp as queue manager Post by: ajb on May 24, 2011, 05:37:06 PM Hi all,
Is it possible to use Parallel Python as a queue manager, where the server runs in the background as a daemon and multiple jobs will wait in a queue until a free processor is available (like PBS, Torque, or Maui)? When I run multiple instances of a program (like the examples) they seem to run independently and just keep spawning new processes instead of waiting in a queue until CPUs are available. Right now, I am just trying to run everything on a single 2-processor machine. Basically, I am hoping to use pp as a lightweight queue manager. Thanks, Andrew Title: Re: pp as queue manager Post by: saml on June 09, 2011, 09:02:48 AM <snip> Basically, I am hoping to use pp as a lightweight queue manager. Subscribe. Also, if it is not possible, are there big changes needed to enable this? Title: Re: pp as queue manager Post by: afaulconbridge on August 19, 2011, 07:21:42 AM I think you can do this using separate ppserver instances instead of specifying the number of CPUs. See http://www.parallelpython.com/content/view/15/30#ADVANCEDCLUSTERS
|