Parallel Python

ajb

New Python

Karma: 0
Posts: 1

« on: May 24, 2011, 05:37:06 PM »

Hi all,
Is it possible to use Parallel Python as a queue manager, where the server runs in the background as a daemon and multiple jobs will wait in a queue until a free processor is available (like PBS, Torque, or Maui)? When I run multiple instances of a program (like the examples) they seem to run independently and just keep spawning new processes instead of waiting in a queue until CPUs are available. Right now, I am just trying to run everything on a single 2-processor machine. Basically, I am hoping to use pp as a lightweight queue manager.
Thanks,
Andrew


	Logged

saml

New Python

Karma: 0
Posts: 2

Re: pp as queue manager

« Reply #1 on: June 09, 2011, 09:02:48 AM »

Quote from: ajb on May 24, 2011, 05:37:06 PM

<snip>
Basically, I am hoping to use pp as a lightweight queue manager.

Subscribe.

Also, if it is not possible, are there big changes needed to enable this?


« Last Edit: June 09, 2011, 09:04:31 AM by saml »	Logged

afaulconbridge

Full Python

Karma: 1
Posts: 16

Re: pp as queue manager

« Reply #2 on: August 19, 2011, 07:21:42 AM »

I think you can do this using separate ppserver instances instead of specifying the number of CPUs. See http://www.parallelpython.com/content/view/15/30#ADVANCEDCLUSTERS


	Logged

Pages: [1]

« previous next »

	Author	Topic: pp as queue manager (Read 3107 times)
0 Members and 1 Guest are viewing this topic.