I'm developing a system where I want to use pp for executing external python scripts. These scripts are often numpy calculations that require 0.5sec to 2min and may have other dependencies too.
For each job, the
only input is a module name (i.e. "mymodule.py"), function name (i.e. "myfunction"), and a string input to the function (i.e. "/tmp/in.tif").
I've been using the "proxy" approach as described elsewhere in this forum, plus python's
__import__(...) and
getattr(...). The following code seems to work but I feel that there may be better ways to do this, thus I'm asking for your advice.
#mymodule.py
#mymodule.py
import time
import numpy as np
def myfunction(filename):
return filename + internalfunction()
def internalfunction():
return str(np.arange(10))
#pp_system.py
#pp_system.py
import pp
modulestr = "mymodule"
funcstr = "myfunction"
argstr = "/tmp/in.tif"
def execfunc(modulestr, funcstr, argstr):
''' proxy function '''
coremodule = __import__(modulestr)
mainfunction = getattr(coremodule, funcstr)
ret = mainfunction(argstr)
return ret
job_server = pp.Server()
job1 = job_server.submit(execfunc, args=(modulestr,funcstr,argstr))
r1 = job1()
print r1
Forum 
