Hi, I'm trying to use numpy in my distributed function, but keep running into errors. I can run the code without an error on my windows machine, but when i try to use a remote compute node running Ubuntu 12.04, I get errors. It looks like my problem is similar to this one:
http://www.parallelpython.com/component/option,com_smf/Itemid,1/topic,413.0 but I never saw a definitive solution posted in that thread.
I ran the following code which was requested in the other thread (which i tweaked to only run on the remote machine):
import pp, os, sys, numpy
def get_val(i):
print "Job ", i, sys.version, os.environ.get("PYTHONPATH")
print numpy.random.rand()
ppservers = ("*",)
job_server = pp.Server(0, ppservers=ppservers')
print "Main", numpy.random.rand()
jobs = []
for i in xrange(5):
jobs.append(job_server.submit(get_val, (i, ), (), ("numpy",)))
print sys.version, os.environ.get("PYTHONPATH")
for i,job in enumerate(jobs):
job()
and it gave me back the following output:
Main 0.426715069092
2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] C:/Users/jordan/Desktop;C:\Python27\lib\lib-tk;C:\Python27\lib\site-packages\xy;C:\Python27\lib\site-packages;C:\Python27\lib\site-packages\win32;C:\Python27\lib;C:\Python27\Lib\idlelib;C:\Python27\lib\site-packages\PIL;C:\Python27;C:\Python27\lib\site-packages\win32\lib;C:\Python27\lib\site-packages\Pythonwin;C:\Python27\lib\site-packages\wx-2.8-msw-unicode;C:\Python27\lib\plat-win;C:\Windows\system32\python27.zip;C:\Python27\DLLs;C:\Users\jordan\Desktop
An error has occured during the module import
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 49, in preprocess
exec module
File "<string>", line 1, in <module>
ImportError: No module named numpy
Job 0 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] /usr/lib/python2.7/lib-old:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/lib-dynload:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol:/usr/lib/python2.7/lib-tk:/home/jordan:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/ubuntuone-couch:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7:/usr/local/bin:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/python2.7/dist-packages/PIL
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
An error has occured during the module import
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 49, in preprocess
exec module
File "<string>", line 1, in <module>
ImportError: No module named numpy
Job 1 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] /usr/lib/python2.7/lib-old:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/lib-dynload:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol:/usr/lib/python2.7/lib-tk:/home/jordan:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/ubuntuone-couch:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7:/usr/local/bin:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/python2.7/dist-packages/PIL
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 2 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] /usr/lib/python2.7/lib-old:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/lib-dynload:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol:/usr/lib/python2.7/lib-tk:/home/jordan:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/ubuntuone-couch:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7:/usr/local/bin:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/python2.7/dist-packages/PIL
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 3 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] /usr/lib/python2.7/lib-old:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/lib-dynload:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol:/usr/lib/python2.7/lib-tk:/home/jordan:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/ubuntuone-couch:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7:/usr/local/bin:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/python2.7/dist-packages/PIL
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 4 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] /usr/lib/python2.7/lib-old:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/lib-dynload:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol:/usr/lib/python2.7/lib-tk:/home/jordan:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/ubuntuone-couch:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7:/usr/local/bin:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/python2.7/dist-packages/PIL
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Also, I don't know if this is related, but i noticed that when i set up a secret key on the remote machine, it will run jobs sent to it from the master node even if the key sent to it is incorrect (or missing completely). I'm not sure if this is related, but thought I would mention it. Also, i have been able to get the md5 example running on the machine.
If this is a problem with my version of Ubuntu (12.04), or this version of PP (1.6.2), could you recommend a pair that does work without throwing the error. Also, numpy isn't installed on the remote machines, although i thought that PP doesn't require it to be.
Forum 
