Parallel Python

psederberg

Full Python

Karma: 0
Posts: 14

pp 1.5.7 and numpy bug

« on: January 23, 2010, 09:34:15 PM »

Hi PP-folks:

I've getting a very strange bug when I put use numpy as a dependent module. This code fails:

Code:

import pp
import numpy

def get_val():
return numpy.random.rand()

job_server = pp.Server()
jobs = []
for i in xrange(5):
jobs.append(job_server.submit(get_val,
(),
(),
("numpy",)))
for i,job in enumerate(jobs):
print '%d: %f' % (i,job())

with the following error:

Code:

$ python breakpp2.py
An error has occured during the module import
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
globals()[module.split('.')[0]] = __import__(module)
File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
import add_newdocs
File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
from lib import add_newdoc
File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: No module named multiarray
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 2, in get_val
NameError: global name 'numpy' is not defined
Traceback (most recent call last):
File "breakpp2.py", line 18, in <module>
print '%d: %f' % (i,job())
TypeError: float argument required

The crazy thing is that I can import numpy just fine from the python interpreter. Also, the code above runs if I replace numpy with the random module, so it's specifically an issue importing numpy in a worker process.

Any ideas?

Thanks,
Per

PS-> I'm running Debian testing 32-bit with python 2.5.


	Logged

Midnighter

Full Python

Karma: 0
Posts: 13

Re: pp 1.5.7 and numpy bug

« Reply #1 on: January 25, 2010, 01:49:28 AM »

Hmm, this line makes me suspicious:

Code:

File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx

In my personal code using import ... as ... did not work with parallel python, maybe that's the same problem with numpy. Can you edit the numpy file accordingly and try again?


	Logged

psederberg

Full Python

Karma: 0
Posts: 14

Re: pp 1.5.7 and numpy bug

« Reply #2 on: January 25, 2010, 11:58:29 AM »

Hi Midnighter:

There are thousands of instances of import ... as ... in the numpy project, and, given that everything had been working great up until recently, I think its something with the environment on the worker when pp imports numpy.

Does anyone know if the worker imports are done in a special way that might induce this error out of numpy?

Does it matter that multiarray.so is an compiled object and not a python module when it is clearly many levels down in the numpy imports? Given that it is getting imported on the worker side, it doesn't seem like it needs to be pickle-able.

Can someone else check to see if the sample code from my initial post breaks on their system? It is breaking on the three machines I have access to that also have parallel python on them.

Thanks,
Per


	Logged

psederberg

Full Python

Karma: 0
Posts: 14

Re: pp 1.5.7 and numpy bug (DEFINITE PP-1.5.7 bug)

« Reply #3 on: January 27, 2010, 07:35:37 AM »

OK, I downloaded and installed version 1.5.4 of pp and the code I posted above works without a problem. There are handfuls of other recent posts getting similar errors with 1.5.7, so this implies that something has changed between 1.5.4 and 1.5.7 to cause these imports to fail.

So, version 1.5.4 works and version 1.5.7 does not. Does anyone know what could be causing the error?

Thanks,
Per


	Logged

Vitalii

Global Moderator
Parallel Python

Karma: 2
Posts: 516

Re: pp 1.5.7 and numpy bug

« Reply #4 on: January 27, 2010, 09:29:09 AM »

PP 1.5.7 does not fail with python 2.6.4 on this example.
python breakpp2.py
0: 0.084370
1: 0.628528
2: 0.787746
3: 0.411172
4: 0.222136

That makes me think that there was something in python 2.5 update that causes the failure.


	Logged

Join Python Blogs!

Parallel Python

aditya

Jr. Python

Karma: 0
Posts: 5

Re: pp 1.5.7 and numpy bug

« Reply #5 on: January 27, 2010, 09:40:21 AM »

Yup,
tried importing numpy for the worker jobs using python 2.6.4 . No issues as Vitalii says. Looks like python 2.5 vs 2.6 here.
Best,
Aditya.


	Logged

psederberg

Full Python

Karma: 0
Posts: 14

Re: pp 1.5.7 and numpy bug

« Reply #6 on: January 27, 2010, 09:43:34 AM »

Yes, and using proto=2 seems to fix it in python2.5, so maybe that will help pinpoint the issue in the code...

Best,
Per


	Logged

Vitalii

Global Moderator
Parallel Python

Karma: 2
Posts: 516

Re: pp 1.5.7 and numpy bug

« Reply #7 on: January 27, 2010, 11:03:32 PM »

Since it was working before and now got broken, I suppose it is limited to a specific version of Python 2.5 (or perhaps a version of numpy). I've tested it with python 2.5 version I have and numpy 1.4.0:
/opt/pythons/bin/python2.5 --version
Python 2.5
/opt/pythons/bin/python2.5 breakpp2.py
0: 0.939030
1: 0.981239
2: 0.089505
3: 0.721238
4: 0.924152

and was not able to reproduce this issue.


	Logged

Join Python Blogs!

Parallel Python

Vitalii

Global Moderator
Parallel Python

Karma: 2
Posts: 516

Re: pp 1.5.7 and numpy bug

« Reply #8 on: January 27, 2010, 11:05:31 PM »

Could you please run the following script and post the output:

Code: (python)

import pp, os, sys, numpy
 
def get_val(i):
    print "Job ", i, sys.version, os.environ.get("PYTHONPATH")
    print numpy.random.rand()
 
job_server = pp.Server()
print "Main", numpy.random.rand()
jobs = []
for i in xrange(5):
    jobs.append(job_server.submit(get_val, (i, ), (), ("numpy",)))
 
print sys.version, os.environ.get("PYTHONPATH")
for i,job in enumerate(jobs):
    job()


	Logged

Join Python Blogs!

Parallel Python

psederberg

Full Python

Karma: 0
Posts: 14

Re: pp 1.5.7 and numpy bug

« Reply #9 on: January 28, 2010, 05:18:39 PM »

Hi Vitalii:

Here's the output:

Code:

$ python fromv.py
Main 0.288405616544
2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the module import
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
globals()[module.split('.')[0]] = __import__(module)
File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
import add_newdocs
File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
from lib import add_newdoc
File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: No module named multiarray
Job 0 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
An error has occured during the module import
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
globals()[module.split('.')[0]] = __import__(module)
File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
import add_newdocs
File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
from lib import add_newdoc
File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: No module named multiarray
Job 1 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 2 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 3 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job 4 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
__result = __f(*__args)
File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined

Now, here's the interesting part! If I check out pp from the google code repository, and make the following change to the code:

Code:

$ git diff
diff --git a/pp.py b/pp.py
index 82c9dab..4d60780 100644
--- a/pp.py
+++ b/pp.py
@@ -360,7 +360,8 @@ class Server(object):
raise TypeError("secret must be of a string type")
self.secret = str(secret)
elif hasattr(user, "pp_secret"):
- secret = user["pp_secret"]
+ #secret = user["pp_secret"]
+ secret = getattr(user,"pp_secret")
if not isinstance(secret, types.StringType):
raise TypeError("secret must be of a string type")
self.secret = str(secret)

BTW, is there some version of the user module that treats its attributes as a dict? My Python2.5 requires attributes...

Anyway, I point at that code with:

Code:

$ export PYTHONPATH=~/code/other/pp/parallelpython-read-only:$PYTHONPATH

and it runs without error!!!

Code:

$ python fromv.py
Main 0.721674871376
2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
Job 0 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.374807427939
Job 1 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.43653652051
Job 2 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.218321394857
Job 3 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.183824951636
Job 4 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.863557457308

Do you think this could be some weird Debian packaging bug? I am using the Debian Testing version of python-pp_1.5.7-1. The only real diffs I found was the user module change in pp and the fact that they don't like the use of a default secret, which is clearly not the error.

Perhaps I should submit this as a bug to the Debian folks...

Thanks for looking into this. In the meantime, I can easily just use the version from source. All my code is running just fine with it.

Best,
Per


	Logged

Vitalii

Global Moderator
Parallel Python

Karma: 2
Posts: 516

Re: pp 1.5.7 and numpy bug

« Reply #10 on: February 06, 2010, 09:02:32 AM »

It is great that it worked out for you at the end.

I am not sure what is the rootcause of the issue, but I would not expect Debian Testing to be a stable distribution.


	Logged

Join Python Blogs!

Parallel Python

csh

New Python

Karma: 0
Posts: 2

Re: pp 1.5.7 and numpy bug

« Reply #11 on: March 31, 2010, 05:51:12 PM »

Same problem here on Ubuntu 9.10 using the default packages (Python 2.6.4-0ubuntu3, NumPy 1.3.0-3, Parallel Python 1.5.6-2ubuntu1) - as stable as it gets. Using pp 1.6.0-RC3 resolved the issue.
Christoph


	Logged

Camillio

New Python

Karma: 0
Posts: 1

Re: pp 1.5.7 and numpy bug

« Reply #12 on: September 16, 2010, 09:20:35 AM »

The same issue with pp 1.6 on debian unstable. If I compare the sys.path in the worker's __main__():

Code:

['/usr/share/pyshared', '/usr/lib/python2.6/plat-linux2', '/home/user/python', '/usr/lib/pymodules/python2.6/gtk-2.0', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6', '/usr/lib/python2.6/lib-dynload', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/lib-old']

and the sys.path in the script I run:

Code:

['/home/user', '/usr/lib/pymodules/python2.6', '/home/user/python', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6/gtk-2.0', '/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode']

I can see that these two differ. Python complained about things in /usr/share/pyshared/numpy, which led me to try the following nasty hack - addition of sys.path.pop() to /usr/lib/pymodules/python2.6/ppworker.py:

Code:

if __name__ == "__main__":
# add the directory with ppworker.py to the path
sys.path.append(os.path.dirname(__file__))
sys.path.pop(0)
f.close()
wp = _WorkerProcess()
wp.run()

And voila, it works Cheesy

So I think that there is some issue with setting worker's sys.path.


	Logged

Vitalii

Global Moderator
Parallel Python

Karma: 2
Posts: 516

Re: pp 1.5.7 and numpy bug

« Reply #13 on: October 16, 2010, 08:42:23 PM »

Problem arises from the fact that /usr/share/pyshared is added to the sys.path.
But it is hard to say why it is getting added and by whom.
Little be better workaround would be to remove exactly the offending item:

Code: (python)

sys.path.remove('/usr/share/pyshared')


	Logged

Join Python Blogs!

Parallel Python

ckn

New Python

Karma: 0
Posts: 2

Re: pp 1.5.7 and numpy bug

« Reply #14 on: November 09, 2010, 04:44:23 PM »

I too am getting similar errors when i try to use numpy package. The discussion above did not clearly say how to overcome the error. Can one of you please summarize in steps on what has to be done?


	Logged

Pages: [1]

« previous next »

	Author	Topic: pp 1.5.7 and numpy bug (Read 13089 times)
0 Members and 1 Guest are viewing this topic.