Parallel Python
Home arrow Forum arrow Python Forums arrow Parallel Python Forum arrow pp 1.5.7 and numpy bug

Monitor your credit card charges online for free at BillGuard.com
Parallel Python Community Forums rss  
May 20, 2013, 02:49:48 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Parallel python forum is up and running!
 
   Home   Help Search Login Register  
Parallel Python Community Forums > Python Forums > Parallel Python Forum > pp 1.5.7 and numpy bug
Pages: [1]
« previous next »
  Print  
Author Topic: pp 1.5.7 and numpy bug  (Read 13022 times)
0 Members and 1 Guest are viewing this topic.
psederberg
Full Python
***

Karma: 0
Posts: 14


View Profile
« on: January 23, 2010, 09:34:15 PM »
Hi PP-folks:

I've getting a very strange bug when I put use numpy as a dependent module.  This code fails:

Code:
import pp
import numpy

def get_val():
    return numpy.random.rand()

job_server = pp.Server()
jobs = []
for i in xrange(5):
    jobs.append(job_server.submit(get_val,
                                  (),
                                  (),
                                  ("numpy",)))
for i,job in enumerate(jobs):
    print '%d: %f' % (i,job())

with the following error:

Code:
$ python breakpp2.py
An error has occured during the module import
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
    globals()[module.split('.')[0]] = __import__(module)
  File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
    import add_newdocs
  File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
    from lib import add_newdoc
  File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: No module named multiarray
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 2, in get_val
NameError: global name 'numpy' is not defined
Traceback (most recent call last):
  File "breakpp2.py", line 18, in <module>
    print '%d: %f' % (i,job())
TypeError: float argument required

The crazy thing is that I can import numpy just fine from the python interpreter.  Also, the code above runs if I replace numpy with the random module, so it's specifically an issue importing numpy in a worker process.

Any ideas?

Thanks,
Per

PS-> I'm running Debian testing 32-bit with python 2.5.
Logged
Midnighter
Full Python
***

Karma: 0
Posts: 13


View Profile
« Reply #1 on: January 25, 2010, 01:49:28 AM »
Hmm, this line makes me suspicious:

Code:
File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx

In my personal code using import ... as ... did not work with parallel python, maybe that's the same problem with numpy. Can you edit the numpy file accordingly and try again?
Logged
psederberg
Full Python
***

Karma: 0
Posts: 14


View Profile
« Reply #2 on: January 25, 2010, 11:58:29 AM »
Hi Midnighter:

There are thousands of instances of import ... as ... in the numpy project, and, given that everything had been working great up until recently, I think its something with the environment on the worker when pp imports numpy. 

Does anyone know if the worker imports are done in a special way that might induce this error out of numpy?

Does it matter that multiarray.so is an compiled object and not a python module when it is clearly many levels down in the numpy imports?  Given that it is getting imported on the worker side, it doesn't seem like it needs to be pickle-able.

Can someone else check to see if the sample code from my initial post breaks on their system?  It is breaking on the three machines I have access to that also have parallel python on them.

Thanks,
Per
Logged
psederberg
Full Python
***

Karma: 0
Posts: 14


View Profile
« Reply #3 on: January 27, 2010, 07:35:37 AM »
OK, I downloaded and installed version 1.5.4 of pp and the code I posted above works without a problem.  There are handfuls of other recent posts getting similar errors with 1.5.7, so this implies that something has changed between 1.5.4 and 1.5.7 to cause these imports to fail.

So, version 1.5.4 works and version 1.5.7 does not.  Does anyone know what could be causing the error?

Thanks,
Per
Logged
Vitalii
Global Moderator
Parallel Python
*****

Karma: 2
Posts: 516


View Profile WWW
« Reply #4 on: January 27, 2010, 09:29:09 AM »
PP 1.5.7 does not fail with python 2.6.4 on this example.
python breakpp2.py
0: 0.084370
1: 0.628528
2: 0.787746
3: 0.411172
4: 0.222136

That makes me think that there was something in python 2.5 update that causes the failure.
Logged
Join Python Blogs!

Parallel Python
aditya
Jr. Python
**

Karma: 0
Posts: 5


View Profile
« Reply #5 on: January 27, 2010, 09:40:21 AM »
Yup,
tried importing numpy for the worker jobs using python 2.6.4 . No issues as Vitalii says. Looks like python 2.5 vs 2.6 here.
Best,
Aditya.
Logged
psederberg
Full Python
***

Karma: 0
Posts: 14


View Profile
« Reply #6 on: January 27, 2010, 09:43:34 AM »
Yes, and using proto=2 seems to fix it in python2.5, so maybe that will help pinpoint the issue in the code...

Best,
Per
Logged
Vitalii
Global Moderator
Parallel Python
*****

Karma: 2
Posts: 516


View Profile WWW
« Reply #7 on: January 27, 2010, 11:03:32 PM »
Since it was working before and now got broken, I suppose it is limited to a specific version of Python 2.5 (or perhaps a version of numpy). I've tested it with python 2.5 version I have and numpy 1.4.0:
/opt/pythons/bin/python2.5 --version
Python 2.5
/opt/pythons/bin/python2.5 breakpp2.py
0: 0.939030
1: 0.981239
2: 0.089505
3: 0.721238
4: 0.924152

and was not able to reproduce this issue.
Logged
Join Python Blogs!

Parallel Python
Vitalii
Global Moderator
Parallel Python
*****

Karma: 2
Posts: 516


View Profile WWW
« Reply #8 on: January 27, 2010, 11:05:31 PM »
Could you please run the following script and post the output:
Code: (python)
import pp, os, sys, numpy
 
def get_val(i):
    print "Job ", i, sys.version, os.environ.get("PYTHONPATH")
    print numpy.random.rand()
 
job_server = pp.Server()
print "Main", numpy.random.rand()
jobs = []
for i in xrange(5):
    jobs.append(job_server.submit(get_val, (i, ), (), ("numpy",)))
 
print sys.version, os.environ.get("PYTHONPATH")
for i,job in enumerate(jobs):
    job()
 
Logged
Join Python Blogs!

Parallel Python
psederberg
Full Python
***

Karma: 0
Posts: 14


View Profile
« Reply #9 on: January 28, 2010, 05:18:39 PM »
Hi Vitalii:

Here's the output:

Code:
$ python fromv.py
Main 0.288405616544
2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the module import
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
    globals()[module.split('.')[0]] = __import__(module)
  File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
    import add_newdocs
  File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
    from lib import add_newdoc
  File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: No module named multiarray
Job  0 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
An error has occured during the module import
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 55, in preprocess
    globals()[module.split('.')[0]] = __import__(module)
  File "/usr/share/pyshared/numpy/__init__.py", line 130, in <module>
    import add_newdocs
  File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
    from lib import add_newdoc
  File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: No module named multiarray
Job  1 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job  2 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job  3 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined
Job  4 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
An error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/ppworker.py", line 97, in run
    __result = __f(*__args)
  File "<string>", line 3, in get_val
NameError: global name 'numpy' is not defined


Now, here's the interesting part!  If I check out pp from the google code repository, and make the following change to the code:

Code:
$ git diff
diff --git a/pp.py b/pp.py
index 82c9dab..4d60780 100644
--- a/pp.py
+++ b/pp.py
@@ -360,7 +360,8 @@ class Server(object):
                 raise TypeError("secret must be of a string type")
             self.secret = str(secret)
         elif hasattr(user, "pp_secret"):
-            secret = user["pp_secret"]
+            #secret = user["pp_secret"]
+            secret = getattr(user,"pp_secret")
             if not isinstance(secret, types.StringType):
                 raise TypeError("secret must be of a string type")
             self.secret = str(secret)

BTW, is there some version of the user module that treats its attributes as a dict?  My Python2.5 requires attributes... 

Anyway, I point at that code with:

Code:
$ export PYTHONPATH=~/code/other/pp/parallelpython-read-only:$PYTHONPATH

and it runs without error!!!

Code:
$ python fromv.py
Main 0.721674871376
2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
Job  0 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.374807427939
Job  1 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.43653652051
Job  2 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.218321394857
Job  3 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.183824951636
Job  4 2.5.4 (r254:67916, Jan 24 2010, 12:18:00)
[GCC 4.4.3] /home/per/lib/python/SquareMap-1.0.0b22-py2.5.egg::/usr/lib/python2.5/site-packages/PIL:/home/per/lib/python:/tmp:/usr/lib/python2.5/site-packages:/usr/lib/python2.5/plat-linux2:/home/per/lib/python/cocos2d-0.3.0-py2.5.egg:/home/per/lib/python/argparse-0.8.0-py2.5.egg:/home/per/lib/python/RunSnakeRun-2.0.0b4-py2.5.egg:/usr/lib/python2.5/lib-tk:/usr/local/lib/python2.5/site-packages:/home/per/lib/python/grin-1.1.1-py2.5.egg:/usr/lib/python2.5/site-packages/gst-0.10:/usr/lib/python2.5/lib-dynload:/home/per/code/other/pp/parallelpython-read-only:/usr/lib/pymodules/python2.5:/usr/lib/pymodules/python2.5/gtk-2.0:/home/per/lib/python/pyglet-1.1.2-py2.5.egg:/usr/lib/python2.5:/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode
0.863557457308

Do you think this could be some weird Debian packaging bug?  I am using the Debian Testing version of python-pp_1.5.7-1.  The only real diffs I found was the user module change in pp and the fact that they don't like the use of a default secret, which is clearly not the error.

Perhaps I should submit this as a bug to the Debian folks...

Thanks for looking into this.  In the meantime, I can easily just use the version from source.  All my code is running just fine with it.

Best,
Per
Logged
Vitalii
Global Moderator
Parallel Python
*****

Karma: 2
Posts: 516


View Profile WWW
« Reply #10 on: February 06, 2010, 09:02:32 AM »
It is great that it worked out for you at the end.

I am not sure what is the rootcause of the issue, but I would not expect Debian Testing to be a stable distribution.
Logged
Join Python Blogs!

Parallel Python
csh
New Python
*

Karma: 0
Posts: 2


View Profile
« Reply #11 on: March 31, 2010, 05:51:12 PM »
Same problem here on Ubuntu 9.10 using the default packages (Python 2.6.4-0ubuntu3, NumPy 1.3.0-3, Parallel Python 1.5.6-2ubuntu1) - as stable as it gets. Using pp 1.6.0-RC3 resolved the issue.
Christoph
Logged
Camillio
New Python
*

Karma: 0
Posts: 1


View Profile
« Reply #12 on: September 16, 2010, 09:20:35 AM »
The same issue with pp 1.6 on debian unstable. If I compare the sys.path in the worker's __main__():

Code:
['/usr/share/pyshared', '/usr/lib/python2.6/plat-linux2', '/home/user/python', '/usr/lib/pymodules/python2.6/gtk-2.0', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6', '/usr/lib/python2.6/lib-dynload', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/lib-old']

and the sys.path in the script I run:
Code:
['/home/user', '/usr/lib/pymodules/python2.6', '/home/user/python', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6/gtk-2.0', '/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode']

I can see that these two differ. Python complained about things in /usr/share/pyshared/numpy, which led me to try the following nasty hack - addition of sys.path.pop() to /usr/lib/pymodules/python2.6/ppworker.py:
Code:
if __name__ == "__main__":
        # add the directory with ppworker.py to the path
        sys.path.append(os.path.dirname(__file__))
        sys.path.pop(0)
        f.close()
        wp = _WorkerProcess()
        wp.run()

And voila, it works  Cheesy  So I think that there is some issue with setting worker's sys.path.
Logged
Vitalii
Global Moderator
Parallel Python
*****

Karma: 2
Posts: 516


View Profile WWW
« Reply #13 on: October 16, 2010, 08:42:23 PM »
Problem arises from the fact that /usr/share/pyshared is added to the sys.path.
But it is hard to say why it is getting added and by whom.
Little be better workaround would be to remove exactly the offending item:
Code: (python)
sys.path.remove('/usr/share/pyshared')
Logged
Join Python Blogs!

Parallel Python
ckn
New Python
*

Karma: 0
Posts: 2


View Profile
« Reply #14 on: November 09, 2010, 04:44:23 PM »
I too am getting similar errors when i try to use numpy package. The discussion above did not clearly say how to overcome the error. Can one of you please summarize in steps on what has to be done?
Logged
Pages: [1]
  Print  
« previous next »
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.16 | SMF © 2011, Simple Machines Valid XHTML 1.0! Valid CSS!
Nutrition facts and analysis
By using this website, you signify your acceptance of Terms and Conditions and Privacy Policy.
Parallel Python Solutions | Parallel Python Forums | Parallel Python Community
Parallel Python © 2005 - 2013 Vitalii Vanovschi. All rights reserved