Parallel Python
Wednesday, 10 March 2010
Home arrow Forums arrow Python Forums arrow Parallel Python Forum arrow PP-1.5.7 is released!
Use PP to run your software in parallel
Write about Parallel Python in your blog
Participate in Parallel Python Forum
Parallel Python Community Forums rss  
March 10, 2010, 10:05:13 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Parallel python forum is up and running!
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: PP-1.5.7 is released!  (Read 6081 times)
0 Members and 1 Guest are viewing this topic.
Vitalii
Global Moderator
Sr. Member
*****
Posts: 358


View Profile WWW
« on: February 23, 2009, 12:56:13 AM »

Changes w.r.t. pp-1.5.6:

    1) Added ppworker restart after task completion functionality.
    2) Added pickle protocol option.
    3) Merges patch for Python 2.6 compatibility (contributed by mrtss).
    4) Merged patch for config file support (contributed by stevemilner).
    5) Documentation has been moved to doc folder.

Downloads:
http://www.parallelpython.com/downloads/pp/pp-1.5.7.zip
http://www.parallelpython.com/downloads/pp/pp-1.5.7.tar.gz
http://www.parallelpython.com/downloads/pp/pp-1.5.7.tar.bz2
http://www.parallelpython.com/downloads/pp/pp-1.5.7.exe

SVN source:

Code:
svn checkout http://parallelpython.googlecode.com/svn/trunk/ parallelpython-read-only

NOTE1: The documentation section of this website has been updated to reflect changes (1) and (2).

NOTE2: using proto=2 (and -n 2 argument of ppserver.py) significantly speeds up the network transfers.
The default is protocol 0.

NOTE3: ppworker restart after task completion is enabled by -r switch on ppserver and
by restart=True parameter for local ppworkers.
Use this option if you would like to run each task in a fresh environment.
This is useful if there are memory leaks in libraries or you would like to eliminate side effects.
This option also reduces the footprint of an idle ppserver.

NOTE4: Config file support requires configobj module.
Everything else will work without this module.
An example config file can be found at doc/example.config.

Regards,
Vitalii
« Last Edit: February 23, 2009, 11:50:41 PM by Vitalii » Logged

Vitalii
Global Moderator
Sr. Member
*****
Posts: 358


View Profile WWW
« Reply #1 on: February 23, 2009, 11:55:43 PM »

Here is a benchmark demonstrating the performance improvement with (proto=2 and -n2).

The code benchmarked against PP-1.5.7 on 100MB local LAN
(tasks were executed exclusively on a remote ppserver):
Code
import sys
import pp
 
N = 1000001
 
def reverse(l):
   """Calculates a sum"""
   l.reverse()
   return l
 
ppservers = ("192.168.10.101", )
proto = 0 # choose 2 for faster network transfers
 
job_server = pp.Server(0, ppservers=ppservers, proto=proto)
job1 = job_server.submit(reverse, (range(N), ))
result = job1()
print result[:10]
 
job_server.print_stats()
 

With proto=0 on local client and -n0 on remote server (default):
Code:
> python test-proto0.py
[1000000, 999999, 999998, 999997, 999996, 999995, 999994, 999993, 999992, 999991]
Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         1 |        100.00 |       1.5756 |     1.575559 | 192.168.10.101:60000
Time elapsed since server creation 3.76052188873

With proto=2  on local client and -n2 on remote server:
Code:
> python test-proto2.py
[1000000, 999999, 999998, 999997, 999996, 999995, 999994, 999993, 999992, 999991]
Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         1 |        100.00 |       0.3642 |     0.364187 | 192.168.10.101:60000
Time elapsed since server creation 1.48450303078

The speedup is 3.76/1.48 = 2.54 times
« Last Edit: February 24, 2009, 07:51:56 AM by Vitalii » Logged

jay
Jr. Member
**
Posts: 16


View Profile
« Reply #2 on: February 24, 2009, 12:55:45 PM »

Thank you Vitalii!! I had about a week ago started using py26 and was having trouble getting my sytem to run reliably. I updated to 1.5.7 and it once again ran beatifully. Thanks again!

Your a hacker and a gentleman!
jt
Logged
dsdale
Jr. Member
**
Posts: 9


View Profile
« Reply #3 on: March 02, 2009, 01:54:16 PM »

How do you suggest the config file be used? For example, where should it be saved?
Logged
Vitalii
Global Moderator
Sr. Member
*****
Posts: 358


View Profile WWW
« Reply #4 on: March 24, 2009, 09:42:43 PM »

Config can be used to store a default server configuration.

Also md5 sums have been added to the download page.
« Last Edit: March 24, 2009, 10:04:59 PM by Vitalii » Logged

dsdale
Jr. Member
**
Posts: 9


View Profile
« Reply #5 on: March 29, 2009, 10:16:16 AM »

Could you provide an example in the tarball?
Logged
Vitalii
Global Moderator
Sr. Member
*****
Posts: 358


View Profile WWW
« Reply #6 on: March 31, 2009, 09:13:26 PM »

There is an example in doc/example.config.

It is used to configure ppserver.py.
All parameters correspond to the command line parameters of ppserver.
Logged

stevemilner
Jr. Member
**
Posts: 8


View Profile WWW
« Reply #7 on: April 30, 2009, 06:27:51 PM »

Somehow I missed this release on the packaging end ... sorry about that guys Embarrassed! I'm working on builds for Fedora (devel, 11, 10). If you need this in an earlier version of Fedora please let me know.
Logged
stevemilner
Jr. Member
**
Posts: 8


View Profile WWW
« Reply #8 on: April 30, 2009, 06:58:57 PM »

You can watch the progress here:

Fedora Devel - (should be there shortly on it's own)
Fedora 11 - https://admin.fedoraproject.org/updates/python-pp-1.5.7-2.fc11
Fedora 10 - https://admin.fedoraproject.org/updates/python-pp-1.5.7-2.fc10
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.4 | SMF © 2006-2007, Simple Machines LLC Valid XHTML 1.0! Valid CSS!