Parallel Python Community Forums

Python Forums => Parallel Python Forum => Topic started by: Vitalii on February 23, 2009, 12:56:13 AM



Title: PP-1.5.7 is released!
Post by: Vitalii on February 23, 2009, 12:56:13 AM
Changes w.r.t. pp-1.5.6 (http://www.parallelpython.com/component/option,com_smf/Itemid,29/topic,274.msg803#msg803):

    1) Added ppworker restart after task completion functionality.
    2) Added pickle protocol option.
    3) Merges patch for Python 2.6 compatibility (contributed by mrtss).
    4) Merged patch for config file support (contributed by stevemilner).
    5) Documentation has been moved to doc folder.

Downloads:
http://www.parallelpython.com/downloads/pp/pp-1.5.7.zip
http://www.parallelpython.com/downloads/pp/pp-1.5.7.tar.gz
http://www.parallelpython.com/downloads/pp/pp-1.5.7.tar.bz2
http://www.parallelpython.com/downloads/pp/pp-1.5.7.exe

SVN source:

Code:
svn checkout http://parallelpython.googlecode.com/svn/trunk/ parallelpython-read-only

NOTE1: The documentation (http://www.parallelpython.com/content/view/15/30/) section of this website has been updated to reflect changes (1) and (2).

NOTE2: using proto=2 (and -n 2 argument of ppserver.py) significantly speeds up the network transfers.
The default is protocol 0.

NOTE3: ppworker restart after task completion is enabled by -r switch on ppserver and
by restart=True parameter for local ppworkers.
Use this option if you would like to run each task in a fresh environment.
This is useful if there are memory leaks in libraries or you would like to eliminate side effects.
This option also reduces the footprint of an idle ppserver.

NOTE4: Config file support requires configobj module.
Everything else will work without this module.
An example config file can be found at doc/example.config.

Regards,
Vitalii


Title: Re: PP-1.5.7 is released!
Post by: Vitalii on February 23, 2009, 11:55:43 PM
Here is a benchmark demonstrating the performance improvement with (proto=2 and -n2).

The code benchmarked against PP-1.5.7 on 100MB local LAN
(tasks were executed exclusively on a remote ppserver):
Code: (python)
import sys
import pp
 
N = 1000001
 
def reverse(l):
   """Calculates a sum"""
   l.reverse()
   return l
 
ppservers = ("192.168.10.101", )
proto = 0 # choose 2 for faster network transfers
 
job_server = pp.Server(0, ppservers=ppservers, proto=proto)
job1 = job_server.submit(reverse, (range(N), ))
result = job1()
print result[:10]
 
job_server.print_stats()
 

With proto=0 on local client and -n0 on remote server (default):
Code:
> python test-proto0.py
[1000000, 999999, 999998, 999997, 999996, 999995, 999994, 999993, 999992, 999991]
Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         1 |        100.00 |       1.5756 |     1.575559 | 192.168.10.101:60000
Time elapsed since server creation 3.76052188873

With proto=2  on local client and -n2 on remote server:
Code:
> python test-proto2.py
[1000000, 999999, 999998, 999997, 999996, 999995, 999994, 999993, 999992, 999991]
Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         1 |        100.00 |       0.3642 |     0.364187 | 192.168.10.101:60000
Time elapsed since server creation 1.48450303078

The speedup is 3.76/1.48 = 2.54 times


Title: Re: PP-1.5.7 is released!
Post by: jay on February 24, 2009, 12:55:45 PM
Thank you Vitalii!! I had about a week ago started using py26 and was having trouble getting my sytem to run reliably. I updated to 1.5.7 and it once again ran beatifully. Thanks again!

Your a hacker and a gentleman!
jt


Title: Re: PP-1.5.7 is released!
Post by: dsdale on March 02, 2009, 01:54:16 PM
How do you suggest the config file be used? For example, where should it be saved?


Title: Re: PP-1.5.7 is released!
Post by: Vitalii on March 24, 2009, 09:42:43 PM
Config can be used to store a default server configuration.

Also md5 sums have been added to the download page.


Title: Re: PP-1.5.7 is released!
Post by: dsdale on March 29, 2009, 10:16:16 AM
Could you provide an example in the tarball?


Title: Re: PP-1.5.7 is released!
Post by: Vitalii on March 31, 2009, 09:13:26 PM
There is an example in doc/example.config.

It is used to configure ppserver.py.
All parameters correspond to the command line parameters of ppserver.


Title: Re: PP-1.5.7 is released!
Post by: stevemilner on April 30, 2009, 06:27:51 PM
Somehow I missed this release on the packaging end ... sorry about that guys :-[! I'm working on builds for Fedora (devel, 11, 10). If you need this in an earlier version of Fedora please let me know.


Title: Re: PP-1.5.7 is released!
Post by: stevemilner on April 30, 2009, 06:58:57 PM
You can watch the progress here:

Fedora Devel - (should be there shortly on it's own)
Fedora 11 - https://admin.fedoraproject.org/updates/python-pp-1.5.7-2.fc11
Fedora 10 - https://admin.fedoraproject.org/updates/python-pp-1.5.7-2.fc10