Parallel Python Community Forums

Python Forums => Parallel Python Forum => Topic started by: KSJ on May 08, 2012, 10:50:43 PM



Title: how to prevent job rescheduling?
Post by: KSJ on May 08, 2012, 10:50:43 PM
Hi all,
I use pp for a calculation, found some strange thing and have some question:

I use time.time() to know the whole process spend about 168.969000101 sec

print job statistics:

Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         3 |          4.11 |       0.0000 |     0.000000 | 192.168.1.101:59999
        43 |         58.90 |       0.0000 |     0.000000 | 140.112.63.243:60000
        14 |         19.18 |       0.0000 |     0.000000 | 140.112.63.132:59996
         6 |          8.22 |     236.5780 |    39.429667 | local
         7 |          9.59 |       0.0000 |     0.000000 | 192.168.1.168:59997
Time elapsed since server creation 168.969000101
WARNING: statistics provided above is not accurate due to job rescheduling

I don't know when pp will rescheduling?
1.send job to workerA but it has no response or still calculating for certain sec?
  ( is that depend on TRANSPORT_SOCKET_TIMEOUT ?? )
  ( I try to change TRANSPORT_SOCKET_TIMEOUT  to 3600 but no use )
2.pp find that workerA is faster than workerB, so pp send the same job to workerA and cancel workerB??
  (the speed of these computers are very different)

Why local job time sum is 236 > 168 ?? b/s rescheduling??

I try to divide job into small piece, and job statistics work great
  ( but may spend more time while communicate, right? ),
  so what could I setting for use large task with job statstics

thx


Title: Re: how to prevent job rescheduling?
Post by: Vitalii on June 02, 2012, 12:22:32 AM
Why local job time sum is 236 > 168 ?? b/s rescheduling??
That is because it run them in parallel.
To debug it better please run ppserver.py with -d flag.