Development for LHC@home

This job appers to be mixing two actions together:
===> [runRivet] Tue May 17 16:12:35 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - sherpa 2.1.0 default 90000 264]

That's the same job I reported yesterday with the exception messages.
What you have shown is so far so good.
The only useless action is the "updating display" during initializing and optimization phases of a sherpa job.
Histograms can only be created when a few thousand events are processed.
But later on that job it starts looping after 32 the same exception messages.

Challenge is running Sherpa-Tasks,
when they finished correct in about 8.500 seconds - Two and a half hour duration.

Saw this today on two own PC's.

Other tasks under Challenge (for example Herwig++) need up to half an hour.

Challenge is running Sherpa-Tasks,
when they finished correct in about 8.500 seconds - Two and a half hour duration.

The duration of a Sherpa job fully depends on the used parameters.
In the past I've even seen sherpa jobs running longer than 24 hours when nevts=100000.

A few hours ago I had a looping Sherpa again in a Challenge 2015 VM:

===> [runRivet] Wed May 18 14:20:15 CEST 2016 [boinc ee zhad 133 - - sherpa 1.2.3 default 100000 220]

Is there any progress on the multiple processing of the same job issue, yet?

It would really be nice , if admin could, every once in a while , actually say something?
Just a few words, that we know, admin is still alive.

We talked about improving communication a while back.It did not help.

We disabled the Condor jobs automatic resubmission for the moment.
We are not sure it was the cause of duplicating job so it would be good to hear back from you.

Many thanks.

The very first two tasks, i started are identical jobs.

0===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
4===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]

Output of the job wrapper may appear here.
15:13:54 +0200 2016-05-19 [INFO] New Job Starting
15:13:54 +0200 2016-05-19 [INFO] Condor JobID: 314816
15:13:59 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463

Output of the job wrapper may appear here.
15:13:54 +0200 2016-05-19 [INFO] New Job Starting
15:13:54 +0200 2016-05-19 [INFO] Condor JobID: 314815
15:13:59 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463

So it looks like duplicate jobs appear only in different tasks, never in the same task.
Can you please retrieve the MC-plot id number and the Condor job one for them?
Do you see a "jobdata" file in the job execution dir?

So it looks like duplicate jobs appear only in different tasks, never in the same task.

Incorrect.


The second job in each task is identical to the ones before

0===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
4===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
0===> [runRivet] Thu May 19 15:58:07 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
4===> [runRivet] Thu May 19 15:57:32 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]

Output of the job wrapper may appear here.
15:13:54 +0200 2016-05-19 [INFO] New Job Starting
15:13:54 +0200 2016-05-19 [INFO] Condor JobID: 314816
15:13:59 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463
15:58:03 +0200 2016-05-19 [INFO] Job finished with 0.
15:58:07 +0200 2016-05-19 [INFO] New Job Starting
15:58:07 +0200 2016-05-19 [INFO] Condor JobID: 314840
15:58:13 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463

Output of the job wrapper may appear here.
15:13:54 +0200 2016-05-19 [INFO] New Job Starting
15:13:54 +0200 2016-05-19 [INFO] Condor JobID: 314815
15:13:59 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463
15:57:28 +0200 2016-05-19 [INFO] Job finished with 0.
15:57:31 +0200 2016-05-19 [INFO] New Job Starting
15:57:32 +0200 2016-05-19 [INFO] Condor JobID: 314839
15:57:37 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463

I finally got a differnt job.

0===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
4===> [runRivet] Thu May 19 15:13:54 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
0===> [runRivet] Thu May 19 15:58:07 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
4===> [runRivet] Thu May 19 15:57:32 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.212 default-CD 100000 268]
0===> [runRivet] Thu May 19 16:41:51 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - pythia8 8.210 default-CD 100000 268]
4===> [runRivet] Thu May 19 16:41:12 CEST 2016 [boinc ppbar z 1960 -,-,50,120 - herwig++ 2.5.2 LHC-UE-EE-3-2760 100000 268]

The first number in each row is a slot number to tell the tasks apart.

Ok thanks, we are investigating with Condor experts.
If you can find the "jobdata" file in the job execution dir it would be helpful to know the "runid" and "seed" values listed in the first lines.

Do you mean running.log?

I don't know if "runid" and "seed" are written in the running.log, for sure they are in "jobdata".
Wherever you find them is fine.

Ther is no such word in any of the "show graphics" log files.

Do you mean this:
Unpack data histograms...
dataFiles =
/cvmfs/sft.cern.ch/lcg/external/MCGenerators_lcgcmt65/rivet/2.4.0/x86_64-slc6-gcc47-opt/share/Rivet/D0_2010_S8671338.yoda
/cvmfs/sft.cern.ch/lcg/external/MCGenerators_lcgcmt65/rivet/2.4.0/x86_64-slc6-gcc47-opt/share/Rivet/D0_2010_S8821313.yoda
output = /var/lib/condor/execute/dir_4349/tmp/tmp.5sRZTQbz2H/flat
make: Entering directory `/var/lib/condor/execute/dir_4349/rivetvm'
g++ yoda2flat-split.cc -o yoda2flat-split.exe -Wfatal-errors -Wl,-rpath /cvmfs/sft.cern.ch/lcg/external/MCGenerators_lcgcmt65/yoda/1.5.5/x86_64-slc6-gcc47-opt/lib `/cvmfs/sft.cern.ch/lcg/external/MCGenerators_lcgcmt65/yoda/1.5.5/x86_64-slc6-gcc47-opt/bin/yoda-config --cppflags --libs`
make: Leaving directory `/var/lib/condor/execute/dir_4349/rivetvm'

The "jobdata" file should be in /var/lib/condor/execute/dir_4349/
as from your output.

You should know, as a volunteer, i cannot get into the virtual box.

After a one job break the same task gets the same job again:

Output of the job wrapper may appear here.
15:13:54 +0200 2016-05-19 [INFO] New Job Starting
15:13:54 +0200 2016-05-19 [INFO] Condor JobID: 314815
15:13:59 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463
15:57:28 +0200 2016-05-19 [INFO] Job finished with 0.
15:57:31 +0200 2016-05-19 [INFO] New Job Starting
15:57:32 +0200 2016-05-19 [INFO] Condor JobID: 314839
15:57:37 +0200 2016-05-19 [INFO] MCPlots JobID: 30486463
16:41:08 +0200 2016-05-19 [INFO] Job finished with 0.
16:41:12 +0200 2016-05-19 [INFO] New Job Starting
16:41:12 +0200 2016-05-19 [INFO] Condor JobID: 314866
16:41:17 +0200 2016-05-19 [INFO] MCPlots JobID: 30484435
17:17:10 +0200 2016-05-19 [INFO] Job finished with 0.
17:17:14 +0200 2016-05-19 [INFO] New Job Starting
17:17:14 +0200 2016-05-19 [INFO] Condor JobID: 314894
17:17:19 +0200 2016-05-19 [INFO] MCPlots JobID: 30484603
17:48:52 +0200 2016-05-19 [INFO] Job finished with 0.
17:48:56 +0200 2016-05-19 [INFO] New Job Starting
17:48:56 +0200 2016-05-19 [INFO] Condor JobID: 314922
17:49:01 +0200 2016-05-19 [INFO] MCPlots JobID: 30483844
18:44:47 +0200 2016-05-19 [INFO] Job finished with 0.
18:44:52 +0200 2016-05-19 [INFO] New Job Starting
18:44:52 +0200 2016-05-19 [INFO] Condor JobID: 314973
18:44:57 +0200 2016-05-19 [INFO] MCPlots JobID: 30484603

After running a number of jobs: More than 90% of all new jobs are repeated.

That is only within one computer. I do not know, how may times the same job has been run on other computers.

Ok I will try to submit jobs from a different place every time.
I will let you know when "fresh" jobs will be available.

Thanks

Thanks Leonardo.

From Condor JobID 316094 onwards jobs are submitted from a different location every 15'.
Let's see if that solves the problem.