Name pa5MDmPvv2unShfckohDCDFpABFKDmABFKDmGN1ODmABFKDmOPDebn_0
Workunit 1907795
Created 3 Jul 2019, 15:02:05 UTC
Sent 3 Jul 2019, 15:09:46 UTC
Report deadline 10 Jul 2019, 15:09:46 UTC
Received 5 Jul 2019, 17:49:59 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 195 (0x000000C3) EXIT_CHILD_FAILED
Computer ID 2244
Run time 1 days 15 hours 0 min 17 sec
CPU time 1 hours 38 min 28 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 2.09 GFLOPS
Application version ATLAS Simulation v0.62 (native_mt)
x86_64-pc-linux-gnu
Peak working set size 1.49 GB
Peak swap size 2.13 GB
Peak disk usage 734.06 MB

Stderr output

<core_client_version>7.5.1</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)
</message>
<stderr_txt>
17:11:03 (19581): wrapper (7.7.26015): starting
17:11:03 (19581): wrapper: running run_atlas (--nthreads 1)
singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img
sys.argv = ['run_atlas', '--nthreads', '1']
THREADS=1
Checking for CVMFS
CVMFS is installed
OS:Scientific Linux release 6.10 (Carbon)

This is SLC or CentOS release 6, run the atlas job without Singularity
copy /root/Downloads/BOINC/slots/7/shared/input.tar.gz
copy /root/Downloads/BOINC/slots/7/shared/start_atlas.sh
copy /root/Downloads/BOINC/slots/7/shared/ATLAS.root_0
copy /root/Downloads/BOINC/slots/7/shared/RTE.tar.gz
start atlas job with 
cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err
06:08:57 (2677): wrapper (7.7.26015): starting
06:08:57 (2677): wrapper: running run_atlas (--nthreads 1)
singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img
sys.argv = ['run_atlas', '--nthreads', '1']
THREADS=1
This is not an Event Service job
This is trying to run the run_atlas wrapper for the 2nd time,but it is not an Event Service job,so will restart the job
output.list does not exist...
Checking for CVMFS
CVMFS is installed
OS:Scientific Linux release 6.10 (Carbon)

This is SLC or CentOS release 6, run the atlas job without Singularity
copy /root/Downloads/BOINC/slots/7/shared/input.tar.gz
copy /root/Downloads/BOINC/slots/7/shared/start_atlas.sh
copy /root/Downloads/BOINC/slots/7/shared/ATLAS.root_0
copy /root/Downloads/BOINC/slots/7/shared/RTE.tar.gz
start atlas job with 
cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err
20:40:43 (15610): wrapper (7.7.26015): starting
20:40:43 (15610): wrapper: running run_atlas (--nthreads 1)
singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img
sys.argv = ['run_atlas', '--nthreads', '1']
THREADS=1
This is not an Event Service job
This is trying to run the run_atlas wrapper for the 2nd time,but it is not an Event Service job,so will restart the job
output.list does not exist...
Checking for CVMFS
CVMFS is installed
OS:Scientific Linux release 6.10 (Carbon)

This is SLC or CentOS release 6, run the atlas job without Singularity
copy /root/Downloads/BOINC/slots/7/shared/input.tar.gz
copy /root/Downloads/BOINC/slots/7/shared/start_atlas.sh
copy /root/Downloads/BOINC/slots/7/shared/ATLAS.root_0
copy /root/Downloads/BOINC/slots/7/shared/pilot2
caught an exception running start_atlas

running start_atlas return value is 4
tar cvf shared/result.tar.gz runtime_log.err runtime_log pilotlog.txt log.18411005._044702.job.log.1

*****************The last 100 lines of the pilot log******************
2019-07-05 15:19:26,238 | INFO     | monitor             | pilot.control.monitor            | control                   | 126577 s have passed since pilot start
2019-07-05 15:21:26,637 | INFO     | monitor             | pilot.control.monitor            | control                   | 126698 s have passed since pilot start
2019-07-05 15:23:27,004 | INFO     | monitor             | pilot.control.monitor            | control                   | 126818 s have passed since pilot start
2019-07-05 15:25:27,434 | INFO     | monitor             | pilot.control.monitor            | control                   | 126938 s have passed since pilot start
2019-07-05 15:27:27,826 | INFO     | monitor             | pilot.control.monitor            | control                   | 127059 s have passed since pilot start
2019-07-05 15:29:28,238 | INFO     | monitor             | pilot.control.monitor            | control                   | 127179 s have passed since pilot start
2019-07-05 15:31:28,685 | INFO     | monitor             | pilot.control.monitor            | control                   | 127300 s have passed since pilot start
2019-07-05 15:33:29,046 | INFO     | monitor             | pilot.control.monitor            | control                   | 127420 s have passed since pilot start
2019-07-05 15:35:29,443 | INFO     | monitor             | pilot.control.monitor            | control                   | 127540 s have passed since pilot start
2019-07-05 15:37:29,826 | INFO     | monitor             | pilot.control.monitor            | control                   | 127661 s have passed since pilot start
2019-07-05 15:39:30,286 | INFO     | monitor             | pilot.control.monitor            | control                   | 127781 s have passed since pilot start
2019-07-05 15:41:30,715 | INFO     | monitor             | pilot.control.monitor            | control                   | 127902 s have passed since pilot start
2019-07-05 15:43:31,124 | INFO     | monitor             | pilot.control.monitor            | control                   | 128022 s have passed since pilot start
2019-07-05 15:45:31,517 | INFO     | monitor             | pilot.control.monitor            | control                   | 128143 s have passed since pilot start
2019-07-05 15:47:31,958 | INFO     | monitor             | pilot.control.monitor            | control                   | 128263 s have passed since pilot start
2019-07-05 15:49:32,347 | INFO     | monitor             | pilot.control.monitor            | control                   | 128383 s have passed since pilot start
2019-07-05 15:51:32,750 | INFO     | monitor             | pilot.control.monitor            | control                   | 128504 s have passed since pilot start
2019-07-05 15:53:33,118 | INFO     | monitor             | pilot.control.monitor            | control                   | 128624 s have passed since pilot start
2019-07-05 15:55:33,489 | INFO     | monitor             | pilot.control.monitor            | control                   | 128745 s have passed since pilot start
2019-07-05 15:57:33,895 | INFO     | monitor             | pilot.control.monitor            | control                   | 128865 s have passed since pilot start
2019-07-05 15:59:34,252 | INFO     | monitor             | pilot.control.monitor            | control                   | 128985 s have passed since pilot start
2019-07-05 16:01:34,704 | INFO     | monitor             | pilot.control.monitor            | control                   | 129106 s have passed since pilot start
2019-07-05 16:03:35,076 | INFO     | monitor             | pilot.control.monitor            | control                   | 129226 s have passed since pilot start
2019-07-05 16:05:35,504 | INFO     | monitor             | pilot.control.monitor            | control                   | 129347 s have passed since pilot start
2019-07-05 16:07:35,919 | INFO     | monitor             | pilot.control.monitor            | control                   | 129467 s have passed since pilot start
2019-07-05 16:09:36,372 | INFO     | monitor             | pilot.control.monitor            | control                   | 129587 s have passed since pilot start
2019-07-05 16:11:36,815 | INFO     | monitor             | pilot.control.monitor            | control                   | 129708 s have passed since pilot start
2019-07-05 16:13:37,199 | INFO     | monitor             | pilot.control.monitor            | control                   | 129828 s have passed since pilot start
2019-07-05 16:15:37,617 | INFO     | monitor             | pilot.control.monitor            | control                   | 129949 s have passed since pilot start
2019-07-05 16:17:38,042 | INFO     | monitor             | pilot.control.monitor            | control                   | 130069 s have passed since pilot start
2019-07-05 16:19:38,433 | INFO     | monitor             | pilot.control.monitor            | control                   | 130189 s have passed since pilot start
2019-07-05 16:21:39,300 | INFO     | monitor             | pilot.control.monitor            | control                   | 130310 s have passed since pilot start
2019-07-05 16:23:39,889 | INFO     | monitor             | pilot.control.monitor            | control                   | 130431 s have passed since pilot start
2019-07-05 16:25:40,264 | INFO     | monitor             | pilot.control.monitor            | control                   | 130551 s have passed since pilot start
2019-07-05 16:27:40,667 | INFO     | monitor             | pilot.control.monitor            | control                   | 130672 s have passed since pilot start
2019-07-05 16:29:41,113 | INFO     | monitor             | pilot.control.monitor            | control                   | 130792 s have passed since pilot start
2019-07-05 16:31:41,478 | INFO     | monitor             | pilot.control.monitor            | control                   | 130913 s have passed since pilot start
2019-07-05 16:33:41,833 | INFO     | monitor             | pilot.control.monitor            | control                   | 131033 s have passed since pilot start
2019-07-05 16:35:42,199 | INFO     | monitor             | pilot.control.monitor            | control                   | 131153 s have passed since pilot start
2019-07-05 16:37:42,630 | INFO     | monitor             | pilot.control.monitor            | control                   | 131274 s have passed since pilot start
2019-07-05 16:39:43,015 | INFO     | monitor             | pilot.control.monitor            | control                   | 131394 s have passed since pilot start
2019-07-05 16:41:43,398 | INFO     | monitor             | pilot.control.monitor            | control                   | 131514 s have passed since pilot start
2019-07-05 16:43:43,833 | INFO     | monitor             | pilot.control.monitor            | control                   | 131635 s have passed since pilot start
2019-07-05 16:45:44,229 | INFO     | monitor             | pilot.control.monitor            | control                   | 131755 s have passed since pilot start
2019-07-05 16:47:44,609 | INFO     | monitor             | pilot.control.monitor            | control                   | 131876 s have passed since pilot start
2019-07-05 16:49:44,959 | INFO     | monitor             | pilot.control.monitor            | control                   | 131996 s have passed since pilot start
2019-07-05 16:51:45,453 | INFO     | monitor             | pilot.control.monitor            | control                   | 132116 s have passed since pilot start
2019-07-05 16:53:45,825 | INFO     | monitor             | pilot.control.monitor            | control                   | 132237 s have passed since pilot start
2019-07-05 16:55:46,265 | INFO     | monitor             | pilot.control.monitor            | control                   | 132357 s have passed since pilot start
2019-07-05 16:57:46,646 | INFO     | monitor             | pilot.control.monitor            | control                   | 132478 s have passed since pilot start
2019-07-05 16:59:47,056 | INFO     | monitor             | pilot.control.monitor            | control                   | 132598 s have passed since pilot start
2019-07-05 17:01:47,450 | INFO     | monitor             | pilot.control.monitor            | control                   | 132718 s have passed since pilot start
2019-07-05 17:03:47,869 | INFO     | monitor             | pilot.control.monitor            | control                   | 132839 s have passed since pilot start
2019-07-05 17:05:48,351 | INFO     | monitor             | pilot.control.monitor            | control                   | 132959 s have passed since pilot start
2019-07-05 17:07:48,747 | INFO     | monitor             | pilot.control.monitor            | control                   | 133080 s have passed since pilot start
2019-07-05 17:09:49,157 | INFO     | monitor             | pilot.control.monitor            | control                   | 133200 s have passed since pilot start
2019-07-05 17:11:49,531 | INFO     | monitor             | pilot.control.monitor            | control                   | 133321 s have passed since pilot start
2019-07-05 17:13:49,863 | INFO     | monitor             | pilot.control.monitor            | control                   | 133441 s have passed since pilot start
2019-07-05 17:15:50,236 | INFO     | monitor             | pilot.control.monitor            | control                   | 133561 s have passed since pilot start
2019-07-05 17:17:50,530 | INFO     | monitor             | pilot.control.monitor            | control                   | 133682 s have passed since pilot start
2019-07-05 17:19:50,828 | INFO     | monitor             | pilot.control.monitor            | control                   | 133802 s have passed since pilot start
2019-07-05 17:21:51,190 | INFO     | monitor             | pilot.control.monitor            | control                   | 133922 s have passed since pilot start
2019-07-05 17:23:51,515 | INFO     | monitor             | pilot.control.monitor            | control                   | 134043 s have passed since pilot start
2019-07-05 17:25:51,898 | INFO     | monitor             | pilot.control.monitor            | control                   | 134163 s have passed since pilot start
2019-07-05 17:27:52,237 | INFO     | monitor             | pilot.control.monitor            | control                   | 134283 s have passed since pilot start
2019-07-05 17:29:52,620 | INFO     | monitor             | pilot.control.monitor            | control                   | 134404 s have passed since pilot start
2019-07-05 17:31:53,034 | INFO     | monitor             | pilot.control.monitor            | control                   | 134524 s have passed since pilot start
2019-07-05 17:33:53,522 | INFO     | monitor             | pilot.control.monitor            | control                   | 134645 s have passed since pilot start
2019-07-05 17:35:53,930 | INFO     | monitor             | pilot.control.monitor            | control                   | 134765 s have passed since pilot start
2019-07-05 17:37:54,347 | INFO     | monitor             | pilot.control.monitor            | control                   | 134885 s have passed since pilot start
2019-07-05 17:39:54,711 | INFO     | monitor             | pilot.control.monitor            | control                   | 135006 s have passed since pilot start
2019-07-05 17:41:55,147 | INFO     | monitor             | pilot.control.monitor            | control                   | 135126 s have passed since pilot start
2019-07-05 17:43:55,576 | INFO     | monitor             | pilot.control.monitor            | control                   | 135247 s have passed since pilot start
2019-07-05 17:45:55,965 | INFO     | monitor             | pilot.control.monitor            | control                   | 135367 s have passed since pilot start
2019-07-05 17:47:56,371 | INFO     | monitor             | pilot.control.monitor            | control                   | 135487 s have passed since pilot start
2019-07-05 17:49:56,904 | INFO     | monitor             | pilot.control.monitor            | control                   | 135608 s have passed since pilot start
2019-07-05 17:51:57,302 | INFO     | monitor             | pilot.control.monitor            | control                   | 135728 s have passed since pilot start
2019-07-05 17:53:57,741 | INFO     | monitor             | pilot.control.monitor            | control                   | 135849 s have passed since pilot start
2019-07-05 17:55:58,140 | INFO     | monitor             | pilot.control.monitor            | control                   | 135969 s have passed since pilot start
2019-07-05 17:57:58,512 | INFO     | monitor             | pilot.control.monitor            | control                   | 136090 s have passed since pilot start
2019-07-05 17:59:58,953 | INFO     | monitor             | pilot.control.monitor            | control                   | 136210 s have passed since pilot start
2019-07-05 18:01:59,436 | INFO     | monitor             | pilot.control.monitor            | control                   | 136330 s have passed since pilot start
2019-07-05 18:03:59,852 | INFO     | monitor             | pilot.control.monitor            | control                   | 136451 s have passed since pilot start
2019-07-05 18:06:00,221 | INFO     | monitor             | pilot.control.monitor            | control                   | 136571 s have passed since pilot start
2019-07-05 18:08:00,604 | INFO     | monitor             | pilot.control.monitor            | control                   | 136692 s have passed since pilot start
2019-07-05 18:10:00,955 | INFO     | monitor             | pilot.control.monitor            | control                   | 136812 s have passed since pilot start
2019-07-05 18:12:01,331 | INFO     | monitor             | pilot.control.monitor            | control                   | 136932 s have passed since pilot start
2019-07-05 18:14:01,722 | INFO     | monitor             | pilot.control.monitor            | control                   | 137053 s have passed since pilot start
2019-07-05 18:16:02,112 | INFO     | monitor             | pilot.control.monitor            | control                   | 137173 s have passed since pilot start
2019-07-05 18:18:02,486 | INFO     | monitor             | pilot.control.monitor            | control                   | 137294 s have passed since pilot start
2019-07-05 18:20:02,870 | INFO     | monitor             | pilot.control.monitor            | control                   | 137414 s have passed since pilot start
2019-07-05 18:22:03,222 | INFO     | monitor             | pilot.control.monitor            | control                   | 137534 s have passed since pilot start
2019-07-05 18:24:03,604 | INFO     | monitor             | pilot.control.monitor            | control                   | 137655 s have passed since pilot start
2019-07-05 18:26:03,940 | INFO     | monitor             | pilot.control.monitor            | control                   | 137775 s have passed since pilot start
2019-07-05 18:28:04,282 | INFO     | monitor             | pilot.control.monitor            | control                   | 137895 s have passed since pilot start
2019-07-05 18:30:04,701 | INFO     | monitor             | pilot.control.monitor            | control                   | 138016 s have passed since pilot start
2019-07-05 18:32:05,193 | INFO     | monitor             | pilot.control.monitor            | control                   | 138136 s have passed since pilot start
2019-07-05 18:34:05,560 | INFO     | monitor             | pilot.control.monitor            | control                   | 138257 s have passed since pilot start
2019-07-05 18:36:05,979 | INFO     | monitor             | pilot.control.monitor            | control                   | 138377 s have passed since pilot start
2019-07-05 18:38:06,371 | INFO     | monitor             | pilot.control.monitor            | control                   | 138497 s have passed since pilot start
***************diag file************
runtimeenvironments=APPS/HEP/ATLAS-SITE;
nodename=APU8S
Processors=1
runtimeenvironments=APPS/HEP/ATLAS-SITE;
nodename=APU8S
Processors=1
******************************WorkDir***********************
insgesamt 381764
drwxrwx--x. 7 root root      4096  5. Jul 20:40 .
drwxr-x--x. 3 root root      4096  5. Jul 20:40 ..
-rw-------. 1 root root   7347416  3. Jul 17:12 agis_ddmendpoints.json
-rw-------. 1 root root   4642882  4. Jul 06:09 agis_schedconf.cvmfs.json
drwx------. 2 root root      4096  4. Jul 06:09 .alrb
drwxr-xr-x. 3 root root      4096  3. Jul 17:11 APPS
drwxr-xr-x. 2 root root      4096  3. Jul 17:12 .arc
-rw-------. 1 root root       549  3. Jul 17:11 .asetup
-rw-------. 1 root root      4198  3. Jul 17:13 .asetup.save
-rw-r--r--. 1 root root 377662155  5. Jul 20:40 ATLAS.root_0
-rw-r--r--. 1 root root         0  3. Jul 17:11 boinc_lockfile
-rw-r--r--. 1 root root      8192  5. Jul 20:40 boinc_mmap_file
-rw-r--r--. 1 root root       537  5. Jul 20:39 boinc_task_state.xml
-rw-------. 1 root root      1950  4. Jul 06:10 heartbeat.json
-rw-r--r--. 1 root root      5478  5. Jul 20:40 init_data.xml
-rw-r--r--. 1 root root    246328  5. Jul 20:40 input.tar.gz
-rw-r--r--. 1 root root       112  3. Jul 17:11 job.xml
-rw-------. 1 root root    267756  5. Jul 20:38 log.18411005._044702.job.log.1
-rw-------. 1 root root       138  4. Jul 06:09 pa5MDmPvv2unShfckohDCDFpABFKDmABFKDmGN1ODmABFKDmOPDebn.diag
-rw-------. 1 4871 1028      2887  3. Jul 17:00 pandaJobData.out
drwxrwx---. 2 root root      4096  4. Jul 06:10 PanDA_Pilot-4403244921
-rw-r--r--. 1 root root    237372 24. Jun 09:16 pilot2.tar.gz
-rw-------. 1 root root    337213  5. Jul 20:38 pilotlog.txt
-rw-r--r--. 1 root root      4468  3. Jul 17:00 queuedata.json
-rw-r--r--. 1 root root       815  4. Jul 06:09 RTE.tar.gz
-rwxr-xr-x. 1 root root      8512  3. Jul 17:11 run_atlas
-rwx------. 1 4871 1028     15232  3. Jul 17:00 runpilot2-wrapper.sh
-rw-r--r--. 1 root root       479  4. Jul 06:09 runtime_log
-rw-r--r--. 1 root root      4038  4. Jul 06:09 runtime_log.err
drwxrwx--x. 3 root root      4096  5. Jul 20:40 shared
-rw-r--r--. 1 root root      8659  5. Jul 20:40 start_atlas.sh
-rw-r--r--. 1 root root     18925  5. Jul 20:40 stderr.txt
-rw-r--r--. 1 root root       107  3. Jul 17:11 wrapper_26015_x86_64-pc-linux-gnu
-rw-r--r--. 1 root root        28  5. Jul 20:39 wrapper_checkpoint.txtparent process exit 4
child process exit 4
20:50:46 (15610): run_atlas exited; CPU time 0.342947
20:50:46 (15610): app exit status: 0x4
20:50:46 (15610): called boinc_finish(195)

</stderr_txt>
]]>


©2024 CERN