Name 9lWLDma1NxunShfckohDCDFpABFKDmABFKDmieTTDmABFKDmBC9LVo_1
Workunit 1905419
Created 19 Jun 2019, 16:01:41 UTC
Sent 26 Jun 2019, 17:22:58 UTC
Report deadline 3 Jul 2019, 17:22:58 UTC
Received 28 Jun 2019, 5:59:53 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 2244
Run time 3 hours 48 min 55 sec
CPU time 3 hours 45 min 14 sec
Validate state Valid
Credit 66.57
Device peak FLOPS 2.09 GFLOPS
Application version ATLAS Simulation v0.62 (native_mt)
x86_64-pc-linux-gnu
Peak working set size 1.91 GB
Peak swap size 2.58 GB
Peak disk usage 769.43 MB

Stderr output

<core_client_version>7.5.1</core_client_version>
<![CDATA[
<stderr_txt>
20:18:08 (22602): wrapper (7.7.26015): starting
20:18:08 (22602): wrapper: running run_atlas (--nthreads 1)
singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img
sys.argv = ['run_atlas', '--nthreads', '1']
THREADS=1
Checking for CVMFS
CVMFS is installed
OS:Scientific Linux release 6.10 (Carbon)

This is SLC or CentOS release 6, run the atlas job without Singularity
copy /root/Downloads/BOINC/slots/3/shared/input.tar.gz
copy /root/Downloads/BOINC/slots/3/shared/start_atlas.sh
copy /root/Downloads/BOINC/slots/3/shared/ATLAS.root_0
copy /root/Downloads/BOINC/slots/3/shared/RTE.tar.gz
start atlas job with 
cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err
07:56:46 (3353): wrapper (7.7.26015): starting
07:56:46 (3353): wrapper: running run_atlas (--nthreads 1)
singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img
sys.argv = ['run_atlas', '--nthreads', '1']
THREADS=1
This is not an Event Service job
This is trying to run the run_atlas wrapper for the 2nd time,but it is not an Event Service job,so will restart the job
output.list does not exist...
Checking for CVMFS
CVMFS is installed
OS:Scientific Linux release 6.10 (Carbon)

This is SLC or CentOS release 6, run the atlas job without Singularity
copy /root/Downloads/BOINC/slots/3/shared/input.tar.gz
copy /root/Downloads/BOINC/slots/3/shared/start_atlas.sh
copy /root/Downloads/BOINC/slots/3/shared/ATLAS.root_0
copy /root/Downloads/BOINC/slots/3/shared/RTE.tar.gz
start atlas job with 
cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err
running cmd return value is 0

*****************The last 100 lines of the pilot log******************
2019-06-28 05:58:19,322 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | update_server             | will not send fileinfo
2019-06-28 05:58:19,322 | INFO     | queue_monitor       | pilot.control.job                | send_state                | pilot will not update the server (heartbeat message will be written to file)
2019-06-28 05:58:19,322 | INFO     | queue_monitor       | pilot.control.job                | send_state                | job 4381875134 has failed - writing final server update
2019-06-28 05:58:19,323 | WARNING  | queue_monitor       | pilot.control.job                | send_state                | making sure that job.state is set to failed since a pilot error code is set
2019-06-28 05:58:19,376 | WARNING  | queue_monitor       | pilot.info.jobdata               | get_max_workdir_size      | found no stored workdir sizes
2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | will not add max space = 0 B to job metrics
2019-06-28 05:58:19,377 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | job metrics="coreCount=1"
2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.control.job                | get_data_structure        | payload/TRF did not report the number of read events
2019-06-28 05:58:19,378 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist
2019-06-28 05:58:19,379 | WARNING  | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt
2019-06-28 05:58:19,379 | DEBUG    | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | summary_dictionary={}
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | memory summary dictionary not yet available
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . Timing measurements:
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . get job = 6 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . initial setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload setup = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . total setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-in = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload execution = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-out = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,388 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_log_extracts          | building log extracts (sent to the server as 'pilotLog')
2019-06-28 05:58:19,389 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_panda_tracer_log      | PanDA tracer log does not exist: pandatracerlog.txt (ignoring)
2019-06-28 05:58:19,390 | INFO     | queue_monitor       | pilot.util.container             | execute                   | executing command: tail -n 20 pilotlog.txt
2019-06-28 05:58:19,595 | WARNING  | queue_monitor       | pilot.util.auxiliary             | get_log_extracts          | detected the following tail of warning/fatal messages in the pilot log:
- Log from pilotlog.txt -2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | will not add max space = 0 B to job metrics
2019-06-28 05:58:19,377 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | job metrics="coreCount=1"
2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.control.job                | get_data_structure        | payload/TRF did not report the number of read events
2019-06-28 05:58:19,378 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist
2019-06-28 05:58:19,379 | WARNING  | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt
2019-06-28 05:58:19,379 | DEBUG    | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | summary_dictionary={}
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | memory summary dictionary not yet available
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . Timing measurements:
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . get job = 6 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . initial setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload setup = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . total setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-in = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload execution = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-out = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,388 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_log_extracts          | building log extracts (sent to the server as 'pilotLog')
2019-06-28 05:58:19,389 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_panda_tracer_log      | PanDA tracer log does not exist: pandatracerlog.txt (ignoring)
2019-06-28 05:58:19,390 | INFO     | queue_monitor       | pilot.util.container             | execute                   | executing command: tail -n 20 pilotlog.txt
2019-06-28 05:58:19,596 | WARNING  | queue_monitor       | pilot.control.job                | add_timing_and_extracts   | pilot log extracts:
- Log from pilotlog.txt -2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | will not add max space = 0 B to job metrics
2019-06-28 05:58:19,377 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_job_metrics           | job metrics="coreCount=1"
2019-06-28 05:58:19,377 | INFO     | queue_monitor       | pilot.control.job                | get_data_structure        | payload/TRF did not report the number of read events
2019-06-28 05:58:19,378 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist
2019-06-28 05:58:19,379 | WARNING  | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt
2019-06-28 05:58:19,379 | DEBUG    | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | summary_dictionary={}
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | memory summary dictionary not yet available
2019-06-28 05:58:19,379 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . Timing measurements:
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . get job = 6 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . initial setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload setup = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . total setup = 16 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-in = 0 s
2019-06-28 05:58:19,380 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . payload execution = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | . stage-out = 0 s
2019-06-28 05:58:19,381 | INFO     | queue_monitor       | pilot.util.auxiliary             | timing_report             | ..............................
2019-06-28 05:58:19,388 | INFO     | queue_monitor       | pilot.util.auxiliary             | get_log_extracts          | building log extracts (sent to the server as 'pilotLog')
2019-06-28 05:58:19,389 | DEBUG    | queue_monitor       | pilot.util.auxiliary             | get_panda_tracer_log      | PanDA tracer log does not exist: pandatracerlog.txt (ignoring)
2019-06-28 05:58:19,390 | INFO     | queue_monitor       | pilot.util.container             | execute                   | executing command: tail -n 20 pilotlog.txt
2019-06-28 05:58:19,597 | WARNING  | queue_monitor       | pilot.control.job                | add_error_codes           | pilotErrorCodes = [1199] (will report primary/first error code)
2019-06-28 05:58:19,598 | WARNING  | queue_monitor       | pilot.control.job                | add_error_codes           | pilotErrorDiags = ['Failed to create local directory'] (will report primary/first error diag)
2019-06-28 05:58:19,613 | DEBUG    | queue_monitor       | pilot.control.job                | send_state                | wrote heartbeat to file /root/Downloads/BOINC/slots/3/heartbeat.json
2019-06-28 05:58:20,627 | WARNING  | queue_monitor       | pilot.control.job                | queue_monitor             | failed to dequeue job: queue is empty (did job fail before job monitor started?)
2019-06-28 05:58:20,628 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | 
2019-06-28 05:58:20,628 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | job summary report
2019-06-28 05:58:20,628 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | --------------------------------------------------
2019-06-28 05:58:20,628 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | PanDA job id: 4381875134
2019-06-28 05:58:20,629 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | task id: 18251691
2019-06-28 05:58:20,629 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | error 1/1: 1199: Failed to create local directory
2019-06-28 05:58:20,629 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | status: LOG_TRANSFER = IN_PROGRESS 
2019-06-28 05:58:20,629 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | pilot state: failed 
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | transexitcode: 0
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | exeerrorcode: 0
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | exeerrordiag: 
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | exitcode: 0
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | exitmsg: 
2019-06-28 05:58:20,630 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | cpuconsumptiontime: -1 
2019-06-28 05:58:20,631 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | nevents: 0
2019-06-28 05:58:20,631 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | neventsw: 0
2019-06-28 05:58:20,631 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | pid: None
2019-06-28 05:58:20,631 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | pgrp: None
2019-06-28 05:58:20,632 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | corecount: 1
2019-06-28 05:58:20,632 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | event service: False
2019-06-28 05:58:20,632 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | --------------------------------------------------
2019-06-28 05:58:20,632 | INFO     | queue_monitor       | pilot.util.auxiliary             | make_job_report           | 
2019-06-28 05:58:20,632 | WARNING  | queue_monitor       | pilot.util.common                | should_abort              | job:queue_monitor:received graceful stop - abort after this iteration
2019-06-28 05:58:20,633 | WARNING  | queue_monitor       | pilot.control.job                | pause_queue_monitor       | since job:queue_monitor is responsible for sending job updates, we sleep for 20 s
2019-06-28 05:58:42,811 | INFO     | queue_monitor       | pilot.control.job                | queue_monitor             | [job] queue monitor has finished
2019-06-28 05:58:42,812 | INFO     | MainThread          | pilot.workflow.generic           | run                       | end of generic workflow (traces error code: 1199)
2019-06-28 05:58:42,812 | INFO     | MainThread          | root                             | wrap_up                   | traces error code: 1199
2019-06-28 05:58:42,812 | INFO     | MainThread          | root                             | wrap_up                   | an exit code was already set: 1199 (will be converted to a standard shell code)
2019-06-28 05:58:42,813 | INFO     | MainThread          | root                             | wrap_up                   | pilot has finished
***************diag file************
runtimeenvironments=APPS/HEP/ATLAS-SITE;
Processors=1
runtimeenvironments=APPS/HEP/ATLAS-SITE;
Processors=1
WallTime=109.88s
KernelTime=8.93s
UserTime=67.76s
CPUUsage=69%
MaxResidentMemory=104884kB
AverageResidentMemory=0kB
AverageTotalMemory=0kB
AverageUnsharedMemory=0kB
AverageUnsharedStack=0kB
AverageSharedMemory=0kB
PageSize=4096B
MajorPageFaults=198
MinorPageFaults=871154
Swaps=0
ForcedSwitches=1611
WaitSwitches=105769
Inputs=97272
Outputs=14048
SocketReceived=0
SocketSent=0
Signals=0

nodename=maeax@APU8S
exitcode=0
******************************WorkDir***********************
insgesamt 13172
drwxrwx--x. 8 root root     4096 28. Jun 07:58 .
drwxr-x--x. 3 root root     4096 28. Jun 07:56 ..
-rw-------. 1 root root      528 28. Jun 07:58 9lWLDma1NxunShfckohDCDFpABFKDmABFKDmieTTDmABFKDmBC9LVo.diag
-rw-------. 1 root root  7345006 26. Jun 20:18 agis_ddmendpoints.json
-rw-------. 1 root root  4708756 28. Jun 07:57 agis_schedconf.cvmfs.json
drwx------. 2 root root     4096 28. Jun 07:56 .alrb
drwxr-xr-x. 3 root root     4096 26. Jun 20:18 APPS
drwxr-xr-x. 2 root root     4096 26. Jun 20:18 .arc
-rw-------. 1 root root      549 26. Jun 20:18 .asetup
-rw-------. 1 root root     4198 26. Jun 20:19 .asetup.save
-rw-r--r--. 1 root root        0 26. Jun 20:18 boinc_lockfile
-rw-r--r--. 1 root root     8192 28. Jun 07:58 boinc_mmap_file
-rw-r--r--. 1 root root      537 28. Jun 07:58 boinc_task_state.xml
-rw-------. 1 root root     1658 28. Jun 07:58 heartbeat.json
-rw-r--r--. 1 root root     5475 28. Jun 07:56 init_data.xml
-rw-r--r--. 1 root root   250195 28. Jun 07:56 input.tar.gz
-rw-r--r--. 1 root root      112 26. Jun 20:18 job.xml
-rw-------. 1 root root    81302 28. Jun 07:58 log.18251691._053921.job.log.1
-rw-------. 1 4871  1028    2887 18. Jun 09:47 pandaJobData.out
drwxrwx---. 2 root root     4096 28. Jun 07:57 PanDA_Pilot-4381875134
drwxr-xr-x. 3  501 games    4096 11. Jun 00:38 pilot2
-rw-r--r--. 1 root root   241086 18. Jun 09:47 pilot2.tar.gz
-rw-------. 1 root root   671454 28. Jun 07:58 pilotlog.txt
-rw-r--r--. 1 root root     4468 18. Jun 09:45 queuedata.json
-rw-r--r--. 1 root root      786 28. Jun 07:56 RTE.tar.gz
-rwxr-xr-x. 1 root root     8512 26. Jun 20:18 run_atlas
-rwx------. 1 4871  1028   15232 18. Jun 09:47 runpilot2-wrapper.sh
-rw-r--r--. 1 root root      643 28. Jun 07:58 runtime_log
-rw-r--r--. 1 root root     6654 28. Jun 07:58 runtime_log.err
drwxrwx--x. 2 root root     4096 28. Jun 07:58 shared
-rw-r--r--. 1 root root     8714 28. Jun 07:56 start_atlas.sh
-rw-r--r--. 1 root root    18209 28. Jun 07:58 stderr.txt
-rw-r--r--. 1 root root      107 26. Jun 20:18 wrapper_26015_x86_64-pc-linux-gnu
-rw-r--r--. 1 root root       28 28. Jun 07:58 wrapper_checkpoint.txt
running start_atlas return value is 0
Parent exit 0
child process exit 0
07:58:43 (3353): run_atlas exited; CPU time 68.128642
07:58:43 (3353): called boinc_finish(0)

</stderr_txt>
]]>


©2024 CERN