Name | 9lWLDma1NxunShfckohDCDFpABFKDmABFKDmieTTDmABFKDmBC9LVo_1 |
Workunit | 1905419 |
Created | 19 Jun 2019, 16:01:41 UTC |
Sent | 26 Jun 2019, 17:22:58 UTC |
Report deadline | 3 Jul 2019, 17:22:58 UTC |
Received | 28 Jun 2019, 5:59:53 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 2244 |
Run time | 3 hours 48 min 55 sec |
CPU time | 3 hours 45 min 14 sec |
Validate state | Valid |
Credit | 66.57 |
Device peak FLOPS | 2.09 GFLOPS |
Application version | ATLAS Simulation v0.62 (native_mt) x86_64-pc-linux-gnu |
Peak working set size | 1.91 GB |
Peak swap size | 2.58 GB |
Peak disk usage | 769.43 MB |
<core_client_version>7.5.1</core_client_version> <![CDATA[ <stderr_txt> 20:18:08 (22602): wrapper (7.7.26015): starting 20:18:08 (22602): wrapper: running run_atlas (--nthreads 1) singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img sys.argv = ['run_atlas', '--nthreads', '1'] THREADS=1 Checking for CVMFS CVMFS is installed OS:Scientific Linux release 6.10 (Carbon) This is SLC or CentOS release 6, run the atlas job without Singularity copy /root/Downloads/BOINC/slots/3/shared/input.tar.gz copy /root/Downloads/BOINC/slots/3/shared/start_atlas.sh copy /root/Downloads/BOINC/slots/3/shared/ATLAS.root_0 copy /root/Downloads/BOINC/slots/3/shared/RTE.tar.gz start atlas job with cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err 07:56:46 (3353): wrapper (7.7.26015): starting 07:56:46 (3353): wrapper: running run_atlas (--nthreads 1) singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-slc6.img sys.argv = ['run_atlas', '--nthreads', '1'] THREADS=1 This is not an Event Service job This is trying to run the run_atlas wrapper for the 2nd time,but it is not an Event Service job,so will restart the job output.list does not exist... Checking for CVMFS CVMFS is installed OS:Scientific Linux release 6.10 (Carbon) This is SLC or CentOS release 6, run the atlas job without Singularity copy /root/Downloads/BOINC/slots/3/shared/input.tar.gz copy /root/Downloads/BOINC/slots/3/shared/start_atlas.sh copy /root/Downloads/BOINC/slots/3/shared/ATLAS.root_0 copy /root/Downloads/BOINC/slots/3/shared/RTE.tar.gz start atlas job with cmd = sh start_atlas.sh > runtime_log 2> runtime_log.err running cmd return value is 0 *****************The last 100 lines of the pilot log****************** 2019-06-28 05:58:19,322 | DEBUG | queue_monitor | pilot.util.auxiliary | update_server | will not send fileinfo 2019-06-28 05:58:19,322 | INFO | queue_monitor | pilot.control.job | send_state | pilot will not update the server (heartbeat message will be written to file) 2019-06-28 05:58:19,322 | INFO | queue_monitor | pilot.control.job | send_state | job 4381875134 has failed - writing final server update 2019-06-28 05:58:19,323 | WARNING | queue_monitor | pilot.control.job | send_state | making sure that job.state is set to failed since a pilot error code is set 2019-06-28 05:58:19,376 | WARNING | queue_monitor | pilot.info.jobdata | get_max_workdir_size | found no stored workdir sizes 2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.util.auxiliary | get_job_metrics | will not add max space = 0 B to job metrics 2019-06-28 05:58:19,377 | DEBUG | queue_monitor | pilot.util.auxiliary | get_job_metrics | job metrics="coreCount=1" 2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.control.job | get_data_structure | payload/TRF did not report the number of read events 2019-06-28 05:58:19,378 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist 2019-06-28 05:58:19,379 | WARNING | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt 2019-06-28 05:58:19,379 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={} 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | memory summary dictionary not yet available 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . Timing measurements: 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . get job = 6 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . initial setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload setup = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . total setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-in = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload execution = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-out = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,388 | INFO | queue_monitor | pilot.util.auxiliary | get_log_extracts | building log extracts (sent to the server as 'pilotLog') 2019-06-28 05:58:19,389 | DEBUG | queue_monitor | pilot.util.auxiliary | get_panda_tracer_log | PanDA tracer log does not exist: pandatracerlog.txt (ignoring) 2019-06-28 05:58:19,390 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 pilotlog.txt 2019-06-28 05:58:19,595 | WARNING | queue_monitor | pilot.util.auxiliary | get_log_extracts | detected the following tail of warning/fatal messages in the pilot log: - Log from pilotlog.txt -2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.util.auxiliary | get_job_metrics | will not add max space = 0 B to job metrics 2019-06-28 05:58:19,377 | DEBUG | queue_monitor | pilot.util.auxiliary | get_job_metrics | job metrics="coreCount=1" 2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.control.job | get_data_structure | payload/TRF did not report the number of read events 2019-06-28 05:58:19,378 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist 2019-06-28 05:58:19,379 | WARNING | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt 2019-06-28 05:58:19,379 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={} 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | memory summary dictionary not yet available 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . Timing measurements: 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . get job = 6 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . initial setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload setup = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . total setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-in = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload execution = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-out = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,388 | INFO | queue_monitor | pilot.util.auxiliary | get_log_extracts | building log extracts (sent to the server as 'pilotLog') 2019-06-28 05:58:19,389 | DEBUG | queue_monitor | pilot.util.auxiliary | get_panda_tracer_log | PanDA tracer log does not exist: pandatracerlog.txt (ignoring) 2019-06-28 05:58:19,390 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 pilotlog.txt 2019-06-28 05:58:19,596 | WARNING | queue_monitor | pilot.control.job | add_timing_and_extracts | pilot log extracts: - Log from pilotlog.txt -2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.util.auxiliary | get_job_metrics | will not add max space = 0 B to job metrics 2019-06-28 05:58:19,377 | DEBUG | queue_monitor | pilot.util.auxiliary | get_job_metrics | job metrics="coreCount=1" 2019-06-28 05:58:19,377 | INFO | queue_monitor | pilot.control.job | get_data_structure | payload/TRF did not report the number of read events 2019-06-28 05:58:19,378 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | neither memory_monitor_summary.json, nor /root/Downloads/BOINC/slots/3/memory_monitor_summary.json exist 2019-06-28 05:58:19,379 | WARNING | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info_path | file does not exist either: memory_monitor_output.txt 2019-06-28 05:58:19,379 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={} 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | memory summary dictionary not yet available 2019-06-28 05:58:19,379 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . Timing measurements: 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . get job = 6 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . initial setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload setup = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . total setup = 16 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-in = 0 s 2019-06-28 05:58:19,380 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . payload execution = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | . stage-out = 0 s 2019-06-28 05:58:19,381 | INFO | queue_monitor | pilot.util.auxiliary | timing_report | .............................. 2019-06-28 05:58:19,388 | INFO | queue_monitor | pilot.util.auxiliary | get_log_extracts | building log extracts (sent to the server as 'pilotLog') 2019-06-28 05:58:19,389 | DEBUG | queue_monitor | pilot.util.auxiliary | get_panda_tracer_log | PanDA tracer log does not exist: pandatracerlog.txt (ignoring) 2019-06-28 05:58:19,390 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 pilotlog.txt 2019-06-28 05:58:19,597 | WARNING | queue_monitor | pilot.control.job | add_error_codes | pilotErrorCodes = [1199] (will report primary/first error code) 2019-06-28 05:58:19,598 | WARNING | queue_monitor | pilot.control.job | add_error_codes | pilotErrorDiags = ['Failed to create local directory'] (will report primary/first error diag) 2019-06-28 05:58:19,613 | DEBUG | queue_monitor | pilot.control.job | send_state | wrote heartbeat to file /root/Downloads/BOINC/slots/3/heartbeat.json 2019-06-28 05:58:20,627 | WARNING | queue_monitor | pilot.control.job | queue_monitor | failed to dequeue job: queue is empty (did job fail before job monitor started?) 2019-06-28 05:58:20,628 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | 2019-06-28 05:58:20,628 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | job summary report 2019-06-28 05:58:20,628 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | -------------------------------------------------- 2019-06-28 05:58:20,628 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | PanDA job id: 4381875134 2019-06-28 05:58:20,629 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | task id: 18251691 2019-06-28 05:58:20,629 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | error 1/1: 1199: Failed to create local directory 2019-06-28 05:58:20,629 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | status: LOG_TRANSFER = IN_PROGRESS 2019-06-28 05:58:20,629 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | pilot state: failed 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | transexitcode: 0 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | exeerrorcode: 0 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | exeerrordiag: 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | exitcode: 0 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | exitmsg: 2019-06-28 05:58:20,630 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | cpuconsumptiontime: -1 2019-06-28 05:58:20,631 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | nevents: 0 2019-06-28 05:58:20,631 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | neventsw: 0 2019-06-28 05:58:20,631 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | pid: None 2019-06-28 05:58:20,631 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | pgrp: None 2019-06-28 05:58:20,632 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | corecount: 1 2019-06-28 05:58:20,632 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | event service: False 2019-06-28 05:58:20,632 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | -------------------------------------------------- 2019-06-28 05:58:20,632 | INFO | queue_monitor | pilot.util.auxiliary | make_job_report | 2019-06-28 05:58:20,632 | WARNING | queue_monitor | pilot.util.common | should_abort | job:queue_monitor:received graceful stop - abort after this iteration 2019-06-28 05:58:20,633 | WARNING | queue_monitor | pilot.control.job | pause_queue_monitor | since job:queue_monitor is responsible for sending job updates, we sleep for 20 s 2019-06-28 05:58:42,811 | INFO | queue_monitor | pilot.control.job | queue_monitor | [job] queue monitor has finished 2019-06-28 05:58:42,812 | INFO | MainThread | pilot.workflow.generic | run | end of generic workflow (traces error code: 1199) 2019-06-28 05:58:42,812 | INFO | MainThread | root | wrap_up | traces error code: 1199 2019-06-28 05:58:42,812 | INFO | MainThread | root | wrap_up | an exit code was already set: 1199 (will be converted to a standard shell code) 2019-06-28 05:58:42,813 | INFO | MainThread | root | wrap_up | pilot has finished ***************diag file************ runtimeenvironments=APPS/HEP/ATLAS-SITE; Processors=1 runtimeenvironments=APPS/HEP/ATLAS-SITE; Processors=1 WallTime=109.88s KernelTime=8.93s UserTime=67.76s CPUUsage=69% MaxResidentMemory=104884kB AverageResidentMemory=0kB AverageTotalMemory=0kB AverageUnsharedMemory=0kB AverageUnsharedStack=0kB AverageSharedMemory=0kB PageSize=4096B MajorPageFaults=198 MinorPageFaults=871154 Swaps=0 ForcedSwitches=1611 WaitSwitches=105769 Inputs=97272 Outputs=14048 SocketReceived=0 SocketSent=0 Signals=0 nodename=maeax@APU8S exitcode=0 ******************************WorkDir*********************** insgesamt 13172 drwxrwx--x. 8 root root 4096 28. Jun 07:58 . drwxr-x--x. 3 root root 4096 28. Jun 07:56 .. -rw-------. 1 root root 528 28. Jun 07:58 9lWLDma1NxunShfckohDCDFpABFKDmABFKDmieTTDmABFKDmBC9LVo.diag -rw-------. 1 root root 7345006 26. Jun 20:18 agis_ddmendpoints.json -rw-------. 1 root root 4708756 28. Jun 07:57 agis_schedconf.cvmfs.json drwx------. 2 root root 4096 28. Jun 07:56 .alrb drwxr-xr-x. 3 root root 4096 26. Jun 20:18 APPS drwxr-xr-x. 2 root root 4096 26. Jun 20:18 .arc -rw-------. 1 root root 549 26. Jun 20:18 .asetup -rw-------. 1 root root 4198 26. Jun 20:19 .asetup.save -rw-r--r--. 1 root root 0 26. Jun 20:18 boinc_lockfile -rw-r--r--. 1 root root 8192 28. Jun 07:58 boinc_mmap_file -rw-r--r--. 1 root root 537 28. Jun 07:58 boinc_task_state.xml -rw-------. 1 root root 1658 28. Jun 07:58 heartbeat.json -rw-r--r--. 1 root root 5475 28. Jun 07:56 init_data.xml -rw-r--r--. 1 root root 250195 28. Jun 07:56 input.tar.gz -rw-r--r--. 1 root root 112 26. Jun 20:18 job.xml -rw-------. 1 root root 81302 28. Jun 07:58 log.18251691._053921.job.log.1 -rw-------. 1 4871 1028 2887 18. Jun 09:47 pandaJobData.out drwxrwx---. 2 root root 4096 28. Jun 07:57 PanDA_Pilot-4381875134 drwxr-xr-x. 3 501 games 4096 11. Jun 00:38 pilot2 -rw-r--r--. 1 root root 241086 18. Jun 09:47 pilot2.tar.gz -rw-------. 1 root root 671454 28. Jun 07:58 pilotlog.txt -rw-r--r--. 1 root root 4468 18. Jun 09:45 queuedata.json -rw-r--r--. 1 root root 786 28. Jun 07:56 RTE.tar.gz -rwxr-xr-x. 1 root root 8512 26. Jun 20:18 run_atlas -rwx------. 1 4871 1028 15232 18. Jun 09:47 runpilot2-wrapper.sh -rw-r--r--. 1 root root 643 28. Jun 07:58 runtime_log -rw-r--r--. 1 root root 6654 28. Jun 07:58 runtime_log.err drwxrwx--x. 2 root root 4096 28. Jun 07:58 shared -rw-r--r--. 1 root root 8714 28. Jun 07:56 start_atlas.sh -rw-r--r--. 1 root root 18209 28. Jun 07:58 stderr.txt -rw-r--r--. 1 root root 107 26. Jun 20:18 wrapper_26015_x86_64-pc-linux-gnu -rw-r--r--. 1 root root 28 28. Jun 07:58 wrapper_checkpoint.txt running start_atlas return value is 0 Parent exit 0 child process exit 0 07:58:43 (3353): run_atlas exited; CPU time 68.128642 07:58:43 (3353): called boinc_finish(0) </stderr_txt> ]]>
©2024 CERN