| Name | dcpLDms2A2vnShfckohDCDFpABFKDmABFKDmECANDmABFKDm5OF5Un_0 |
| Workunit | 1962874 |
| Created | 19 Dec 2019, 13:29:29 UTC |
| Sent | 26 Dec 2019, 9:25:06 UTC |
| Report deadline | 2 Jan 2020, 9:25:06 UTC |
| Received | 26 Dec 2019, 9:53:15 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 4064 |
| Run time | 25 min 32 sec |
| CPU time | 8 min 38 sec |
| Validate state | Valid |
| Credit | 8.89 |
| Device peak FLOPS | 2.51 GFLOPS |
| Application version | ATLAS Simulation v0.98 (native_mt) x86_64-pc-linux-gnu |
| Peak working set size | 153.87 MB |
| Peak swap size | 1.42 GB |
| Peak disk usage | 710.28 MB |
Stderr output
<core_client_version>7.16.1</core_client_version>
<![CDATA[
<stderr_txt>
10:26:30 (802): wrapper (7.7.26015): starting
10:26:30 (802): wrapper: running run_atlas (--nthreads 2)
Do 26. Dez 10:26:30 CET 2019: Arguments: --nthreads 2
Do 26. Dez 10:26:30 CET 2019: Threads: 2
Do 26. Dez 10:26:30 CET 2019: Checking for CVMFS
Do 26. Dez 10:26:32 CET 2019: Probing /cvmfs/atlas.cern.ch... OK
Do 26. Dez 10:26:34 CET 2019: Probing /cvmfs/atlas-condb.cern.ch... OK
Do 26. Dez 10:26:35 CET 2019: Probing /cvmfs/grid.cern.ch... OK
Do 26. Dez 10:26:35 CET 2019: Probing /cvmfs/cernvm-prod.cern.ch... OK
Do 26. Dez 10:26:36 CET 2019: Probing /cvmfs/sft.cern.ch... OK
Do 26. Dez 10:26:37 CET 2019: Probing /cvmfs/alice.cern.ch... OK
Do 26. Dez 10:26:38 CET 2019: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
Do 26. Dez 10:26:38 CET 2019: 2.7.0.0 952 0 28568 58427 3 1 2425693 4194304 0 65024 0 0 n/a 0 0 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch DIRECT 1
Do 26. Dez 10:26:38 CET 2019: CVMFS is ok
Do 26. Dez 10:26:38 CET 2019: Singularity not required
Do 26. Dez 10:26:39 CET 2019: Set ATHENA_PROC_NUMBER=2
Do 26. Dez 10:26:40 CET 2019: Starting ATLAS job with PandaID=4002876565
Do 26. Dez 10:26:40 CET 2019: Running command: sh start_atlas.sh
Do 26. Dez 10:52:01 CET 2019: *** The last 200 lines of the pilot log: ***
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,672 | DEBUG | queue_monitor | pilot.control.job.4002876565 | get_data_structure | building data structure to be sent to server with heartbeat
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,673 | WARNING | queue_monitor | pilot.user.atlas.common | get_db_info | format EVNTtoHITS has no such key: dbData
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,673 | WARNING | queue_monitor | pilot.user.atlas.common | get_db_info | format EVNTtoHITS has no such key: dbTime
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,673 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | get_job_metrics | will not add max space = -365193805 B to job metrics
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,674 | DEBUG | queue_monitor | pilot.api.analytics | get_fitted_data | removing tails from data to be fitted
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,674 | INFO | queue_monitor | pilot.api.analytics | get_fitted_data | fitting pss+swap vs Time
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.api.analytics | get_fitted_data | current memory leak: 61.49 B/s (using 9 data points, chi2=10364)
10:52:01 (802): run_atlas exited; CPU time 234.746424
10:52:01 (802): called boinc_finish(0)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_job_metrics | job metrics="coreCount=2 actualCoreCount=3 leak=61.49 chi2=10364"
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.control.job.4002876565 | get_data_structure | payload/TRF did not report the number of read events
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,679 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/memory_monitor_summary.json (trf name=prmon)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 11451, 'nprocs': 12, 'nthreads': 0, 'rx_bytes': 3095021, 'wtime': 1104, 'rss': 258464, 'write_bytes': 0, 'vmem': 1444636, 'read_bytes': 0, 'stime': 417, 'tx_bytes': 2837772, 'pss': 245974, 'wchar': 0, 'rchar': 0, 'tx_packets': 7460, 'swap': 0, 'utime': 67}, 'Avg': {'write_bytes': 0, 'nprocs': 6, 'nthreads': 0, 'rx_bytes': 2803, 'rx_packets': 10, 'vmem': 1008806, 'read_bytes': 0, 'swap': 0, 'tx_bytes': 2570, 'pss': 164890, 'wchar': 0, 'rchar': 0, 'tx_packets': 6, 'rss': 172987}}
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . Timing measurements:
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . get job = 1 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . initial setup = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload setup = 0 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . total setup = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-in = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload execution = 1271 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-out = 3 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_panda_tracer_log | PanDA tracer log does not exist: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pandatracerlog.txt (ignoring)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,693 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pilotlog.txt
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:22,029 | WARNING | queue_monitor | pilot.util.auxiliary.4002876565 | get_log_extracts | detected the following tail of warning/fatal messages in the pilot log:
Do 26. Dez 10:52:01 CET 2019: - Log from pilotlog.txt -
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.api.analytics | get_fitted_data | current memory leak: 61.49 B/s (using 9 data points, chi2=10364)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_job_metrics | job metrics="coreCount=2 actualCoreCount=3 leak=61.49 chi2=10364"
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.control.job.4002876565 | get_data_structure | payload/TRF did not report the number of read events
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,679 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/memory_monitor_summary.json (trf name=prmon)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 11451, 'nprocs': 12, 'nthreads': 0, 'rx_bytes': 3095021, 'wtime': 1104, 'rss': 258464, 'write_bytes': 0, 'vmem': 1444636, 'read_bytes': 0, 'stime': 417, 'tx_bytes': 2837772, 'pss': 245974, 'wchar': 0, 'rchar': 0, 'tx_packets': 7460, 'swap': 0, 'utime': 67}, 'Avg': {'write_bytes': 0, 'nprocs': 6, 'nthreads': 0, 'rx_bytes': 2803, 'rx_packets': 10, 'vmem': 1008806, 'read_bytes': 0, 'swap': 0, 'tx_bytes': 2570, 'pss': 164890, 'wchar': 0, 'rchar': 0, 'tx_packets': 6, 'rss': 172987}}
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . Timing measurements:
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . get job = 1 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . initial setup = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload setup = 0 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . total setup = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-in = 2 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload execution = 1271 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-out = 3 s
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,683 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_panda_tracer_log | PanDA tracer log does not exist: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pandatracerlog.txt (ignoring)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,693 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pilotlog.txt
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:22,030 | WARNING | queue_monitor | pilot.control.job | add_timing_and_extracts |
Do 26. Dez 10:52:01 CET 2019: XXXXXXXXXXXXXXXXXXXXX[begin log extracts]
Do 26. Dez 10:52:01 CET 2019: - Log from pilotlog.txt -
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.api.analytics | get_fitted_data | current memory leak: 61.49 B/s (using 9 data points, chi2=10364)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_job_metrics | job metrics="coreCount=2 actualCoreCount=3 leak=61.49 chi2=10364"
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,675 | INFO | queue_monitor | pilot.control.job.4002876565 | get_data_structure | payload/TRF did not report the number of read events
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,679 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/memory_monitor_summary.json (trf name=prmon)
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 11451, 'nprocs': 12, 'nthreads': 0, 'rx_bytes': 3095021, 'wtime': 1104, 'rss': 258464, 'write_bytes': 0, 'vmem': 1444636, 'read_bytes': 0, 'stime': 417, 'tx_bytes': 2837772, 'pss': 245974, 'wchar': 0, 'rchar': 0, 'tx_packets': 7460, 'swap': 0, 'utime': 67}, 'Avg': {'write_bytes': 0, 'nprocs': 6, 'nthreads': 0, 'rx_bytes': 2803, 'rx_packets': 10, 'vmem': 1008806, 'read_bytes': 0, 'swap': 0, 'tx_bytes': 2570, 'pss': 164890, 'wchar': 0, 'rchar': 0, 'tx_packets': 6, 'rss': 172987}}
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
Do 26. Dez 10:52:01 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,681 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . Timing measurements:
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . get job = 1 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . initial setup = 2 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload setup = 0 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . total setup = 2 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-in = 2 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . payload execution = 1271 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,682 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | . stage-out = 3 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | timing_report | ..............................
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,683 | INFO | queue_monitor | pilot.util.auxiliary.4002876565 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,683 | DEBUG | queue_monitor | pilot.util.auxiliary.4002876565 | get_panda_tracer_log | PanDA tracer log does not exist: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pandatracerlog.txt (ignoring)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:21,693 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/pilotlog.txt
Do 26. Dez 10:52:02 CET 2019: XXXXXXXXXXXXXXXXXXXXX[end log extracts]
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,042 | DEBUG | queue_monitor | pilot.control.job.4002876565 | send_state | wrote heartbeat to file /var/lib/boinc/slots/0/heartbeat.json
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,042 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | job 4002876565 was dequeued from the monitored payloads queue
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,042 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | tmp job object deleted
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,042 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | [job] queue monitor thread has finished
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,074 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report |
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,074 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | job summary report
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,074 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | --------------------------------------------------
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,074 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | PanDA job id: 4002876565
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | task id: 000649-2078388-18750
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | errors: (none)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | status: LOG_TRANSFER = DONE
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | pilot state: failed
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | transexitcode: 65
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,075 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | exeerrorcode: 65
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | exeerrordiag: EVNTtoHITS got a SIGBUS signal (exit code 135)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | exitcode: 65
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | exitmsg: EVNTtoHITS got a SIGBUS signal (exit code 135)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | cpuconsumptiontime: 644 s
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | nevents: 0
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,076 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | neventsw: 0
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,077 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | pid: 7671
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,077 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | pgrp: 7671
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,077 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | corecount: 2
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,077 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | event service: False
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,078 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report | --------------------------------------------------
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,080 | INFO | retrieve | pilot.util.auxiliary.4002876565 | make_job_report |
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,080 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue jobs has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,080 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue payloads has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,080 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue data_in has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,081 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue data_out has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,081 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue current_data_in has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,081 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue validated_jobs has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue validated_payloads has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue monitored_payloads has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue finished_jobs has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue finished_payloads has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue finished_data_in has 1 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue finished_data_out has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,082 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue failed_jobs has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,083 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue failed_payloads has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,083 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue failed_data_in has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,083 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue failed_data_out has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,083 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue completed_jobs has 0 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,083 | INFO | retrieve | pilot.util.queuehandling | queue_report | queue completed_jobids has 1 job(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,084 | INFO | retrieve | pilot.control.job.4002876565 | has_job_completed | job 4002876565 has completed (purged errors)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,084 | INFO | retrieve | pilot.util.processes | cleanup | overall cleanup function is called
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:22,086 | DEBUG | retrieve | pilot.util.processes | cleanup | work directory was removed: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:23,092 | INFO | retrieve | pilot.info.jobdata | collect_zombies | --- collectZombieJob: --- 10, [7671]
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:23,093 | INFO | retrieve | pilot.info.jobdata | collect_zombies | zombie collector trying to kill pid 7671
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:23,093 | INFO | retrieve | pilot.info.jobdata | collect_zombies | harmless exception when collecting zombies: [Errno 10] No child processes
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:23,404 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 16 threads
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:23,405 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140466973611840)>, <ExcThread(job, started 140466706171648)>, <ExcThread(validate, started 140466697778944)>, <ExcThread(failed_post, started 140466116859648)>, <ExcThread(create_data_payload, started 140466158823168)>, <ExcThread(validate_pre, started 140466647422720)>, <ExcThread(monitor, started 140466167215872)>, <ExcThread(copytool_in, started 140466655815424)>, <ExcThread(data, started 140466689386240)>, <ExcThread(payload, started 140466664208128)>, <ExcThread(retrieve, started 140466150430464)>, <ExcThread(execute_payloads, started 140465630344960)>, <ExcThread(job_monitor, started 140466142037760)>, <ExcThread(copytool_out, started 140466680993536)>, <ExcThread(validate_post, started 140466133645056)>, <ExcThread(queue_monitoring, started 140466672600832)>]
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,099 | INFO | retrieve | pilot.util.processes | cleanup | collected zombie processes
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,099 | INFO | retrieve | pilot.util.processes | cleanup | will now attempt to kill all subprocesses of pid=7671
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,267 | INFO | retrieve | pilot.util.processes | kill_processes | process IDs to be killed: [7671] (in reverse order)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,419 | WARNING | retrieve | pilot.util.processes | kill_processes | found no corresponding commands to process id(s)
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,419 | INFO | retrieve | pilot.util.processes | kill_orphans | Do not look for orphan processes in BOINC jobs
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,419 | INFO | retrieve | pilot.control.job | retrieve | ready for new job
Do 26. Dez 10:52:02 CET 2019: 2019-12-26 09:51:24,420 | INFO | retrieve | root | retrieve | pilot has finished for previous job - re-establishing logging
Do 26. Dez 10:52:02 CET 2019: mpi4py not found
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,426 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,427 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | *** PanDA Pilot version 2.3.4 (12) ***
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,427 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,427 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner |
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,427 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | pilot is running in a VM
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,428 | INFO | retrieve | pilot.util.auxiliary | display_architecture_info | architecture information:
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,510 | INFO | retrieve | pilot.util.auxiliary | display_architecture_info |
Do 26. Dez 10:52:03 CET 2019:
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:24,511 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,013 | DEBUG | retrieve | pilot.util.monitoring | check_local_space | checking local space on /var/lib/boinc/slots/0
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,053 | INFO | retrieve | pilot.util.monitoring | check_local_space | sufficient remaining disk space (12491685888 B)
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,053 | WARNING | retrieve | pilot.control.job | proceed_with_getjob | since timefloor is set to 0, pilot was only allowed to run one job
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,053 | DEBUG | retrieve | pilot.control.job | retrieve | [job] retrieve thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,215 | INFO | validate_pre | pilot.control.payload | validate_pre | [payload] validate_pre thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,224 | INFO | monitor | pilot.control.monitor | control | [monitor] control thread has ended
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,295 | INFO | execute_payloads | pilot.control.payload | execute_payloads | [payload] execute_payloads thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,330 | DEBUG | create_data_payload | pilot.control.job | create_data_payload | [job] create_data_payload thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,354 | DEBUG | data | pilot.control.data | control | data control ending since graceful_stop has been set
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,355 | DEBUG | data | pilot.control.data | control | [data] control thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,385 | DEBUG | copytool_in | pilot.control.data | copytool_in | [data] copytool_in thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,459 | INFO | validate_post | pilot.control.payload | validate_post | [payload] validate_post thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,480 | WARNING | queue_monitoring | pilot.util.common | should_abort | data:queue_monitoring:received graceful stop - abort after this iteration
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,558 | DEBUG | job | pilot.control.job | control | job control ending since graceful_stop has been set
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,558 | DEBUG | job | pilot.control.job | control | [job] control thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,816 | DEBUG | payload | pilot.control.payload | control | payload control ending since graceful_stop has been set
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,816 | DEBUG | payload | pilot.control.payload | control | [payload] control thread has finished
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,824 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 6 threads
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,824 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140466973611840)>, <ExcThread(validate, started 140466697778944)>, <ExcThread(failed_post, started 140466116859648)>, <ExcThread(job_monitor, started 140466142037760)>, <ExcThread(copytool_out, started 140466680993536)>, <ExcThread(queue_monitoring, started 140466672600832)>]
Do 26. Dez 10:52:03 CET 2019: 2019-12-26 09:51:25,928 | INFO | failed_post | pilot.control.payload | failed_post | [payload] failed_post thread has finished
</stderr_txt>
]]>
©2024 CERN