Name bO5NDmluoDwnShfckohDCDFpABFKDmABFKDmx9YMDmABFKDmw4QQjm_0
Workunit 1973454
Created 20 Jan 2020, 16:17:15 UTC
Sent 20 Jan 2020, 21:27:48 UTC
Report deadline 27 Jan 2020, 21:27:48 UTC
Received 20 Jan 2020, 22:11:19 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 3848
Run time 41 min 26 sec
CPU time 52 min 25 sec
Validate state Valid
Credit 26.80
Device peak FLOPS 4.66 GFLOPS
Application version ATLAS Simulation v1.00 (native_mt)
x86_64-pc-linux-gnu
Peak working set size 1.80 GB
Peak swap size 2.56 GB
Peak disk usage 717.81 MB

Stderr output

<core_client_version>7.16.1</core_client_version>
<![CDATA[
<stderr_txt>
22:29:38 (23495): wrapper (7.7.26015): starting
22:29:38 (23495): wrapper: running run_atlas (--nthreads 2)
Mo 20. Jan 22:29:38 CET 2020: Arguments: --nthreads 2
Mo 20. Jan 22:29:38 CET 2020: Threads: 2
Mo 20. Jan 22:29:38 CET 2020: Checking for CVMFS
Mo 20. Jan 22:29:38 CET 2020: Probing /cvmfs/atlas.cern.ch... OK
Mo 20. Jan 22:29:38 CET 2020: Probing /cvmfs/atlas-condb.cern.ch... OK
Mo 20. Jan 22:29:39 CET 2020: Probing /cvmfs/grid.cern.ch... OK
Mo 20. Jan 22:29:39 CET 2020: Probing /cvmfs/cernvm-prod.cern.ch... OK
Mo 20. Jan 22:29:39 CET 2020: Probing /cvmfs/sft.cern.ch... OK
Mo 20. Jan 22:29:40 CET 2020: Probing /cvmfs/alice.cern.ch... OK
Mo 20. Jan 22:29:40 CET 2020: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
Mo 20. Jan 22:29:40 CET 2020: 2.7.0.0 3040 9297 82104 59406 0 61 1938776 4194305 589 65024 0 62771 91.7796 946423 1537 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch DIRECT 1
Mo 20. Jan 22:29:40 CET 2020: CVMFS is ok
Mo 20. Jan 22:29:40 CET 2020: Using singularity image /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img
Mo 20. Jan 22:29:40 CET 2020: Checking for singularity binary...
Mo 20. Jan 22:29:40 CET 2020: Using singularity found in PATH at /usr/bin/singularity
Mo 20. Jan 22:29:40 CET 2020: Running /usr/bin/singularity --version
Mo 20. Jan 22:29:40 CET 2020: singularity version 3.5.2-1.1.el7
Mo 20. Jan 22:29:40 CET 2020: Checking singularity works with /usr/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
Mo 20. Jan 22:29:41 CET 2020: ryzcos7
Mo 20. Jan 22:29:41 CET 2020: Singularity works
Mo 20. Jan 22:29:41 CET 2020: Set ATHENA_PROC_NUMBER=2
Mo 20. Jan 22:29:41 CET 2020: Starting ATLAS job with PandaID=4002876565
Mo 20. Jan 22:29:41 CET 2020: Running command: /usr/bin/singularity exec --pwd /var/lib/boinc/slots/0 -B /cvmfs,/var /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img sh start_atlas.sh
Mo 20. Jan 23:11:02 CET 2020:  *** The last 200 lines of the pilot log: ***
Mo 20. Jan 23:11:02 CET 2020:           "cpuTime": 3, 
Mo 20. Jan 23:11:02 CET 2020:           "wallTime": 1
Mo 20. Jan 23:11:02 CET 2020:         }, 
Mo 20. Jan 23:11:02 CET 2020:         "preExe": {
Mo 20. Jan 23:11:02 CET 2020:           "cpuTime": 8, 
Mo 20. Jan 23:11:02 CET 2020:           "wallTime": 35
Mo 20. Jan 23:11:02 CET 2020:         }, 
Mo 20. Jan 23:11:02 CET 2020:         "total": {
Mo 20. Jan 23:11:02 CET 2020:           "cpuTime": 167, 
Mo 20. Jan 23:11:02 CET 2020:           "wallTime": 448
Mo 20. Jan 23:11:02 CET 2020:         }, 
Mo 20. Jan 23:11:02 CET 2020:         "validation": {
Mo 20. Jan 23:11:02 CET 2020:           "cpuTime": 0, 
Mo 20. Jan 23:11:02 CET 2020:           "wallTime": 0
Mo 20. Jan 23:11:02 CET 2020:         }, 
Mo 20. Jan 23:11:02 CET 2020:         "wallTime": 412
Mo 20. Jan 23:11:02 CET 2020:       }
Mo 20. Jan 23:11:02 CET 2020:     }, 
Mo 20. Jan 23:11:02 CET 2020:     "machine": {
Mo 20. Jan 23:11:02 CET 2020:       "cpu_family": "23", 
Mo 20. Jan 23:11:02 CET 2020:       "linux_distribution": [
Mo 20. Jan 23:11:02 CET 2020:         "CentOS Linux", 
Mo 20. Jan 23:11:02 CET 2020:         "7.6.1810", 
Mo 20. Jan 23:11:02 CET 2020:         "Core"
Mo 20. Jan 23:11:02 CET 2020:       ], 
Mo 20. Jan 23:11:02 CET 2020:       "model": "8", 
Mo 20. Jan 23:11:02 CET 2020:       "model_name": "AMD Ryzen 7 2700 Eight-Core Processor", 
Mo 20. Jan 23:11:02 CET 2020:       "node": "ryzcos7", 
Mo 20. Jan 23:11:02 CET 2020:       "platform": "Linux-3.10.0-693.el7.x86_64-x86_64-with-centos-7.6.1810-Core"
Mo 20. Jan 23:11:02 CET 2020:     }, 
Mo 20. Jan 23:11:02 CET 2020:     "transform": {
Mo 20. Jan 23:11:02 CET 2020:       "cpuEfficiency": 0.6739, 
Mo 20. Jan 23:11:02 CET 2020:       "cpuPWEfficiency": 0.7102, 
Mo 20. Jan 23:11:02 CET 2020:       "cpuTime": 11, 
Mo 20. Jan 23:11:02 CET 2020:       "cpuTimeTotal": 3300, 
Mo 20. Jan 23:11:02 CET 2020:       "externalCpuTime": 29, 
Mo 20. Jan 23:11:02 CET 2020:       "processedEvents": 10, 
Mo 20. Jan 23:11:02 CET 2020:       "trfPredata": null, 
Mo 20. Jan 23:11:02 CET 2020:       "wallTime": 2312
Mo 20. Jan 23:11:02 CET 2020:     }
Mo 20. Jan 23:11:02 CET 2020:   }
Mo 20. Jan 23:11:02 CET 2020: }
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,182 | DEBUG    | queue_monitor       | pilot.util.auxiliary.4002876565  | update_server             | xml:will send fileinfo
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,182 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | state=finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,182 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | serverstate=running
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,182 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | serverstate=finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,183 | INFO     | queue_monitor       | pilot.control.job.4002876565     | send_state                | pilot will not update the server (heartbeat message will be written to file)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,183 | INFO     | queue_monitor       | pilot.control.job.4002876565     | send_state                | job 4002876565 has finished - writing final server update
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,183 | DEBUG    | queue_monitor       | pilot.control.job.4002876565     | get_data_structure        | building data structure to be sent to server with heartbeat
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,183 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | get_job_metrics           | will not add max space = -353094221 B to job metrics
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,184 | DEBUG    | queue_monitor       | pilot.api.analytics              | get_fitted_data           | removing tails from data to be fitted
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,184 | INFO     | queue_monitor       | pilot.api.analytics              | get_fitted_data           | fitting pss+swap vs Time
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,185 | INFO     | queue_monitor       | pilot.api.analytics              | get_fitted_data           | current memory leak: -836.10 B/s (using 32 data points, chi2=6973144)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,185 | DEBUG    | queue_monitor       | pilot.util.auxiliary.4002876565  | get_job_metrics           | job metrics="coreCount=2 actualCoreCount=4 nEvents=10 leak=-836.10 chi2=6973144"
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,185 | INFO     | queue_monitor       | pilot.control.job.4002876565     | get_data_structure        | total number of processed events: 10 (read)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,186 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_values         | using path: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565/memory_monitor_summary.json (trf name=prmon)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,187 | DEBUG    | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | summary_dictionary={'Max': {'rx_packets': 53544, 'nprocs': 7, 'nthreads': 1, 'rx_bytes': 45613999, 'wtime': 2340, 'rss': 5705644, 'write_bytes': 0, 'vmem': 8723300, 'read_bytes': 0, 'stime': 147, 'tx_bytes': 12782603, 'pss': 2238315, 'wchar': 0, 'rchar': 0, 'tx_packets': 30421, 'swap': 0, 'utime': 3039}, 'Avg': {'write_bytes': 0, 'nprocs': 6, 'nthreads': 0, 'rx_bytes': 19486, 'rx_packets': 22, 'vmem': 5903412, 'read_bytes': 0, 'swap': 0, 'tx_bytes': 5460, 'pss': 1601471, 'wchar': 0, 'rchar': 0, 'tx_packets': 12, 'rss': 3615290}}
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,187 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | extracted standard info from prmon json
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | extracted standard memory fields from prmon json
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | ..............................
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . Timing measurements:
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . get job = 0 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . initial setup = 1 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . payload setup = 0 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . total setup = 1 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . stage-in = 0 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,188 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . payload execution = 2386 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,189 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | . stage-out = 3 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,189 | INFO     | queue_monitor       | pilot.util.auxiliary.4002876565  | timing_report             | ..............................
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,190 | DEBUG    | queue_monitor       | pilot.control.job.4002876565     | send_state                | wrote heartbeat to file /var/lib/boinc/slots/0/heartbeat.json
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,191 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | job 4002876565 was dequeued from the monitored payloads queue
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,191 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | tmp job object deleted
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,212 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | job summary report
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | --------------------------------------------------
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | PanDA job id: 4002876565
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | task id: 000649-197212-17933
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | errors: (none)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | status: LOG_TRANSFER = DONE 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | pilot state: finished 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | transexitcode: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | exeerrorcode: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | exeerrordiag: 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | exitcode: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,213 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | exitmsg: OK
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | cpuconsumptiontime: 3198 s
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | nevents: 10
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | neventsw: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | pid: 30386
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | pgrp: 30386
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | corecount: 2
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | event service: False
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,214 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | --------------------------------------------------
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.auxiliary.4002876565  | make_job_report           | 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue jobs has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue payloads has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue data_in has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue data_out has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue current_data_in has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue validated_jobs has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue validated_payloads has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue monitored_payloads has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_jobs has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_payloads has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,215 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_data_in has 1 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_data_out has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_jobs has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_payloads has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_data_in has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_data_out has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue completed_jobs has 0 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue completed_jobids has 1 job(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.control.job.4002876565     | has_job_completed         | job 4002876565 has completed (purged errors)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,216 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | overall cleanup function is called
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:04,221 | DEBUG    | retrieve            | pilot.util.processes             | cleanup                   | work directory was removed: /var/lib/boinc/slots/0/PanDA_Pilot-4002876565
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:05,224 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | --- collectZombieJob: --- 10, [30386]
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:05,224 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | zombie collector trying to kill pid 30386
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:05,225 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | harmless exception when collecting zombies: [Errno 10] No child processes
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,227 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | collected zombie processes
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,227 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | will now attempt to kill all subprocesses of pid=30386
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,308 | INFO     | retrieve            | pilot.util.processes             | kill_processes            | process IDs to be killed: [30386] (in reverse order)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,391 | WARNING  | retrieve            | pilot.util.processes             | kill_processes            | found no corresponding commands to process id(s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,391 | INFO     | retrieve            | pilot.util.processes             | kill_orphans              | Do not look for orphan processes in BOINC jobs
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,392 | INFO     | retrieve            | pilot.control.job                | retrieve                  | ready for new job
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,392 | INFO     | retrieve            | root                             | retrieve                  | pilot has finished for previous job - re-establishing logging
Mo 20. Jan 23:11:02 CET 2020: mpi4py not found
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,402 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,403 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ***  PanDA Pilot version 2.3.4 (12)  ***
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,403 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,403 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | 
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,403 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | pilot is running in a VM
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,403 | INFO     | retrieve            | pilot.util.auxiliary             | display_architecture_info | architecture information:
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,530 | INFO     | retrieve            | pilot.util.auxiliary             | display_architecture_info | 
Mo 20. Jan 23:11:02 CET 2020: LSB Version:	:core-4.1-amd64:core-4.1-noarch
Mo 20. Jan 23:11:02 CET 2020: Distributor ID:	CentOS
Mo 20. Jan 23:11:02 CET 2020: Description:	CentOS Linux release 7.6.1810 (Core) 
Mo 20. Jan 23:11:02 CET 2020: Release:	7.6.1810
Mo 20. Jan 23:11:02 CET 2020: Codename:	Core
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:06,530 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,040 | DEBUG    | retrieve            | pilot.util.monitoring            | check_local_space         | checking local space on /var/lib/boinc/slots/0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,088 | INFO     | retrieve            | pilot.util.monitoring            | check_local_space         | sufficient remaining disk space (7460618240 B)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,089 | WARNING  | retrieve            | pilot.control.job                | proceed_with_getjob       | since timefloor is set to 0, pilot was only allowed to run one job
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,089 | DEBUG    | retrieve            | pilot.control.job                | retrieve                  | [job] retrieve thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,166 | DEBUG    | data                | pilot.control.data               | control                   | data control ending since graceful_stop has been set
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,166 | DEBUG    | data                | pilot.control.data               | control                   | [data] control thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,226 | INFO     | execute_payloads    | pilot.control.payload            | execute_payloads          | [payload] execute_payloads thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,255 | INFO     | monitor             | pilot.control.monitor            | control                   | [monitor] control thread has ended
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,566 | DEBUG    | payload             | pilot.control.payload            | control                   | payload control ending since graceful_stop has been set
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,567 | DEBUG    | payload             | pilot.control.payload            | control                   | [payload] control thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,581 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 12 threads
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,581 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 139784756082496)>, <ExcThread(job, started 139784593663744)>, <ExcThread(validate_post, started 139783602960128)>, <ExcThread(queue_monitor, started 139784181761792)>, <ExcThread(failed_post, started 139783594567424)>, <ExcThread(validate, started 139784585271040)>, <ExcThread(queue_monitoring, started 139784190154496)>, <ExcThread(copytool_out, started 139784206939904)>, <ExcThread(validate_pre, started 139784164976384)>, <ExcThread(copytool_in, started 139784173369088)>, <ExcThread(job_monitor, started 139784560092928)>, <ExcThread(create_data_payload, started 139784576878336)>]
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,768 | DEBUG    | job                 | pilot.control.job                | control                   | job control ending since graceful_stop has been set
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,768 | DEBUG    | job                 | pilot.control.job                | control                   | [job] control thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:07,825 | WARNING  | copytool_out        | pilot.util.common                | should_abort              | data:copytool_out:received graceful stop - abort after this iteration
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,034 | INFO     | failed_post         | pilot.control.payload            | failed_post               | [payload] failed_post thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,037 | INFO     | validate_post       | pilot.control.payload            | validate_post             | [payload] validate_post thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,529 | DEBUG    | copytool_in         | pilot.control.data               | copytool_in               | [data] copytool_in thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,529 | INFO     | validate_pre        | pilot.control.payload            | validate_pre              | [payload] validate_pre thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,534 | DEBUG    | validate            | pilot.control.job                | validate                  | [job] validate thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,535 | DEBUG    | create_data_payload | pilot.control.job                | create_data_payload       | [job] create_data_payload thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,588 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 5 threads
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,588 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 139784756082496)>, <ExcThread(queue_monitor, started 139784181761792)>, <ExcThread(queue_monitoring, started 139784190154496)>, <ExcThread(copytool_out, started 139784206939904)>, <ExcThread(job_monitor, started 139784560092928)>]
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:08,827 | DEBUG    | copytool_out        | pilot.control.data               | copytool_out              | [data] copytool_out thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:09,209 | WARNING  | queue_monitor       | pilot.util.common                | should_abort              | job:queue_monitor:received graceful stop - abort after this iteration
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:09,209 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | [job] queue monitor thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:09,601 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 3 threads
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:09,601 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 139784756082496)>, <ExcThread(queue_monitoring, started 139784190154496)>, <ExcThread(job_monitor, started 139784560092928)>]
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:10,252 | WARNING  | queue_monitoring    | pilot.util.common                | should_abort              | data:queue_monitoring:received graceful stop - abort after this iteration
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:13,257 | DEBUG    | queue_monitoring    | pilot.control.data               | queue_monitoring          | [data] queue_monitor thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:13,676 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 2 threads
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:10:13,676 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 139784756082496)>, <ExcThread(job_monitor, started 139784560092928)>]
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02,120 | WARNING  | job_monitor         | pilot.control.job                | check_job_monitor_waiting_time | no jobs in monitored_payloads queue (waited for 62 s)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02,120 | DEBUG    | job_monitor         | pilot.control.job                | job_monitor               | [job] job monitor thread has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02,197 | INFO     | MainThread          | pilot.workflow.generic           | run                       | end of generic workflow (traces error code: 0)
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02,198 | INFO     | MainThread          | root                             | wrap_up                   | traces error code: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02,198 | INFO     | MainThread          | root                             | wrap_up                   | pilot has finished
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] ==== pilot stdout END ====
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] ==== wrapper stdout RESUME ====
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] Pilot exit status: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] STATUSCODE: 0
Mo 20. Jan 23:11:02 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] apfmon messages muted
Mo 20. Jan 23:11:02 CET 2020: ---- find pandaID.out ----
Mo 20. Jan 23:11:02 CET 2020: total 60
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc 11357 Jul 25 16:38 LICENSE
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc    20 Sep  9 13:04 MANIFEST.IN
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc     8 Dec 12 19:00 PILOTVERSION
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc  2212 Nov 14 11:01 README.md
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc   221 Jul 25 16:38 TODO.md
Mo 20. Jan 23:11:02 CET 2020: -rw-------.  1 boinc boinc    11 Jan 20 22:29 pandaIDs.out
Mo 20. Jan 23:11:02 CET 2020: drwx------. 14 boinc boinc   216 Jan 20 22:29 pilot
Mo 20. Jan 23:11:03 CET 2020: -rwx------.  1 boinc boinc 21225 Dec 12 19:00 pilot.py
Mo 20. Jan 23:11:03 CET 2020: -rw-------.  1 boinc boinc   766 Oct 10 16:01 setup.py
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc 11 Jan 20 22:29 /var/lib/boinc/slots/0/pilot2/pandaIDs.out
Mo 20. Jan 23:11:03 CET 2020: 4002876565
Mo 20. Jan 23:11:03 CET 2020: 
Mo 20. Jan 23:11:03 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] Test setup, not cleaning
Mo 20. Jan 23:11:03 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] ==== wrapper stdout END ====
Mo 20. Jan 23:11:03 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] ==== wrapper stderr END ====
Mo 20. Jan 23:11:03 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] wrapper wrapperexiting ec=0, duration=2480
Mo 20. Jan 23:11:03 CET 2020: 2020-01-20 22:11:02 UTC [wrapper] apfmon messages muted
Mo 20. Jan 23:11:03 CET 2020:  *** Error codes and diagnostics ***
Mo 20. Jan 23:11:03 CET 2020:     "exeErrorCode": 0,
Mo 20. Jan 23:11:03 CET 2020:     "exeErrorDiag": "",
Mo 20. Jan 23:11:03 CET 2020:     "pilotErrorCode": 0,
Mo 20. Jan 23:11:03 CET 2020:     "pilotErrorDiag": "",
Mo 20. Jan 23:11:03 CET 2020:  *** Listing of results directory ***
Mo 20. Jan 23:11:03 CET 2020: insgesamt 379052
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc    267260 20. Jan 16:32 pilot2.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      4492 20. Jan 17:15 queuedata.json
Mo 20. Jan 23:11:03 CET 2020: -rwx------. 1 boinc boinc     12641 20. Jan 17:17 runpilot2-wrapper.sh
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       107 20. Jan 22:29 wrapper_26015_x86_64-pc-linux-gnu
Mo 20. Jan 23:11:03 CET 2020: -rwxr-xr-x. 1 boinc boinc      5557 20. Jan 22:29 run_atlas
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       112 20. Jan 22:29 job.xml
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      5991 20. Jan 22:29 init_data.xml
Mo 20. Jan 23:11:03 CET 2020: drwxrwx--x. 2 boinc boinc        86 20. Jan 22:29 shared
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc         0 20. Jan 22:29 boinc_lockfile
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      8549 20. Jan 22:29 start_atlas.sh
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       815 20. Jan 22:29 RTE.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc    275409 20. Jan 22:29 input.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc 365251149 20. Jan 22:29 EVNT.14296418._001447.pool.root.1
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      2948 20. Jan 22:29 pandaJob.out
Mo 20. Jan 23:11:03 CET 2020: drwxr-xr-x. 3 boinc boinc        17 20. Jan 22:29 APPS
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc   3699157 20. Jan 22:29 agis_schedconf.cvmfs.json
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc   7792440 20. Jan 22:29 agis_ddmendpoints.json
Mo 20. Jan 23:11:03 CET 2020: drwx------. 3 boinc boinc       229 20. Jan 22:29 pilot2
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       535 20. Jan 22:57 boinc_task_state.xml
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc   9176612 20. Jan 23:09 HITS.000649-197212-17933._078090.pool.root.1
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc       786 20. Jan 23:09 memory_monitor_summary.json
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc    518885 20. Jan 23:10 log.000649-197212-17933._078090.job.log.tgz.1
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc     11662 20. Jan 23:10 heartbeat.json
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc        26 20. Jan 23:10 wrapper_checkpoint.txt
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      8192 20. Jan 23:10 boinc_mmap_file
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc      8489 20. Jan 23:11 pilotlog.txt
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc    232728 20. Jan 23:11 log.000649-197212-17933._078090.job.log.1
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc       490 20. Jan 23:11 bO5NDmluoDwnShfckohDCDFpABFKDmABFKDmx9YMDmABFKDmw4QQjm.diag
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc       496 20. Jan 23:11 output.list
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      7164 20. Jan 23:11 runtime_log.err
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       739 20. Jan 23:11 runtime_log
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc    778240 20. Jan 23:11 result.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      2213 20. Jan 23:11 stderr.txt
Mo 20. Jan 23:11:03 CET 2020: HITS file was successfully produced:
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc 9176612 20. Jan 23:09 shared/HITS.pool.root.1
Mo 20. Jan 23:11:03 CET 2020:  *** Contents of shared directory: ***
Mo 20. Jan 23:11:03 CET 2020: insgesamt 366704
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc 365251149 20. Jan 22:29 ATLAS.root_0
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc      8549 20. Jan 22:29 start_atlas.sh
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc       815 20. Jan 22:29 RTE.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-r--r--. 1 boinc boinc    275409 20. Jan 22:29 input.tar.gz
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc   9176612 20. Jan 23:09 HITS.pool.root.1
Mo 20. Jan 23:11:03 CET 2020: -rw-------. 1 boinc boinc    778240 20. Jan 23:11 result.tar.gz
23:11:03 (23495): run_atlas exited; CPU time 3145.824883
23:11:03 (23495): called boinc_finish(0)

</stderr_txt>
]]>


©2024 CERN