Name CMS_2078642_1619095895.968948_0
Workunit 2071668
Created 22 Apr 2021, 12:51:37 UTC
Sent 23 Apr 2021, 10:58:37 UTC
Report deadline 30 Apr 2021, 10:58:37 UTC
Received 24 Apr 2021, 6:58:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 4274
Run time 5 hours 59 min 40 sec
CPU time 8 hours 46 min 12 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 2.76 GFLOPS
Application version CMS Simulation v50.00 (vbox64_mt_mcore_cms)
windows_x86_64
Peak working set size 74.05 MB
Peak swap size 85.56 MB
Peak disk usage 2.90 GB

Stderr output

<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
Process still present 5 min after writing finish file; aborting</message>
<stderr_txt>
2021-04-23 06:59:58 (11588): Detected: vboxwrapper 26197
2021-04-23 06:59:58 (11588): Detected: BOINC client v7.7
2021-04-23 06:59:59 (11588): Detected: VirtualBox VboxManage Interface (Version: 5.2.44)
2021-04-23 06:59:59 (11588): Detected: Heartbeat check (file: 'heartbeat' every 1200.000000 seconds)
2021-04-23 06:59:59 (11588): Successfully copied 'init_data.xml' to the shared directory.
2021-04-23 07:00:01 (11588): Create VM. (boinc_149ce3e4b191418a, slot#7)
2021-04-23 07:00:02 (11588): Setting Memory Size for VM. (1896MB)
2021-04-23 07:00:03 (11588): Setting CPU Count for VM. (1)
2021-04-23 07:00:03 (11588): Setting Chipset Options for VM.
2021-04-23 07:00:03 (11588): Setting Boot Options for VM.
2021-04-23 07:00:04 (11588): Setting Network Configuration for NAT.
2021-04-23 07:00:04 (11588): Enabling VM Network Access.
2021-04-23 07:00:05 (11588): Disabling USB Support for VM.
2021-04-23 07:00:05 (11588): Disabling COM Port Support for VM.
2021-04-23 07:00:05 (11588): Disabling LPT Port Support for VM.
2021-04-23 07:00:06 (11588): Disabling Audio Support for VM.
2021-04-23 07:00:06 (11588): Disabling Clipboard Support for VM.
2021-04-23 07:00:06 (11588): Disabling Drag and Drop Support for VM.
2021-04-23 07:00:07 (11588): Adding storage controller(s) to VM.
2021-04-23 07:00:07 (11588): Adding virtual disk drive to VM. (vm_image.vdi)
2021-04-23 07:00:08 (11588): Adding VirtualBox Guest Additions to VM.
2021-04-23 07:00:08 (11588): Adding network bandwidth throttle group to VM. (Defaulting to 1024GB)
2021-04-23 07:00:08 (11588): forwarding host port 62504 to guest port 80
2021-04-23 07:00:09 (11588): Enabling remote desktop for VM.
2021-04-23 07:00:09 (11588): Required extension pack not installed, remote desktop not enabled.
2021-04-23 07:00:09 (11588): Enabling shared directory for VM.
2021-04-23 07:00:10 (11588): Starting VM using VBoxManage interface. (boinc_149ce3e4b191418a, slot#7)
2021-04-23 07:00:22 (11588): Successfully started VM. (PID = '5200')
2021-04-23 07:00:22 (11588): Reporting VM Process ID to BOINC.
2021-04-23 07:00:22 (11588): Guest Log: BIOS: VirtualBox 5.2.44

2021-04-23 07:00:22 (11588): Guest Log: CPUID EDX: 0x078bfbff

2021-04-23 07:00:22 (11588): Guest Log: BIOS: ata0-0: PCHS=16383/16/63 LCHS=1024/255/63

2021-04-23 07:00:22 (11588): VM state change detected. (old = 'PoweredOff', new = 'Running')
2021-04-23 07:00:22 (11588): Detected: Web Application Enabled (http://localhost:62504)
2021-04-23 07:00:22 (11588): Preference change detected
2021-04-23 07:00:22 (11588): Setting CPU throttle for VM. (100%)
2021-04-23 07:00:22 (11588): Setting checkpoint interval to 600 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 600 seconds))
2021-04-23 07:00:24 (11588): Guest Log: BIOS: Boot : bseqnr=1, bootseq=0032

2021-04-23 07:00:24 (11588): Guest Log: BIOS: Booting from Hard Disk...

2021-04-23 07:00:27 (11588): Guest Log: BIOS: KBD: unsupported int 16h function 03

2021-04-23 07:00:27 (11588): Guest Log: BIOS: AX=0305 BX=0000 CX=0000 DX=0000 

2021-04-23 07:00:47 (11588): Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds

2021-04-23 07:00:47 (11588): Guest Log: vboxguest: misc device minor 56, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)

2021-04-23 07:01:22 (11588): Guest Log: VBoxService 5.2.6 r120293 (verbosity: 0) linux.amd64 (Jan 15 2018 14:51:00) release log

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.000511 main     Log opened 2021-04-23T11:01:20.601330000Z

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.000932 main     OS Product: Linux

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.001184 main     OS Release: 4.14.157-17.cernvm.x86_64

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.001224 main     OS Version: #1 SMP Wed Dec 4 17:26:45 CET 2019

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.001258 main     Executable: /usr/share/vboxguest52/usr/sbin/VBoxService

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.001260 main     Process ID: 2959

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.001261 main     Package type: LINUX_64BITS_GENERIC

2021-04-23 07:01:22 (11588): Guest Log: 00:00:00.002838 main     5.2.6 r120293 started. Verbose level = 0

2021-04-23 07:02:01 (11588): Guest Log: [INFO] Mounting the shared directory

2021-04-23 07:02:02 (11588): Guest Log: [INFO] Shared directory mounted, enabling vboxmonitor

2021-04-23 07:02:02 (11588): Guest Log: [DEBUG] Testing network connection to cern.ch on port 80

2021-04-23 07:02:02 (11588): Guest Log: [DEBUG] Connection to cern.ch 80 port [tcp/http] succeeded!

2021-04-23 07:02:02 (11588): Guest Log: [DEBUG] 0

2021-04-23 07:02:02 (11588): Guest Log: [DEBUG] Testing VCCS connection to vccs.cern.ch on port 443

2021-04-23 07:02:03 (11588): Guest Log: [DEBUG] Connection to vccs.cern.ch 443 port [tcp/https] succeeded!

2021-04-23 07:02:03 (11588): Guest Log: [DEBUG] 0

2021-04-23 07:02:03 (11588): Guest Log: [DEBUG] Testing connection to Condor server on port 9618

2021-04-23 07:02:04 (11588): Guest Log: [DEBUG] Connection to vocms0840.cern.ch 9618 port [tcp/condor] succeeded!

2021-04-23 07:02:04 (11588): Guest Log: [DEBUG] 0

2021-04-23 07:02:04 (11588): Guest Log: [DEBUG] Testing connection to WMAgent server on port 4080

2021-04-23 07:02:05 (11588): Guest Log: [DEBUG] Connection to vocms0267.cern.ch 4080 port [tcp/lorica-in] succeeded!

2021-04-23 07:02:05 (11588): Guest Log: [DEBUG] 0

2021-04-23 07:02:05 (11588): Guest Log: [DEBUG] Testing connection to Frontier server on port 8080

2021-04-23 07:02:05 (11588): Guest Log: [DEBUG] Connection to cms-frontier.openhtc.io 8080 port [tcp/webcache] succeeded!

2021-04-23 07:02:05 (11588): Guest Log: [DEBUG] 0

2021-04-23 07:02:08 (11588): Guest Log: [INFO] CVMFS and Frontier will use DIRECT instead of an HTTP proxy.

2021-04-23 07:02:08 (11588): Guest Log: [INFO] A local HTTP proxy could help making the network usage

2021-04-23 07:02:08 (11588): Guest Log: [INFO] of this application more efficient. It would also help to

2021-04-23 07:02:08 (11588): Guest Log: [INFO] offload the project servers.

2021-04-23 07:02:08 (11588): Guest Log: [INFO] Details can be found in the project forum.

2021-04-23 07:02:27 (11588): Guest Log: [INFO] Reloading the CVMFS configuration (can take a while) ...

2021-04-23 07:03:49 (11588): Guest Log: [INFO] Probing CVMFS ...

2021-04-23 07:04:03 (11588): Guest Log: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE

2021-04-23 07:04:03 (11588): Guest Log: 2.4.4.0 3741 2 25852 15007 3 1 1243091 4096000 2 65024 0 2 100 0 0 http://s1cern-cvmfs.openhtc.io/cvmfs/grid.cern.ch DIRECT 1

2021-04-23 07:04:25 (11588): Guest Log: [INFO] Reading volunteer information

2021-04-23 07:04:26 (11588): Guest Log: [INFO] Volunteer: Contact (21)

2021-04-23 07:04:26 (11588): Guest Log: [INFO] VMID: 49b2fac1-df25-48d2-a4ee-4612ca6a31f8

2021-04-23 07:04:27 (11588): Guest Log: [INFO] Requesting an X509 credential from LHC@home

2021-04-23 07:04:29 (11588): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev

2021-04-23 07:04:31 (11588): Guest Log: [INFO] Running the fast benchmark.

2021-04-23 07:05:28 (11588): Guest Log: [INFO] Machine performance 9.10 HEPSPEC06

2021-04-23 07:05:28 (11588): Guest Log: [INFO] CMS application starting. Check log files.

2021-04-23 07:05:31 (11588): Guest Log: [DEBUG] HTCondor ping

2021-04-23 07:05:34 (11588): Guest Log: [DEBUG] 0

2021-04-23 08:40:33 (11588): Status Report: Job Duration: '64800.000000'
2021-04-23 08:40:33 (11588): Status Report: Elapsed Time: '6000.000000'
2021-04-23 08:40:33 (11588): Status Report: CPU Time: '5366.234375'
2021-04-23 10:20:50 (11588): Status Report: Job Duration: '64800.000000'
2021-04-23 10:20:50 (11588): Status Report: Elapsed Time: '12000.000000'
2021-04-23 10:20:50 (11588): Status Report: CPU Time: '11334.281250'
2021-04-23 12:00:57 (11588): Status Report: Job Duration: '64800.000000'
2021-04-23 12:00:57 (11588): Status Report: Elapsed Time: '18000.000000'
2021-04-23 12:00:57 (11588): Status Report: CPU Time: '16315.609375'
2021-04-23 13:08:31 (11588): Stopping VM.
2021-04-23 13:11:47 (4928): Detected: vboxwrapper 26197
2021-04-23 13:11:47 (4928): Detected: BOINC client v7.7
2021-04-23 13:11:48 (4928): Detected: VirtualBox VboxManage Interface (Version: 5.2.44)
2021-04-23 13:11:48 (4928): Detected: Heartbeat check (file: 'heartbeat' every 1200.000000 seconds)
2021-04-23 13:11:48 (4928): Starting VM using VBoxManage interface. (boinc_149ce3e4b191418a, slot#7)
2021-04-23 13:12:57 (4928): Successfully started VM. (PID = '10496')
2021-04-23 13:12:57 (4928): Reporting VM Process ID to BOINC.
2021-04-23 13:12:57 (4928): VM state change detected. (old = 'PoweredOff', new = 'Running')
2021-04-23 13:12:57 (4928): Detected: Web Application Enabled (http://localhost:62504)
2021-04-23 13:12:58 (4928): Status Report: Job Duration: '64800.000000'
2021-04-23 13:12:58 (4928): Status Report: Elapsed Time: '22044.001974'
2021-04-23 13:12:58 (4928): Status Report: CPU Time: '20335.203125'
2021-04-23 13:12:58 (4928): Preference change detected
2021-04-23 13:12:58 (4928): Setting CPU throttle for VM. (100%)
2021-04-23 13:13:13 (4928): Setting checkpoint interval to 600 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 600 seconds))
2021-04-23 14:53:02 (4928): Status Report: Job Duration: '64800.000000'
2021-04-23 14:53:03 (4928): Status Report: Elapsed Time: '28044.756479'
2021-04-23 14:53:03 (4928): Status Report: CPU Time: '26313.484375'
2021-04-23 16:33:05 (4928): Status Report: Job Duration: '64800.000000'
2021-04-23 16:33:05 (4928): Status Report: Elapsed Time: '34045.121521'
2021-04-23 16:33:05 (4928): Status Report: CPU Time: '31207.828125'
2021-04-23 18:13:07 (4928): Status Report: Job Duration: '64800.000000'
2021-04-23 18:13:07 (4928): Status Report: Elapsed Time: '40045.740103'
2021-04-23 18:13:07 (4928): Status Report: CPU Time: '31349.625000'
2021-04-23 19:53:09 (4928): Status Report: Job Duration: '64800.000000'
2021-04-23 19:53:09 (4928): Status Report: Elapsed Time: '46045.854397'
2021-04-23 19:53:09 (4928): Status Report: CPU Time: '31490.406250'
20:46:41 (4928): BOINC client no longer exists - exiting
20:46:41 (4928): timer handler: client dead, exiting
2021-04-23 20:46:42 (4928): Stopping VM.
20:46:51 (4928): BOINC client no longer exists - exiting
20:46:51 (4928): timer handler: client dead, exiting
20:47:01 (4928): BOINC client no longer exists - exiting
20:47:01 (4928): timer handler: client dead, exiting
20:47:12 (4928): BOINC client no longer exists - exiting
20:47:12 (4928): timer handler: client dead, exiting
20:47:22 (4928): BOINC client no longer exists - exiting
20:47:22 (4928): timer handler: client dead, exiting
20:47:32 (4928): BOINC client no longer exists - exiting
20:47:32 (4928): timer handler: client dead, exiting
20:47:43 (4928): BOINC client no longer exists - exiting
20:47:43 (4928): timer handler: client dead, exiting
20:47:54 (4928): BOINC client no longer exists - exiting
20:47:54 (4928): timer handler: client dead, exiting
20:48:05 (4928): BOINC client no longer exists - exiting
20:48:05 (4928): timer handler: client dead, exiting
20:48:16 (4928): BOINC client no longer exists - exiting
20:48:16 (4928): timer handler: client dead, exiting
20:48:27 (4928): BOINC client no longer exists - exiting
20:48:27 (4928): timer handler: client dead, exiting
20:48:38 (4928): BOINC client no longer exists - exiting
20:48:38 (4928): timer handler: client dead, exiting
20:48:49 (4928): BOINC client no longer exists - exiting
20:48:49 (4928): timer handler: client dead, exiting
20:49:00 (4928): BOINC client no longer exists - exiting
20:49:00 (4928): timer handler: client dead, exiting
20:49:11 (4928): BOINC client no longer exists - exiting
20:49:11 (4928): timer handler: client dead, exiting
20:49:22 (4928): BOINC client no longer exists - exiting
20:49:22 (4928): timer handler: client dead, exiting
20:49:33 (4928): BOINC client no longer exists - exiting
20:49:33 (4928): timer handler: client dead, exiting
20:49:44 (4928): BOINC client no longer exists - exiting
20:49:44 (4928): timer handler: client dead, exiting
20:49:54 (4928): BOINC client no longer exists - exiting
20:49:54 (4928): timer handler: client dead, exiting
20:50:05 (4928): BOINC client no longer exists - exiting
20:50:05 (4928): timer handler: client dead, exiting
20:50:16 (4928): BOINC client no longer exists - exiting
20:50:16 (4928): timer handler: client dead, exiting
20:50:27 (4928): BOINC client no longer exists - exiting
20:50:27 (4928): timer handler: client dead, exiting
20:50:38 (4928): BOINC client no longer exists - exiting
20:50:38 (4928): timer handler: client dead, exiting
20:50:49 (4928): BOINC client no longer exists - exiting
20:50:49 (4928): timer handler: client dead, exiting
20:51:00 (4928): BOINC client no longer exists - exiting
20:51:00 (4928): timer handler: client dead, exiting
20:51:11 (4928): BOINC client no longer exists - exiting
20:51:11 (4928): timer handler: client dead, exiting
20:51:22 (4928): BOINC client no longer exists - exiting
20:51:22 (4928): timer handler: client dead, exiting
20:51:33 (4928): BOINC client no longer exists - exiting
20:51:33 (4928): timer handler: client dead, exiting
20:51:44 (4928): BOINC client no longer exists - exiting
20:51:44 (4928): timer handler: client dead, exiting
20:51:55 (4928): BOINC client no longer exists - exiting
20:51:55 (4928): timer handler: client dead, exiting
20:52:05 (4928): BOINC client no longer exists - exiting
20:52:05 (4928): timer handler: client dead, exiting
20:52:16 (4928): BOINC client no longer exists - exiting
20:52:16 (4928): timer handler: client dead, exiting
2021-04-23 20:52:21 (4928): VM did not stop when requested.
2021-04-23 20:52:21 (4928): VM was successfully terminated.
2021-04-23 22:08:59 (6236): Detected: vboxwrapper 26197
2021-04-23 22:08:59 (6236): Detected: BOINC client v7.7
2021-04-23 22:09:01 (6236): Detected: VirtualBox VboxManage Interface (Version: 5.2.44)
2021-04-23 22:09:01 (6236): Detected: Heartbeat check (file: 'heartbeat' every 1200.000000 seconds)
2021-04-23 22:09:01 (6236): Starting VM using VBoxManage interface. (boinc_149ce3e4b191418a, slot#7)
2021-04-23 22:09:58 (6236): Successfully started VM. (PID = '9244')
2021-04-23 22:09:58 (6236): Reporting VM Process ID to BOINC.
2021-04-23 22:09:58 (6236): Guest Log: 13:41:24.024664 timesync vgsvcTimeSyncWorker: Radical host time change: 4 994 657 000 000ns (HostNow=1 619 230 194 716 000 000 ns HostLast=1 619 225 200 059 000 000 ns)

2021-04-23 22:09:58 (6236): VM state change detected. (old = 'PoweredOff', new = 'Running')
2021-04-23 22:09:58 (6236): Detected: Web Application Enabled (http://localhost:62504)
2021-04-23 22:09:58 (6236): Status Report: Job Duration: '64800.000000'
2021-04-23 22:09:58 (6236): Status Report: Elapsed Time: '49257.377041'
2021-04-23 22:09:58 (6236): Status Report: CPU Time: '31572.921875'
2021-04-23 22:09:58 (6236): Preference change detected
2021-04-23 22:09:58 (6236): Setting CPU throttle for VM. (100%)
2021-04-23 22:09:59 (6236): Setting checkpoint interval to 600 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 600 seconds))
2021-04-23 22:10:08 (6236): Guest Log: 13:41:34.066456 timesync vgsvcTimeSyncWorker: Radical guest time change: 4 993 856 607 000ns (GuestNow=1 619 230 205 023 265 000 ns GuestLast=1 619 225 211 166 658 000 ns fSetTimeLastLoop=true )

22:11:45 (4060): Can't acquire lockfile (32) - waiting 35s
22:12:20 (4060): Can't acquire lockfile (32) - exiting
22:12:20 (4060): Error: The process cannot access the file because it is being used by another process.

 (0x20)
22:40:45 (11608): Can't acquire lockfile (32) - waiting 35s
22:41:20 (11608): Can't acquire lockfile (32) - exiting
22:41:20 (11608): Error: The process cannot access the file because it is being used by another process.

 (0x20)
23:38:21 (11488): Can't acquire lockfile (32) - waiting 35s
23:38:56 (11488): Can't acquire lockfile (32) - exiting
23:38:56 (11488): Error: The process cannot access the file because it is being used by another process.

 (0x20)
2021-04-23 23:52:37 (6236): Status Report: Job Duration: '64800.000000'
2021-04-23 23:52:37 (6236): Status Report: Elapsed Time: '55258.149702'
2021-04-23 23:52:37 (6236): Status Report: CPU Time: '31694.968750'
00:30:24 (7596): Can't acquire lockfile (32) - waiting 35s
00:30:59 (7596): Can't acquire lockfile (32) - exiting
00:30:59 (7596): Error: The process cannot access the file because it is being used by another process.

 (0x20)
01:13:51 (5728): Can't acquire lockfile (32) - waiting 35s
01:14:27 (5728): Can't acquire lockfile (32) - exiting
01:14:27 (5728): Error: The process cannot access the file because it is being used by another process.

 (0x20)
2021-04-24 01:32:45 (6236): Status Report: Job Duration: '64800.000000'
2021-04-24 01:32:45 (6236): Status Report: Elapsed Time: '61259.088204'
2021-04-24 01:32:45 (6236): Status Report: CPU Time: '31789.890625'
01:35:01 (4376): Can't acquire lockfile (32) - waiting 35s
01:35:36 (4376): Can't acquire lockfile (32) - exiting
01:35:36 (4376): Error: The process cannot access the file because it is being used by another process.

 (0x20)
01:57:08 (4060): Can't acquire lockfile (32) - waiting 35s
01:57:43 (4060): Can't acquire lockfile (32) - exiting
01:57:43 (4060): Error: The process cannot access the file because it is being used by another process.

 (0x20)
02:10:25 (2212): Can't acquire lockfile (32) - waiting 35s
02:11:00 (2212): Can't acquire lockfile (32) - exiting
02:11:00 (2212): Error: The process cannot access the file because it is being used by another process.

 (0x20)
2021-04-24 02:31:51 (6236): Powering off VM.
2021-04-24 02:32:42 (6236): Error in poweroff VM for VM: -182
Command:
VBoxManage -q controlvm "boinc_149ce3e4b191418a" poweroff
Output:
0%...10%...20%...30%...40%...50%...
2021-04-24 02:32:42 (6236): VM did not power off when requested.
2021-04-24 02:32:42 (6236): VM was successfully terminated.
2021-04-24 02:32:42 (6236): Deregistering VM. (boinc_149ce3e4b191418a, slot#7)
2021-04-24 02:33:02 (6236): Removing network bandwidth throttle group from VM.
2021-04-24 02:33:08 (6236): Removing VM from VirtualBox.
02:33:28 (6236): called boinc_finish(0)

</stderr_txt>
]]>


©2024 CERN