Message boards :
News :
New CMS App v46.26
Message board moderation
| Author | Message |
|---|---|
|
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 862,257 RAC: 8   |
Version No. in title and post conflicting . . . |
|
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
FYI Startup times (on a 10MBit/s line)act. 11.1MBits/s From joining project to CMS task starting: 18min. Time from CMS-task starting to processing 1st event: 8min. |
 ivan Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,310,612 RAC: 37  |
Version No. in title and post conflicting . . . FTFY 
|
|
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
FYI (10 MBits/sec line) Startup time with CMS project already loaded: From boinc cms-Task start to 1st event processing start : 8 min (rounded up). It used to be around 24 min, so quite an improvement. |
Laurence Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0 |
The base CernVM image is 17MB. When contextualizing the image, the new squid monitoring showed that 1.01GB was download. From previous tests, 50MB is what the OS downloads to arrive at the command prompt so the rest is what is needed for the CMS application. The result is a 1.3GB compressed image that is downloaded from the project server. So 10 mins of the first task was mainly used to download the image and should not be needed again as the last time we updated the image was 13th March 2015! When running a task only 48MB is downloaded. Another change will be made within the next day or so to remove the CVMFS reload command on boot which should save another 2mins. |
|
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
I just did a computer restart, no suspending or shutting down boinc or cms. When starting boinc, the old job was not resumed and it started from scratch. |
|
Laurence Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0 |
This is an issue with the BOINC client. When the computer is restarted, the VM goes in to the power-off state. Ideally, the BOINC client should intercept the shutdown request and save the VM. Does the same thing happen with the Theory Simulations in vLHC@home? |
|
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 862,257 RAC: 8 |
I just did a computer restart, no suspending or shutting down boinc or cms. Normal behaviour. If you don't save your (BOINC's) work, before a sudden boot, that work will be lost. Stopping BOINC client first before the reboot, would have saved the work to disk. |
|
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
Does the same thing happen with the Theory Simulations in vLHC@home? I don't know, i have not done any. |
|
Send message Joined: 20 Mar 15 Posts: 243 Credit: 886,442 RAC: 0  |
This is an issue with the BOINC client. When the computer is restarted, the VM goes in to the power-off state. Ideally, the BOINC client should intercept the shutdown request and save the VM. Does the same thing happen with the Theory Simulations in vLHC@home? As far as I'm aware, the saving of VMs works very well with vLHC. This is a piece of the VM Manager screen. VBox 4.3.26, BOINC 7.2.42, CMS 46.24. One of these VMs is CMS, the other is T4T. This is the result of stopping the client using boinccmd --quit. The GUI "Shut down connected client" does the same thing. On the host shown, this is done by a cron job followed a couple of minutes later by shutting down and powering off the host (another cron job). The Manager doesn't normally run. Not sure exactly what happens when the host is restarted; whether there is an intermediate "powered off" VM state or not but vLHC starts up again and T4T events simply carry from where they left off.  |
|
Send message Joined: 20 Mar 15 Posts: 14 Credit: 5,132 RAC: 0 
|
This is an issue with the BOINC client. When the computer is restarted, the VM goes in to the power-off state. Ideally, the BOINC client should intercept the shutdown request and save the VM. Does the same thing happen with the Theory Simulations in vLHC@home? I just verified that the save state stuff is working on CMS (at least as well as it can). There are situations where Windows will not allow applications to take longer than 10-15 seconds to shutdown. In those cases Windows will just terminate the application. One specific case is during the process of installing patches. Another would be if the volunteer has Windows configured to shutdown when a notebook screen is closed. In those cases Windows terminates the processes. Nothing is spared. While BOINC itself is waiting on vboxwrapper to shutdown, it is terminated by the OS. Vboxwrapper is waiting on VirtualBox to write whatever the configured memory size is for the VM to disk. All the while every other application on the system is saving state and shutting down. That is basically why using the save state option is not the default configuration. Ideally using VM snapshots as regular checkpoints means that Vboxwrapper can restore the VM state to a condition where it can resume from a known stable state. It can even survive a power failure. Basically if you see "Stopping VM." instead of "Powering off VM." Vboxwrapper has issued the command to save state. Whether the OS lets BOINC/Vboxwrapper complete the task is anybody's guess. ----- Rom |
|
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 862,257 RAC: 8 |
That is basically why using the save state option is not the default configuration. Ideally using VM snapshots as regular checkpoints means that Vboxwrapper can restore the VM state to a condition where it can resume from a known stable state. It can even survive a power failure. Hello Rom, Long time not tested the use of snapshots, because the enable_vm_savestate is working very fine, except maybe for the uncontrolled system shutdown. Shutdown a host and knowing there are VM's running is like hours working on BOINC code and then push the power button. But for you I tested with a CMS-task the snapshot behavior and was surprised that cleaning previous snapshots wasn't working well any longer. I've several snapshots now. Suspending and resuming from one (hopefully the last) worked. <Machine uuid="{3a92780e-412d-4438-bc9a-176315d70e5f}" name="boinc_ff6509fb82028277" OSType="Linux26_64" currentSnapshot="{b802a470-9956-479a-a99c-dc96f46694b3}" snapshotFolder="Snapshots" lastStateChange="2016-03-04T10:50:00Z">
<Description>wu_1456935090_37_0</Description>
<MediaRegistry>
<HardDisks>
<HardDisk uuid="{dddcc611-854c-4100-a8bd-4d25610356d3}" location="D:/Boinc1/slots/1/vm_image.vdi" format="VDI" type="Normal">
<HardDisk uuid="{7209b04c-a787-4131-9044-a1aaf1b95ce6}" location="Snapshots/{7209b04c-a787-4131-9044-a1aaf1b95ce6}.vdi" format="VDI">
<HardDisk uuid="{fb464037-80fc-4477-8d0b-82798085f527}" location="Snapshots/{fb464037-80fc-4477-8d0b-82798085f527}.vdi" format="VDI">
<HardDisk uuid="{922e2fea-3fd9-41be-aa7a-35af218b948e}" location="Snapshots/{922e2fea-3fd9-41be-aa7a-35af218b948e}.vdi" format="VDI">
<HardDisk uuid="{80becbdb-18e7-4743-96eb-96ce089e0508}" location="Snapshots/{80becbdb-18e7-4743-96eb-96ce089e0508}.vdi" format="VDI">
<HardDisk uuid="{31fd37f5-1595-466e-aecc-3e06a45e922a}" location="Snapshots/{31fd37f5-1595-466e-aecc-3e06a45e922a}.vdi" format="VDI">
<HardDisk uuid="{5a0d2d13-b758-4981-9e2e-617e3afafcd9}" location="Snapshots/{5a0d2d13-b758-4981-9e2e-617e3afafcd9}.vdi" format="VDI"/>
</HardDisk>Finally the task ended in computation error due to missing heartbeat |
|
Laurence Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0 |
Please see if it is now faster. |
|
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
Not really, maybe a few seconds. Still just below 8 min Tested 13.10 UTC. |
|
Send message Joined: 17 Aug 15 Posts: 17 Credit: 228,358 RAC: 0
|
There are situations where Windows will not allow applications to take longer than 10-15 seconds to shutdown. In those cases Windows will just terminate the application. One specific case is during the process of installing patches. Another would be if the volunteer has Windows configured to shutdown when a notebook screen is closed. Hummm. I sometimes use JV16 Power Tools to shorten the delay before Windows (Win7 64-bit for me) closes down a hung application. Maybe the long delay in shutting down the VM is considered a "hung application"? I will avoid doing this on those machines running VBox. |
|
Send message Joined: 20 Mar 15 Posts: 243 Credit: 886,442 RAC: 0 |
There are situations where Windows will not allow applications to take longer than 10-15 seconds to shutdown. In those cases Windows will just terminate the application. One specific case is during the process of installing patches. Another would be if the volunteer has Windows configured to shutdown when a notebook screen is closed. This is dangerous territory and comes with many caveats, but for the adventurous there are (or were in WXP and I reckon there are equivalents on later systems) "WaitToKillAppTimeout" and "HungAppTimeout" registry keys. They're in "HKEY_CURRENT_USER/Control Panel/Desktop" and in "HKEY_USERS/.DEFAULT/Control Panel/Desktop" for all users. In "HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control" there is "WaitToKillServiceTimeout". All values are in mS. I've no idea if there are special conditions around updates or notebooks but they might give starting points for a bit of experimenting. The value on XP boxes around here for "WaitToKillAppTimeout" is 20s. Edit:- This comes from notes I've kept from way back, when the time taken to merge/delete snapshots was a problem. I didn't record their source, so a bit of creative Googling might be in order. |
|
Send message Joined: 17 Aug 15 Posts: 17 Credit: 228,358 RAC: 0
|
In "HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control" Here is some uncreative Googling. I think this is the right one for apps: http://www.eightforums.com/tutorials/37424-waittokillapptimeout-specify-shutdown-windows.html For services: http://www.makeuseof.com/tag/3-ways-speed-windows-7-shutdown-process/ I don't know if you need both, but it wouldn't hurt to set them both to a suitably high level. |
©2025 CERN