Message boards :
News :
Task and CPU limiter
Message board moderation
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Send message Joined: 20 Jan 15 Posts: 1129 Credit: 7,973,351 RAC: 2,540 |
Indeed. Notice the 8-core VBoxHeadless processes running at ~700% CPU utilisation; they will count as one process to the host.I suppose, Ivan showed the top command from the host and not from a guest-VM. Still, strange. BOINC usually runs tasks at low priority in my experience; it may be a per-project setting. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
It's working now somehow without using an app_config.xml, no work cache and asking only tasks for the benchmark application. Max # jobs 1 Max # CPUs 1 - I get 1 task for 1 single core VM Max # jobs 2 Max # CPUs 1 - I get 1 task for 1 single core VM Max # jobs 2 Max # CPUs 2 - I get 2 tasks for 2 dual core VM's Max # jobs 3 Max # CPUs 2 - I get 2 tasks for 2 dual core VM's Max # jobs 2 Max # CPUs 3 - I get 2 tasks for 2 three core VM's Expecting with 8 available threads 1 task more, but with the red figures I get "This computer has reached a limit on tasks in progress". Also with last setting the limit message and asking for over 7 hours of work. |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
It looks like Karolina has been tinkering with the scheduler code today and made some improvements. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
Forgot to mention: Max # jobs 1 Max # CPUs 2 - I get as expected 1 task for 1 dual core VM All VM's are created with 2300 MB Memory. |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
Hi CP, We have made a change to the scheduler. Please can run your tests again. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
Hi CP, I'll repeat the tests after 2 CMS-jobs of the dev-project and 1 multi-core Theory (3 jobs) of the production project have finished. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
Testing with the Benchmark Application without using an app_config.xml on my Windows i7 2600 with 16GB RAM and hyper-threading on (8 threads) All VM's with 2300 MB base memory Remark: Max # jobs 1 Max # CPUs 1 - I get 1 task for 1 single core VM Max # jobs 1 Max # CPUs 2 - I get 1 task for 1 dual core VM Max # jobs 2 Max # CPUs 1 - I get 2 tasks for 2 single core VM's With 1 minute interval Max # jobs 2 Max # CPUs 2 - I get 2 tasks for 2 dual core VM's Max # jobs 1 Max # CPUs 3 - I get 1 task for 1 three core VM When athena's > 3 min: all swap used Max # jobs 3 Max # CPUs 1 - I get 3 tasks for 3 single core VM's With 2 x 1 minute interval Max # jobs 3 Max # CPUs 2 - I get 3 tasks for 3 dual core VM's First 2 then 1 min. later #3 Max # jobs 2 Max # CPUs 3 - I get 2 tasks for 2 three core VM's 2 at once. When athena's > 3 min: all swap used Max # jobs 3 Max # CPUs 3 - I get 3 tasks for 3 three core VM's (faked ncpus); 3 at once. When athena's > 3 min: all swap used Max # jobs 1 Max # CPUs 4 - I get 1 task for 1 four core VM When 4 athena's > 2 min: all swap used (1 athena swapped) Max # jobs 2 Max # CPUs 4 - I get 2 tasks for 2 four core VM's 2 at once. When 4 athena's > 2 min: all swap used (1 athena swapped) Max # jobs 4 Max # CPUs 1 - I get 4 tasks for 4 single core VM's With 1 minute interval each Max # jobs 4 Max # CPUs 2 - I get 4 tasks for 4 dual core VM's 2 at once. 1 minute later 2 others. Max # jobs 8 Max # CPUs 1 - I get 8 tasks for 8 single core VM's With 1 minute interval each. (hard to run on my 16GB host with 2300MB VM's) With app_config I set 1408 MB for the single core benchmark app. For 3 jobs in 1 VM memory is too low, but that was not the reason for this test. Conclusion for the mentioned set up mixtures for # of Jobs and/or CPU's: It's functioning very well. Test results: http://lhcathomedev.cern.ch/vLHCathome-dev/results.php?hostid=37&offset=0&show_names=0&state=0&appid=8 http://lhcathomedev.cern.ch/vLHCathome-dev/results.php?hostid=37&offset=20&show_names=0&state=0&appid=8 |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
Thanks for the thorough test. We know what line needs updating to fix the 1 minute interval so will do that ASAP. The memory issue is due to scaling not being implemented which is not released to the scheduler or project preferences. |
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
fix the 1 minute interval I think, that is not bad, as is. We do not want too many vm's starting up at the same time, do we? |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
The scheduler has been updated. There should now be no interval. |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
I have just tested it myself and it works as expected. |
Send message Joined: 19 Aug 15 Posts: 46 Credit: 3,572,545 RAC: 432 |
seems good to me as well. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
I discovered a new issue. With the settings Max # jobs No limit Max # CPUs 1 I only get 1 task - "vLHCathome-dev 14 Aug 09:34:07 This computer has reached a limit on tasks in progress" When settings set to Max # jobs 8 Max # CPUs 1 I get 8 tasks for 8 single core VM's With 'No limit' I expect to get at least as much tasks as available threads. When setting Max # jobs No limit Max # CPUs 2 I get 2 tasks. |
Send message Joined: 12 Sep 14 Posts: 1067 Credit: 330,774 RAC: 138 |
I have just updated the scheduler with that latest commit from David Anderson rather than our changes. Please check if this now works for you. |
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
I have selected: Max # jobs 4 Max # CPUs 1 getting the message: 8/22/2016 11:08:44 AM | vLHCathome-dev | No tasks are available for Theory Simulation 8/22/2016 11:08:44 AM | vLHCathome-dev | This computer has reached a limit on tasks in progress When trying to start new tasks.Two still running. |
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
I have just updated the scheduler with that latest commit from David Anderson rather than our changes. Please check if this now works for you. It does not work.Please revert back to what was before. |
Send message Joined: 28 Jul 16 Posts: 478 Credit: 394,720 RAC: 79 |
With settings Max # jobs 3 or 4 Max # CPUs 1 I get only 2 WUs (1 core). If I request more WUs I get the message: "This computer has reached a limit on tasks in progress" |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
Max # jobs No limit Max # CPUs 1 I'm getting only 1 task. When requesting more I get "This computer has reached a limit on tasks in progress" With setting that was working correctly before Max # jobs 8 Max # CPUs 1 the same: only 1 task and limit notification. Before the change I got 8 single core VM's. |
Send message Joined: 13 Feb 15 Posts: 1185 Credit: 850,198 RAC: 510 |
With settings With the settings you mentioned I only get 1 task. Only when increasing "Max # CPUs" to 2, I get 2 tasks. Laurence, could you pse restore the old settings. This is not an improvement, but a degeneration. |
Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0 |
Great project. First a whole week of nothing, then a change, that makes things worse and then no reply to requests of restoring the better condition. One might wonder if admin really gives a s@#~. |
©2024 CERN