Development for LHC@home

I suppose, Ivan showed the top command from the host and not from a guest-VM.

Indeed. Notice the 8-core VBoxHeadless processes running at ~700% CPU utilisation; they will count as one process to the host.

Still, strange.
And all processes are at the lowest possible priority, which i find strange, but i do not know linux very well.

BOINC usually runs tasks at low priority in my experience; it may be a per-project setting.

---

It's working now somehow without using an app_config.xml, no work cache and asking only tasks for the benchmark application.

Max # jobs 1
Max # CPUs 1 - I get 1 task for 1 single core VM

Max # jobs 2
Max # CPUs 1 - I get 1 task for 1 single core VM

Max # jobs 2
Max # CPUs 2 - I get 2 tasks for 2 dual core VM's

Max # jobs 3
Max # CPUs 2 - I get 2 tasks for 2 dual core VM's

Max # jobs 2
Max # CPUs 3 - I get 2 tasks for 2 three core VM's

Expecting with 8 available threads 1 task more, but with the red figures I get "This computer has reached a limit on tasks in progress".
Also with last setting the limit message and asking for over 7 hours of work.

It looks like Karolina has been tinkering with the scheduler code today and made some improvements.

Forgot to mention:

Max # jobs 1
Max # CPUs 2 - I get as expected 1 task for 1 dual core VM

All VM's are created with 2300 MB Memory.

Hi CP,

We have made a change to the scheduler. Please can run your tests again.

Hi CP,

We have made a change to the scheduler. Please can run your tests again.

I'll repeat the tests after 2 CMS-jobs of the dev-project and 1 multi-core Theory (3 jobs) of the production project have finished.

Testing with the Benchmark Application without using an app_config.xml on my Windows i7 2600 with 16GB RAM and hyper-threading on (8 threads)

			All VM's with 2300 MB base memory	Remark:
Max # jobs	1		
Max # CPUs	1	- I get 1 task for 1 single core VM	

Max # jobs	1		
Max # CPUs	2	- I get 1 task for 1 dual core VM	

Max # jobs	2		
Max # CPUs	1	- I get 2 tasks for 2 single core VM's	With 1 minute interval

Max # jobs	2		
Max # CPUs	2	- I get 2 tasks for 2 dual core VM's	

Max # jobs	1		
Max # CPUs	3	- I get 1 task for 1 three core VM	When athena's > 3 min: all swap used

Max # jobs	3		
Max # CPUs	1	- I get 3 tasks for 3 single core VM's	With 2 x 1 minute interval

Max # jobs	3		
Max # CPUs	2	- I get 3 tasks for 3 dual core VM's	First 2 then 1 min. later #3

Max # jobs	2		
Max # CPUs	3	- I get 2 tasks for 2 three core VM's	2 at once. When athena's > 3 min: all swap used

Max # jobs	3		
Max # CPUs	3	- I get 3 tasks for 3 three core VM's	(faked ncpus); 3 at once. When athena's > 3 min: all swap used

Max # jobs	1		
Max # CPUs	4	- I get 1 task for 1 four core VM	When 4 athena's > 2 min: all swap used (1 athena swapped)

Max # jobs	2		
Max # CPUs	4	- I get 2 tasks for 2 four core VM's	2 at once. When 4 athena's > 2 min: all swap used (1 athena swapped)

Max # jobs	4		
Max # CPUs	1	- I get 4 tasks for 4 single core VM's	With 1 minute interval each

Max # jobs	4		
Max # CPUs	2	- I get 4 tasks for 4 dual core VM's	2 at once. 1 minute later 2 others.

Max # jobs	8		
Max # CPUs	1	- I get 8 tasks for 8 single core VM's	With 1 minute interval each. (hard to run on my 16GB host with 2300MB VM's)

With app_config I set 1408 MB for the single core benchmark app. For 3 jobs in 1 VM memory is too low, but that was not the reason for this test.

Conclusion for the mentioned set up mixtures for # of Jobs and/or CPU's: It's functioning very well.

Test results:
http://lhcathomedev.cern.ch/vLHCathome-dev/results.php?hostid=37&offset=0&show_names=0&state=0&appid=8
http://lhcathomedev.cern.ch/vLHCathome-dev/results.php?hostid=37&offset=20&show_names=0&state=0&appid=8

Thanks for the thorough test. We know what line needs updating to fix the 1 minute interval so will do that ASAP. The memory issue is due to scaling not being implemented which is not released to the scheduler or project preferences.

fix the 1 minute interval

I think, that is not bad, as is.
We do not want too many vm's starting up at the same time, do we?

The scheduler has been updated. There should now be no interval.

I have just tested it myself and it works as expected.

seems good to me as well.

I discovered a new issue. With the settings

Max # jobs No limit
Max # CPUs 1

I only get 1 task - "vLHCathome-dev 14 Aug 09:34:07 This computer has reached a limit on tasks in progress"

When settings set to

Max # jobs 8
Max # CPUs 1

I get 8 tasks for 8 single core VM's

With 'No limit' I expect to get at least as much tasks as available threads.

When setting
Max # jobs No limit
Max # CPUs 2

I get 2 tasks.

I have just updated the scheduler with that latest commit from David Anderson rather than our changes. Please check if this now works for you.

I have selected:

Max # jobs 4
Max # CPUs 1


getting the message:

8/22/2016 11:08:44 AM | vLHCathome-dev | No tasks are available for Theory Simulation
8/22/2016 11:08:44 AM | vLHCathome-dev | This computer has reached a limit on tasks in progress

When trying to start new tasks.Two still running.

I have just updated the scheduler with that latest commit from David Anderson rather than our changes. Please check if this now works for you.

It does not work.Please revert back to what was before.

With settings

Max # jobs 3 or 4
Max # CPUs 1

I get only 2 WUs (1 core).

If I request more WUs I get the message:
"This computer has reached a limit on tasks in progress"

Max # jobs No limit
Max # CPUs 1

I'm getting only 1 task. When requesting more I get

"This computer has reached a limit on tasks in progress"

With setting that was working correctly before

Max # jobs 8
Max # CPUs 1

the same: only 1 task and limit notification.
Before the change I got 8 single core VM's.

With settings

Max # jobs 3 or 4
Max # CPUs 1

I get only 2 WUs (1 core).

If I request more WUs I get the message:
"This computer has reached a limit on tasks in progress"

With the settings you mentioned I only get 1 task.

Only when increasing "Max # CPUs" to 2, I get 2 tasks.

Laurence, could you pse restore the old settings. This is not an improvement, but a degeneration.

Great project.
First a whole week of nothing, then a change, that makes things worse and then no reply to requests of restoring the better condition.

One might wonder if admin really gives a s@#~.