Message boards : CMS Application : Longer jobs
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5282 - Posted: 18 Dec 2017, 23:57:44 UTC

To ease my workload over the holiday season, I've gone back to the longer jobs we used to have -- 4,000 events/job rather than 2,500. (We'd reduced the event number while we checked if we could run many more jobs per batch than our original operation). So jobs will take correspondingly longer, but tasks should stay roughly the same; as I understand it, tasks finish when the totality of completed jobs in the task exceeds 12 hours, with a hard cut-off after 18 hours per task.
I don't think this should materially affect your running of tasks, except if you are in the habit of suspending computations frequently, where you might find that jobs are abandoned due to the VM being suspended too often.
ID: 5282 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 965
Credit: 1,201,500
RAC: 5
Message 5304 - Posted: 23 Dec 2017, 7:53:49 UTC
Last modified: 23 Dec 2017, 8:00:41 UTC

Are you sure, it has not reverted back to 2500 events/per job?
My last 3 tasks indicate that ,
(roughly tasks started on the 22nd Dec 0.00h UTC or later)
ID: 5304 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 8 Apr 15
Posts: 540
Credit: 7,616,583
RAC: 1,500
Message 5305 - Posted: 23 Dec 2017, 8:15:14 UTC

I just wish I could get more than 2 multi-core tasks on my 8-core pc's

Instead of just 2 of the 2-core tasks I would rather be able to get 4 of them so I didn't have to get other work for the other 4 cores.

Prefs are set at no limit
ID: 5305 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 432
Credit: 1,261,674
RAC: 37
Message 5308 - Posted: 23 Dec 2017, 8:56:59 UTC

Hi Magic,

my experience with Atlas is, when you set the preferences for example to 8 tasks, it take a while for Boinc, to use more downloads.
ID: 5308 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 8 Apr 15
Posts: 540
Credit: 7,616,583
RAC: 1,500
Message 5309 - Posted: 23 Dec 2017, 9:27:23 UTC - in response to Message 5308.  

Hi Magic,

my experience with Atlas is, when you set the preferences for example to 8 tasks, it take a while for Boinc, to use more downloads.


I have had my prefs set the same since multi-core started and still get no more than 2 of the 2-core tasks here.

Do you get more than that on an 8-core pc?
ID: 5309 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 432
Credit: 1,261,674
RAC: 37
Message 5310 - Posted: 23 Dec 2017, 10:45:46 UTC - in response to Message 5309.  

Do you get more than that on an 8-core pc?

Boinc had filled with 8 tasks (including the running tasks and the NOT uploaded tasks), when the prefs are set to 8 tasks.
The number of CPU's you are using in the tasks is ignored.
But... when tasks have a lot of work (running 24 hours for example), than the number of downloaded tasks was reduced.
It's a very dynamically process.
ID: 5310 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5311 - Posted: 23 Dec 2017, 12:24:41 UTC - in response to Message 5304.  

Are you sure, it has not reverted back to 2500 events/per job?
My last 3 tasks indicate that ,
(roughly tasks started on the 22nd Dec 0.00h UTC or later)

Yes, I accidentally typed the wrong command, ("source command" instead of "source command2") and submitted a batch to the old WMAgent. I've renamed command to command.old now so that won't happen again. :-)
ID: 5311 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 8 Apr 15
Posts: 540
Credit: 7,616,583
RAC: 1,500
Message 5312 - Posted: 23 Dec 2017, 15:33:14 UTC - in response to Message 5310.  

Do you get more than that on an 8-core pc?

Boinc had filled with 8 tasks (including the running tasks and the NOT uploaded tasks), when the prefs are set to 8 tasks.
The number of CPU's you are using in the tasks is ignored.
But... when tasks have a lot of work (running 24 hours for example), than the number of downloaded tasks was reduced.
It's a very dynamically process.


What I mean is can anyone get 4 of the 2-core multi-core tasks at the same time and have them running on an 8-core computer?

I can only get two of them which of course uses 4 cores........I want to get 4 of the 2-core tasks so I can run all 4 multi-core tasks at the same time which would run all 8 cores.

I can only get two of the multi-core tasks from the server so I have to get 4 single core tasks over at LHC just to get all 8 cores running.
ID: 5312 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1010
Credit: 591,653
RAC: 2
Message 5313 - Posted: 23 Dec 2017, 16:45:44 UTC - in response to Message 5312.  

What I mean is can anyone get 4 of the 2-core multi-core tasks at the same time and have them running on an 8-core computer?
I only get 2 CMS-tasks, doesn't matter whether set Max # jobs 4 and Max # CPUs 1 or Max # jobs 4 and Max # CPUs 2.
But when I set Max # jobs 4 and Max # CPUs 4 I got 4 tasks. ???
It Seems something is reversed.
So you can get 4 tasks and have to work with the app_config.xml for running 4 tasks dual core.
ID: 5313 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5314 - Posted: 24 Dec 2017, 10:24:51 UTC - in response to Message 5313.  

After your messages yesterday I realised that my cruncher at work was doing 2x2-core tasks while I had its preferences set to 3x2. I changed it to 5x2 but it's still only running 2x2. I'll play around with it a bit later; I just got it to start two LHC@home tasks now that we have tasks in the queue there again, so I'm loth to interfere with it for a while.
ID: 5314 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5315 - Posted: 24 Dec 2017, 14:52:20 UTC - in response to Message 5314.  

Right, I just suspended SETI@Home on my 20-core server, leaving 2x2-core -dev tasks running, and 2x (1 core, of course) LHC@home/CMS tasks as well. When I requested more -dev tasks the response was "not needed". Any thoughts? I think there's an option somewhere to more explicitly spell out BOINC's decisions; I'll look into that later (now's not really the time to be doing such research...).
ID: 5315 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1010
Credit: 591,653
RAC: 2
Message 5316 - Posted: 24 Dec 2017, 20:09:43 UTC - in response to Message 5315.  

Any thoughts? I think there's an option somewhere to more explicitly spell out BOINC's decisions; I'll look into that later (now's not really the time to be doing such research...).

With <work_fetch_debug>1</work_fetch_debug> in the log_flags part of the cc_config.xml you get more info about what is requested.

After editing and saving the cc_config.xml, just select 'Read config files' from the options menu in BOINC Manager.
ID: 5316 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5317 - Posted: 26 Dec 2017, 12:54:03 UTC - in response to Message 5316.  

Cheers, CP, I'll look into that later. Meanwhile, I overslept and let the batch queue run out of jobs, and/or the WMAgent failed. The only WMAgent expert on my list who isn't currently on holidays is at Fermilab, so it may be a few hours before he is able to respond to my alert.
ID: 5317 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5318 - Posted: 26 Dec 2017, 18:25:31 UTC

OK, no response from anyone with the power to restart the WMgent, so advice as usual, set No New Tasks.
ID: 5318 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5319 - Posted: 27 Dec 2017, 6:04:51 UTC

Starting to get some response now, but the time-zones are still working against us. :-(
ID: 5319 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1093
Credit: 6,893,316
RAC: 0
Message 5320 - Posted: 27 Dec 2017, 15:46:22 UTC - in response to Message 5319.  
Last modified: 27 Dec 2017, 15:46:48 UTC

OK, we seem to have jobs again. For some reason, I'm totally fagged out. I'll be hitting the hay soon, but maybe surfacing again for the third day of the Melbourne Test later tonight! :-)
ID: 5320 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : CMS Application : Longer jobs


©2020 CERN