Message boards : Theory Application : New Version (v0.5)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1067
Credit: 329,589
RAC: 107
Message 3103 - Posted: 29 Apr 2016, 11:27:31 UTC

This version has a slight modification to the CVMFS configuration so that it will receive a list of proxies and choose the best. For now there should be no noticeable change as we only have one proxy. The memory required has also been changed to 512MB. Please comment on how the swap looks.
ID: 3103 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1185
Credit: 849,977
RAC: 1,466
Message 3123 - Posted: 30 Apr 2016, 14:14:03 UTC

I noticed since this version that some extra info is added to Console F4 (stdout.log)

12:58:46 +0200 2016-04-30 [INFO] MCPlots JobID: 29883509
13:54:28 +0200 2016-04-30 [INFO] MCPlots JobID: 29883036
14:07:11 +0200 2016-04-30 [INFO] MCPlots JobID: 29882150
14:31:11 +0200 2016-04-30 [INFO] MCPlots JobID: 29849866


Any idea if these JobID's can be accessed publicly from the MCPLOTs database?
ID: 3123 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3124 - Posted: 30 Apr 2016, 15:15:32 UTC

Please comment on how the swap looks.


Highest swap use , i have seen (with 512MB RAM) was about 70MB.
I suggest to increase RAM allocation by 128MB to 630MB.
ID: 3124 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1067
Credit: 329,589
RAC: 107
Message 3125 - Posted: 30 Apr 2016, 17:26:23 UTC - in response to Message 3124.  
Last modified: 30 Apr 2016, 17:26:46 UTC

A new version (v0.6) is now available with the memory set to 630MB.
ID: 3125 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Ray Murray
Avatar

Send message
Joined: 13 Apr 15
Posts: 138
Credit: 2,945,852
RAC: 0
Message 3126 - Posted: 30 Apr 2016, 17:45:45 UTC - in response to Message 3123.  
Last modified: 30 Apr 2016, 17:46:33 UTC

Job IDs going to McPlots might be one of the steps along this line
The next step is to feed the results back into MCPlots, which Leonardo is doing currently, so you will start to get MCPlots stats updates for the work you do.
From Ben, a couple of weeks ago.
ID: 3126 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1185
Credit: 849,977
RAC: 1,466
Message 3129 - Posted: 30 Apr 2016, 19:23:29 UTC - in response to Message 3124.  

Please comment on how the swap looks.

Swap is used, but not that heavy, but memory is used all the time > 95%, so it's indeed better to increase the base memory a bit.
btw 512 + 128 = 640 and you can increase/decrease the <memory_size_mb> yourself by editing the Theory xml-file with <dont_check_file_sizes> set true.
ID: 3129 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3130 - Posted: 30 Apr 2016, 19:40:26 UTC - in response to Message 3129.  

You are correct.
My math sucks.
ID: 3130 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1185
Credit: 849,977
RAC: 1,466
Message 3132 - Posted: 30 Apr 2016, 21:43:10 UTC - in response to Message 3125.  

A new version (v0.6) is now available with the memory set to 630MB.

Extended it myself to 640MB
When running a Pythia6 job, 562k (sometimes up to 579k) memory used and 3k swap.
ID: 3132 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3133 - Posted: 30 Apr 2016, 21:54:52 UTC

I have had occasions, where even with 1280MB of RAM allocated, it was still using a bit of swap.

I understand, that the swap used is kind of a "max hold" function.Once it uses that much, it will not go back down, again.

Just now, with 630MB of RAM, it stll has jobs , where the swap goes up to 42MB.
ID: 3133 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1185
Credit: 849,977
RAC: 1,466
Message 3135 - Posted: 1 May 2016, 7:11:53 UTC - in response to Message 3133.  
Last modified: 1 May 2016, 7:44:53 UTC

..., where the swap goes up to 42MB.

Same here. 2 Theory VM's running with 38 and 40 MB used swap, but they also have 75MB and 140MB memory free (Pythia6 and Pythia8 atm)
I too noticed that used swap never goes down and that's logical, because it's a file using already that space and the file(header) will be created new after a fresh boot.
I think most of the swap is used during the initial startup, so no need to reserve memory for that during the whole VM-lifetime.
I'll reduce the memory size to 576MB for testing.

Edit: I have also 4 CERN Challenge VM's (T4T-64) running configured with 512MB.
On average they have 60MB memory free and 76MB swap space used.
btw: Those VM's have 2537MB allocated for the swapfile. A bit overdone, I think.
ID: 3135 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3149 - Posted: 1 May 2016, 11:44:53 UTC

I noticed since this version that some extra info is added to Console F4 (stdout.log)

12:58:46 +0200 2016-04-30 [INFO] MCPlots JobID: 29883509
13:54:28 +0200 2016-04-30 [INFO] MCPlots JobID: 29883036
14:07:11 +0200 2016-04-30 [INFO] MCPlots JobID: 29882150
14:31:11 +0200 2016-04-30 [INFO] MCPlots JobID: 29849866

Any idea if these JobID's can be accessed publicly from the MCPLOTs database?


Not present any more, not in the logs or console F4.
ID: 3149 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1185
Credit: 849,977
RAC: 1,466
Message 3158 - Posted: 1 May 2016, 13:53:45 UTC - in response to Message 3135.  

I'll reduce the memory size to 576MB for testing.

2 VM's running with this assigned memory for 5 and 3 hours.
The maximum of one time used swap is 108MB.
During the normal events processing there is about 60-65MB memory free, but every now and then plotter.exe is jumping in and needs about 15MB of memory.
ID: 3158 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Theory Application : New Version (v0.5)


©2024 CERN