Message boards : Number crunching : Test task
Message board moderation

To post messages, you must log in.

AuthorMessage
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3841 - Posted: 28 Jul 2016, 18:43:55 UTC

I suggest to implement a tick box in the preferences to enable a test task.

This would consist of a carefully chosen job, and only one job(same on multi-core tasks).
(as a precaution, this could automatically switch to "normal" after a period e.g 24h)

As all normal jobs have different length, they cannot be compared.

Purpose would be, to test different configurations for best speed.
Comments?
ID: 3841 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1064
Credit: 319,556
RAC: 73
Message 3842 - Posted: 28 Jul 2016, 19:09:34 UTC - in response to Message 3841.  
Last modified: 28 Jul 2016, 19:19:56 UTC

Here at CERN we have done quite some investigation into benchmarking VMs in the context of cloud procurement. We do have a benchmarking VM and I could see if we could use it here. The best thing would be to create a benchmarking app.

EDIT: Here is a public presentation on the topic.
ID: 3842 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toby Broom

Send message
Joined: 19 Aug 15
Posts: 46
Credit: 3,543,612
RAC: 700
Message 3843 - Posted: 28 Jul 2016, 19:16:51 UTC

Could you run LINPACK, I assume this would be a relevant test in the context of these Jobs
ID: 3843 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 3844 - Posted: 28 Jul 2016, 19:18:07 UTC - in response to Message 3842.  

Thanks for the reply. I just wanted it to be as close to the "real" thing as possible, including server response and networking.

It would also be interesting, what the bare minimum memory is to run a job, even with a lot of swap usage.( swap could be in RAM and shared between jobs, therefore no slowdown)
ID: 3844 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toby Broom

Send message
Joined: 19 Aug 15
Posts: 46
Credit: 3,543,612
RAC: 700
Message 3859 - Posted: 29 Jul 2016, 14:53:45 UTC

The people that wrote LINPACK developed a more realistic benchmark that it more I/O bound, HPCG benchmark.

I was think that if you took an off the self benchmark could save some time for other project work. I agree if the results are meaning less then it not really saved effort.
ID: 3859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1064
Credit: 319,556
RAC: 73
Message 3867 - Posted: 29 Jul 2016, 21:06:13 UTC - in response to Message 3859.  

Eric McIntosh, who you my know from Sixtrack, was benchmarking mainframes for procurement many years ago and we discussed the topic recently in the context of cloud procurement. One of the things I took away from the discussion was that you have to benchmark with your own application so you are sure that it touches all the same things by the same amounts. The trick is to find a reference workload that is a good representation of what you actually do. As a result we do have a standard workload for benchmarking and a framework to run it. I will try to add it as an application in this project next week.
ID: 3867 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1064
Credit: 319,556
RAC: 73
Message 3962 - Posted: 5 Aug 2016, 13:03:15 UTC - in response to Message 3867.  
Last modified: 5 Aug 2016, 13:03:25 UTC

As promised, the benchmarking application is now available :)
ID: 3962 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Test task


©2024 CERN