Message boards : Number crunching : Wish list of improvements for the project
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1126
Credit: 7,861,186
RAC: 5
Message 958 - Posted: 1 Sep 2015, 19:25:13 UTC

Please use this thread to tell us what improvements you would like; we all know some already and some more have just been suggested tonight in a News topic.
Please keep the discussion to a minimum here, so we can quickly run through the list; if discussion seems desirable, start another thread referencing your idea here.
ID: 958 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yeti
Avatar

Send message
Joined: 29 May 15
Posts: 147
Credit: 2,842,484
RAC: 0
Message 961 - Posted: 1 Sep 2015, 19:34:18 UTC - in response to Message 958.  

Set up the VM similar to upcoming CERN Challenge of this year.

The benefits in short are:

Crunch up to 4 Jobs within one VM
One Task is responsable for up- and downloading
The number of Cores and the Amount of Memory could be adjusted by the user
ID: 961 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 962 - Posted: 1 Sep 2015, 19:34:56 UTC

Is there a way to improve efficiency?
Currently the vboxheadless process has only an overall utilization of 88%.(24h)
This includes a lot of overhead. (atlas is in the mid 90%)

Would it be possible to have the "uploading" and "downloading" running in the background and at the same time the cpu crunching ?(especially useful, if network speed is not that high)
There seems to be a lot of fruitless cpu idling going on.

According to my conservative estimates, it should be possible to double the output.
ID: 962 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yeti
Avatar

Send message
Joined: 29 May 15
Posts: 147
Credit: 2,842,484
RAC: 0
Message 963 - Posted: 1 Sep 2015, 19:36:29 UTC

Communicate from inside the VM with the BOINC-Job:

If the BOINC-Job has only a short rest of runtime and the Job inside the VM has just finished, let the BOINC-Job end and start a new one
ID: 963 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yeti
Avatar

Send message
Joined: 29 May 15
Posts: 147
Credit: 2,842,484
RAC: 0
Message 964 - Posted: 1 Sep 2015, 19:39:18 UTC

If inside the VM a Job is already downloaded, don't throw it away if the VM is exited.

If the VM is restarted, check if relevant data is already there and can be used
ID: 964 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yeti
Avatar

Send message
Joined: 29 May 15
Posts: 147
Credit: 2,842,484
RAC: 0
Message 967 - Posted: 1 Sep 2015, 19:44:45 UTC

Don't finish a BOINC-Job after 24 hours only because time is over. Wait for the Job inside to finish first
ID: 967 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yeti
Avatar

Send message
Joined: 29 May 15
Posts: 147
Credit: 2,842,484
RAC: 0
Message 968 - Posted: 1 Sep 2015, 19:46:41 UTC

From the News thread:

Should be easy. We can add a button in the graphics web app that sets a shutdown signal. When the job finishes it will shutdown the VM rather than run another job.

Yeah, I would love this
ID: 968 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 969 - Posted: 1 Sep 2015, 20:06:52 UTC

Maybe it would be better to not up/download too much in one lot.
Smaller chunks,more often would be better.(less interference with other up/download activities).
Especially, if the are multiple task transferring.
ID: 969 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 978 - Posted: 3 Sep 2015, 9:24:13 UTC

I think it is much better to download, crunch, upload the traditional way than having to constantly be connected to the internet.(even if the files are large)

You can have very short deadlines and/or tasks but it is a much more reliable way.
ID: 978 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 1044 - Posted: 8 Sep 2015, 15:03:15 UTC

Most people have a higher down than upload rate.

I have 10Mbits/s down and 1Mbits/s up.
Would it be possible to compress the upload data to save upload time?

Better yet, to continue crunching while up or downloading(as described earlier in this thread)
ID: 1044 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 8 Apr 15
Posts: 722
Credit: 11,498,298
RAC: 1,105
Message 1053 - Posted: 8 Sep 2015, 19:06:23 UTC

Send me a free round trip to Geneva and a tour of Cern and a week at the hotel of my wifes choice (I'll leave my computers running while I am gone)

Thanks
Mad Scientist For Life
ID: 1053 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1126
Credit: 7,861,186
RAC: 5
Message 1058 - Posted: 8 Sep 2015, 23:00:29 UTC - in response to Message 1053.  

Send me a free round trip to Geneva and a tour of Cern and a week at the hotel of my wifes choice (I'll leave my computers running while I am gone)

Thanks

Don't laugh, TPTB in the background are mooting "rewards", with the ultimate being a trip to CERN. But, as always, don't hold your breath. (The kerfuffle from ten days ago isn't resolved yet, and won't be until at least Friday next week; I'm keeping the project in maintenance mode to keep the faith, it may collapse if I lose support. And I'm just a DFV -- look that acronym up on uk.rec.motorcycles!)
ID: 1058 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Wish list of improvements for the project


©2024 CERN