Message boards : Theory Application : New Theory App - Linux Only
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5784 - Posted: 6 Feb 2019, 14:35:13 UTC
Last modified: 6 Feb 2019, 16:15:21 UTC

As discussed in this thread, I have created a container-based version of the Theory app using runc and CVMFS. For now this is Linux only and requires CVMFS to be installed as a prerequisite similar to the ATLAS native application. As one of the main differences is that the real jobs are pushed through the BOINC server, we can not provide both the old style VM jobs and new container jobs for the same application. In order to test we would need to do one of the following:

  1. Use the current Theory app (Linux Only for now)
  2. Create a new app (TheoryNative)


Which direction would you prefer and why?

ID: 5784 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1188
Credit: 861,609
RAC: 15
Message 5785 - Posted: 6 Feb 2019, 15:53:50 UTC - in response to Message 5784.  

75% of the hosts are running Windows. Why are all their users ignored?

Couldn't you create a VM with your wished container configuration and other prerequisites, so that it can be used by all operating systems?
ID: 5785 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 28 Jul 16
Posts: 484
Credit: 394,839
RAC: 0
Message 5786 - Posted: 6 Feb 2019, 15:58:56 UTC - in response to Message 5784.  

Hi Laurence


Some questions:

Do you refer to this thread?


Would (2.) also satisfy Windows users?
If YES, then this should be preferred, but maybe I misunderstand something.


My local CVMFS is currently configured to serve ATLAS repositories.
What changes would be necessary to also run Theory?
ID: 5786 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5787 - Posted: 6 Feb 2019, 16:19:21 UTC - in response to Message 5785.  

75% of the hosts are running Windows. Why are all their users ignored?

Couldn't you create a VM with your wished container configuration and other prerequisites, so that it can be used by all operating systems?


They are not being ignored. Making the container work natively is the first step. The second is making that work in a VM. In the short term while testing, Windows hosts can be used by running the BOINC client in a Linux VM.
ID: 5787 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5788 - Posted: 6 Feb 2019, 16:21:03 UTC - in response to Message 5786.  

Hi Laurence


Some questions:

Do you refer to this thread?

Thanks, the last '2' was missed from the copy and paste.


Would (2.) also satisfy Windows users?
If YES, then this should be preferred, but maybe I misunderstand something.

No, we would have to address Windows users later.


My local CVMFS is currently configured to serve ATLAS repositories.
What changes would be necessary to also run Theory?

You would have to ensure that CVMFS_REPOSITORIES in default.local contains grid.cern.ch and sft.cern.ch
ID: 5788 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 28 Jul 16
Posts: 484
Credit: 394,839
RAC: 0
Message 5789 - Posted: 6 Feb 2019, 17:00:26 UTC - in response to Message 5784.  

To be honest, I don't understand what's different between (1.) and (2.) if both run on linux.
Could you give some hints?
ID: 5789 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1188
Credit: 861,609
RAC: 15
Message 5790 - Posted: 6 Feb 2019, 17:16:23 UTC - in response to Message 5787.  

In the short term while testing, Windows hosts can be used by running the BOINC client in a Linux VM.

I would like to do that, but I did not manage to get ATLAS native running in my Linux Mint VM running on my Win7 host.
ID: 5790 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 677
Credit: 2,002,766
RAC: 1
Message 5791 - Posted: 6 Feb 2019, 21:02:30 UTC - in response to Message 5784.  
Last modified: 6 Feb 2019, 21:50:07 UTC

Use the current Theory app (Linux Only for now)
Create a new app (TheoryNative)
Which direction would you prefer and why?

Hi Laurence,
in Atlas, the test-function must be activated to get native.
Problem is, the native-App is not seeing in the Boinc-Server-stats.

Otherwhise the same Linux-App as before would running with this stats. No change of the prefs is needed.
Is this native-App reduced for SL69 and CentOS7 normally?
All Hypervisor-also Hyper-V?
ID: 5791 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5792 - Posted: 7 Feb 2019, 8:34:27 UTC - in response to Message 5789.  

To be honest, I don't understand what's different between (1.) and (2.) if both run on linux.
Could you give some hints?


The main difference is with inputs/outputs, job length and credit. With (1) the job is the VM, no input/no output and jobs run for ~13 hours. With (2) there is an input and output file and the jobs run for ~90 mins.
ID: 5792 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5793 - Posted: 7 Feb 2019, 8:36:17 UTC - in response to Message 5791.  


Hi Laurence,
in Atlas, the test-function must be activated to get native.
Problem is, the native-App is not seeing in the Boinc-Server-stats.

Otherwhise the same Linux-App as before would running with this stats. No change of the prefs is needed.
Is this native-App reduced for SL69 and CentOS7 normally?
All Hypervisor-also Hyper-V?


The ATLAS VM and Native apps are the same. They both use the same input file and produce the same outputs.
ID: 5793 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5794 - Posted: 7 Feb 2019, 8:51:59 UTC - in response to Message 5793.  

I will drain the queue of VM tasks before creating the native ones.
ID: 5794 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 28 Jul 16
Posts: 484
Credit: 394,839
RAC: 0
Message 5795 - Posted: 7 Feb 2019, 8:52:26 UTC - in response to Message 5792.  

To be honest, I don't understand what's different between (1.) and (2.) if both run on linux.
Could you give some hints?


The main difference is with inputs/outputs, job length and credit. With (1) the job is the VM, no input/no output and jobs run for ~13 hours. With (2) there is an input and output file and the jobs run for ~90 mins.

90 min seems to be a good runtime lenght.
It could also be good to monitor each job individually to identify app failures (see sherpa).

Thus I would prefer (2.).


The disadvantage is that the BOINC server would have more work but lots of short SixTracks show that this may not be a problem.
ID: 5795 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5796 - Posted: 7 Feb 2019, 12:10:14 UTC - in response to Message 5795.  
Last modified: 7 Feb 2019, 12:21:28 UTC

I have reconfigured the Theory app to use the native Linux version and the first task was successful.
ID: 5796 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 28 Jul 16
Posts: 484
Credit: 394,839
RAC: 0
Message 5797 - Posted: 7 Feb 2019, 12:19:04 UTC - in response to Message 5796.  

To visit the link a CERN account is required.
ID: 5797 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1069
Credit: 334,882
RAC: 0
Message 5799 - Posted: 7 Feb 2019, 12:21:42 UTC - in response to Message 5797.  

I have changed the link. Please try again.
ID: 5799 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Theory Application : New Theory App - Linux Only


©2024 CERN