Message boards :
News :
New native Linux ATLAS application
Message board moderation
Author | Message |
---|---|
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
Hi all, If you don't use Linux you can ignore the rest of this post. If you do you may be interested in trying the experimental ATLAS app which doesn't use virtualbox but runs natively on Linux. IMPORTANT!! To run this app you must install CVMFS, the CERN VM File System, and configure it for ATLAS. This file system contains all the software for ATLAS WU and is normally inside the virtual image (the same as for all LHC vbox apps). A simple installation guide can be found here: https://cernvm.cern.ch/portal/filesystem/quickstart You should set up the repositories as shown in the example for ATLAS. If you have a squid proxy handy you can specify it there - if not I'm not sure whether it will work or not without configuring one. Our target for this app is CERN or ATLAS-related institutes who have idle machines with CVMFS already installed, and we do not expect the average volunteer to install CVMFS and run this app. But I think all of you here are above-average volunteers :) and you may be interested in trying it. Please give feedback on the ATLAS forums. Unfortunately there is no way to check for CVMFS on the client before requesting tasks, so if you don't have CVMFS you can still get these tasks and they will fail straight away. So better to uncheck the ATLAS app if you don't want to run it. |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
Ubuntu 14.04, 16.04 - packages not found sudo apt-get install cvmfs cvmfs-config-default [edit] instructions found wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest_all.deb |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
Unfortunately there is no way to check for CVMFS on the client before requesting tasks, so if you don't have CVMFS you can still get these tasks and they will fail straight away. So better to uncheck the ATLAS app if you don't want to run it. without packages - lhcathome-dev 23.02.2017 19:00:21 No tasks are available for ATLAS Simulation lhcathome-dev 23.02.2017 19:00:21 VirtualBox is not installed |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
Unfortunately cvmfs is not yet in the standard Linux repos so you have to set up a separate repo. without packages - That is very strange, are you sure this message is for ATLAS? I have tested this app successfully on a machine without virtualbox. |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
That is very strange, are you sure this message is for ATLAS? I have tested this app successfully on a machine without virtualbox. Yes, for the Atlas. Only one application is allowed in the settings. Packages installed (including SQUID), configured and tested (CVMFS probe) Error message remained the same (no tasks + no VirtualBox ) Now, however, they do not really have on the server. I am waiting for new work Are there any additional requirements for the host for these tasks? The size of the memory or disk, the number of CPUs, etc. |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
This host has no virtualbox installed and runs ok. But I see that I managed to drain the whole queue by myself overnight :) I have put more tasks in the queue and will put my machines on other projects to allow others to try this one. The tasks are the same as the regular ATLAS@Home WU, so multicore (from 1 to 8 cores) and required memory is 2.3GB + 0.8 * ncores. |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
But I see that I managed to drain the whole queue by myself overnight :) I have put more tasks in the queue and will put my machines on other projects to allow others to try this one. something drain queue a few times before you but without having to install the necessary packages it makes sense to further limit the number of tasks in one hand |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
BOINC9 lhcathome-dev 24.02.2017 11:00:37 Sending scheduler request: To fetch work. do not understand why |
Send message Joined: 6 Mar 15 Posts: 19 Credit: 142,109 RAC: 0 |
That is very strange, are you sure this message is for ATLAS? I have tested this app successfully on a machine without virtualbox. what did you do for the http proxy? i ran the probe and it failed. for fun i tried to get tasks and i too get the virtualbox message |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
what did you do for the http proxy? I don't have any proxy. So I setup a proxy directly to the host sudo apt-get install squid And added "http://127.0.0.1:3128" to config |
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 861,475 RAC: 15 |
Hello David, Although not a real Linux native user, I gave it a try. After some errors, the results show "Successfully finished the ATLAS job!" and are valid in BOINC. But I'm not sure whether the results are useful. I set up tasks to use 2 cores/task. Elapsed time ~10-12 minutes, but very low CPU. Maybe the low CPU is cause BOINC only counts the CPU-time of the wrapper and not of the other processes. Host: https://lhcathomedev.cern.ch/lhcathome-dev/results.php?hostid=1778 Edit: Changed to single core tasks. The run time keeps the same. |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
specially installed virtualbox on the host the problem remains - there is "no tasks for ATLAS" it is necessary to look at the server - why tasks are not sent to the host |
Send message Joined: 28 Jul 16 Posts: 482 Credit: 394,720 RAC: 0 |
1 job finished successfully: https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=311891 But: - runtime was only a few minutes - stderr.txt shows "mv: cannot stat `metadata-*.xml': No such file or directory ERROR: Missing metadata.xml" Independent from this App: The initial download per job is still more than 200 MB. This is far too much as in case of an error every host pulls new jobs after only a few minutes and this would saturate nearly all kind of internet connections. |
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 861,475 RAC: 15 |
Between most valid tasks, I now also have some tasks with "Validate error", although they run like the valid tasks and also have "Successfully finished the ATLAS job!" in the stderr.txt. https://lhcathomedev.cern.ch/lhcathome-dev/results.php?hostid=1778 |
Send message Joined: 28 Jul 16 Posts: 482 Credit: 394,720 RAC: 0 |
... I now also have some tasks with "Validate error", Same here. Below some lines from .../BOINC/slots/x/pilotlog.txt 2017-02-24 13:44:34|16135|SiteInformat| !!WARNING!!2999!! $X509_CERT_DIR is not set and default location /etc/grid-security/certificates does not exist Do we need some configuration parameters that were previously unmentioned? |
Send message Joined: 26 Feb 15 Posts: 26 Credit: 5,042,431 RAC: 11,954 |
I am having problems $ cvmfs_config probe I can't reach the cern servers. Maybe it wants me to set a proxy? But I don't use a proxy. Reno, NV Team: SETI.USA |
Send message Joined: 11 Mar 16 Posts: 23 Credit: 68,680 RAC: 0 |
I can't reach the cern servers. Maybe it wants me to set a proxy? But I don't use a proxy. cvmfs_config chksetup if asks for proxy (CVMFS_HTTP_PROXY) - install squid (Ubuntu - "sudo apt-get install squid") and add CVMFS_HTTP_PROXY="http://127.0.0.1:3128" you need proxies only on a single host others can use it address |
Send message Joined: 28 Jul 16 Posts: 482 Credit: 394,720 RAC: 0 |
Although I recommend the use of a proxy you may try it direct. Insert the following line in your /etc/cvmfs/default.local CVMFS_HTTP_PROXY=DIRECT then execute "cvmfs_config reload". This is documented for Mac OS X but it may work on Linux also. |
Send message Joined: 26 Feb 15 Posts: 26 Credit: 5,042,431 RAC: 11,954 |
Although I recommend the use of a proxy you may try it direct. It worked! Thanks! $ cvmfs_config probe Reno, NV Team: SETI.USA |
Send message Joined: 17 Aug 15 Posts: 17 Credit: 228,358 RAC: 0 |
I am willing to give it a try, but what is the payoff? That is, is there much of a performance advantage? If it works, will you be able to access it on the main LHC site? |
©2024 CERN