Message boards : ATLAS Application : ATLAS native 0.92
Message board moderation

To post messages, you must log in.

AuthorMessage
David Cameron
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 20 Apr 16
Posts: 154
Credit: 1,352,539
RAC: 84
Message 6893 - Posted: 10 Dec 2019, 10:56:49 UTC

The current bootstrap script for native tasks is written in python2. To get around the problems of python2 end of life which happens at the end of the month, I have rewritten the script in bash. The python script was mainly calling shell commands and not doing anything pythonic so it actually makes it simpler to have it in bash like the vbox script.

I'm now testing this here so would be good to hear your feedback, especially on how different failure modes are handled. There should be no differences in how the actual tasks run, but maybe slight differences in the stderr output that you see for each task.

The code is now even in github to make it easier to track changes, and encourage contributions from the community to fix bugs :)

https://github.com/davidgcameron/boinc-scripts/blob/master/native/run_atlas.sh
ID: 6893 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6894 - Posted: 11 Dec 2019, 7:44:31 UTC - in response to Message 6893.  

SL77 - No tasks are available.
ID: 6894 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 20 Apr 16
Posts: 154
Credit: 1,352,539
RAC: 84
Message 6895 - Posted: 11 Dec 2019, 8:10:43 UTC - in response to Message 6894.  

There are 18 tasks in the queue according to the server status page. Did you check "run native" in the preferences?
ID: 6895 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6896 - Posted: 11 Dec 2019, 9:42:18 UTC - in response to Message 6895.  

Sorry, my fault. Had it forgotten to activate native if possible in prefs.
ID: 6896 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1147
Credit: 754,546
RAC: 10
Message 6897 - Posted: 11 Dec 2019, 10:44:31 UTC

wo 11 dec 2019 11:19:36 CET: *** The last 200 lines of the pilot log: ***
wo 11 dec 2019 11:19:36 CET: tail: cannot open 'log.000649-3223980-10833._078090.job.log.1' for reading: No such file or directory
ID: 6897 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6898 - Posted: 11 Dec 2019, 10:50:15 UTC

This is shown in stderr.txt
Mi 11. Dez 11:41:24 CET 2019: Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
Mi 11. Dez 11:41:27 CET 2019: Singularity isnt working: INFO:  Convert SIF file to sandbox...
Mi 11. Dez 11:41:27 CET 2019: FATAL:  while extracting /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img: root filesystem extraction failed: could not extract squashfs data, unsquashfs not found
ID: 6898 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 20 Apr 16
Posts: 154
Credit: 1,352,539
RAC: 84
Message 6899 - Posted: 11 Dec 2019, 13:55:26 UTC - in response to Message 6897.  

wo 11 dec 2019 11:19:36 CET: *** The last 200 lines of the pilot log: ***
wo 11 dec 2019 11:19:36 CET: tail: cannot open 'log.000649-3223980-10833._078090.job.log.1' for reading: No such file or directory


I know about this one, will fix it.

This is shown in stderr.txt
Mi 11. Dez 11:41:24 CET 2019: Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
Mi 11. Dez 11:41:27 CET 2019: Singularity isnt working: INFO:  Convert SIF file to sandbox...
Mi 11. Dez 11:41:27 CET 2019: FATAL:  while extracting /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img: root filesystem extraction failed: could not extract squashfs data, unsquashfs not found


I think you need to install squashfs to solve this problem. Has this computer run native tasks successfully before? The way singularity runs shouldn't be any different from the previous version of the script.
ID: 6899 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6900 - Posted: 11 Dec 2019, 14:15:08 UTC - in response to Message 6899.  
Last modified: 11 Dec 2019, 14:22:03 UTC

I think you need to install squashfs to solve this problem. Has this computer run native tasks successfully before? The way singularity runs shouldn't be any different from the previous version of the script.

Yes,
https://lhcathomedev.cern.ch/lhcathome-dev/results.php?hostid=3723&offset=0&show_names=0&state=0&appid=5
Will tomorrow install squashfs .
Edit: ATM Production native-Theory is running well.
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10586648
ID: 6900 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6901 - Posted: 12 Dec 2019, 7:28:09 UTC
Last modified: 12 Dec 2019, 8:11:01 UTC

After installing squashfs: https://github.com/plougher/squashfs-tools
first task ended with validate error.
second seems to run now:
Do 12. Dez 08:22:10 CET 2019: Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
Do 12. Dez 08:22:38 CET 2019: INFO:  Convert SIF file to sandbox... RYZSL76 INFO:  Cleaning up image...
Do 12. Dez 08:22:38 CET 2019: Singularity works
Do 12. Dez 08:22:40 CET 2019: Set ATHENA_PROC_NUMBER=2
Do 12. Dez 08:22:40 CET 2019: Starting ATLAS job with PandaID=4002876565
Do 12. Dez 08:22:40 CET 2019: Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec --pwd /root/Downloads/BOINC/slots/1 -B /cvmfs,/root /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img sh start_atlas.sh

Edit: Do 12. Dez 09:05:55 CET 2019: HITS file was successfully produced:
Do 12. Dez 09:05:55 CET 2019: -rw-------. 1 root root 9192758 12. Dez 09:03 shared/HITS.pool.root.1
ID: 6901 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 601
Credit: 1,451,233
RAC: 1,366
Message 6902 - Posted: 12 Dec 2019, 16:18:42 UTC

Have 2 Cpu's for this native-Atlas.
When Atlas is running after starting-time, native-Theory from Production is suspended, but
the PID of the Theory is not suspended and will finish the Theory task.
ID: 6902 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : ATLAS Application : ATLAS native 0.92


©2023 CERN