Message boards :
News :
New native Linux ATLAS application
Message board moderation
Previous · 1 · 2 · 3 · 4
Author | Message |
---|---|
Send message Joined: 29 May 15 Posts: 147 Credit: 2,842,484 RAC: 0 |
Do you have a recipe for making this work? I followed this description: https://www.windowspro.de/wolfgang-sommergut/nested-virtualization-esxi-hyper-v-unter-esxi-51-ausfuehren If you search the web with "nested virtualization" you will find more (and newer) scripts, but the mentioned script lead me to being succesfull with my ESX-Hosts. I'm running 4 ESX-5.5-Hosts, 3 of them the clients are capabable to run Atlas, but the fourth has an too old processor that I didn't get it working |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
The outcome of that command: For this you need to apply the workaround mentioned earlier in this thread Then see if you can run Sim_tf.py? If you give no arguments it will print a bunch of messages then exit. I don't have a file Sim_tf.py in that slot (11 in this case) 4 python files available: LFCTools.py RunJob.py VmPeak.py outputList.py[/quote] Sim_tf.py is a command located in CVMFS. When you run the command above it sets up the environment to run these commands in CVMFS. So you can just try typing Sim_tf.py to see if it runs. You can use "which Sim_tf.py" to find out the exact location of the command. |
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 861,475 RAC: 15 |
Symbolic links set yesterday seem to be gone. I set them again. The outcome of the 2 commands: $. /var/lib/boinc-client/slots/11//APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT 1 Using AtlasProduction/19.2.4.9 with platform x86_64-slc6-gcc47-opt at /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 manpath: warning: $MANPATH set, ignoring /etc/manpath.config ERROR:root:code for hash md5 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type md5 ERROR:root:code for hash sha1 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha1 ERROR:root:code for hash sha224 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha224 ERROR:root:code for hash sha256 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha256 ERROR:root:code for hash sha384 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha384 ERROR:root:code for hash sha512 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha512 ================================================================================================================= $ Sim_tf.py PyJobTransforms.<module> 2017-03-03 16:02:14,579 INFO logging set in /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py ERROR:root:code for hash md5 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type md5 ERROR:root:code for hash sha1 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha1 ERROR:root:code for hash sha224 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha224 ERROR:root:code for hash sha256 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha256 ERROR:root:code for hash sha384 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha384 ERROR:root:code for hash sha512 was not found. Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] = __get_hash(__func_name) File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor raise ValueError('unsupported hash type %s' % name) ValueError: unsupported hash type sha512 PyJobTransforms.main 2017-03-03 16:02:16,637 INFO This is /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py INFO:PyJobTransforms:This is /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py PyJobTransforms.transform.parseCmdLineArgs 2017-03-03 16:02:16,652 INFO Transform command line was: '/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py' INFO:PyJobTransforms.transform:Transform command line was: '/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py' PyJobTransforms.transform.execute 2017-03-03 16:02:16,653 INFO Starting to resolve execution graph INFO:PyJobTransforms.transform:Starting to resolve execution graph PyJobTransforms.transform.execute 2017-03-03 16:02:16,656 INFO Execution graph resolved INFO:PyJobTransforms.transform:Execution graph resolved PyJobTransforms.transform.execute 2017-03-03 16:02:16,656 INFO Starting to trace execution path INFO:PyJobTransforms.transform:Starting to trace execution path PyJobTransforms.transform.execute 2017-03-03 16:02:16,656 INFO Execution path found with 1 step(s): EVNTtoHITS INFO:PyJobTransforms.transform:Execution path found with 1 step(s): EVNTtoHITS PyJobTransforms.transform.validateInFiles 2017-03-03 16:02:16,657 INFO Validating input files INFO:PyJobTransforms.transform:Validating input files PyJobTransforms.trfValidation.performStandardFileValidation 2017-03-03 16:02:16,657 INFO Starting legacy (serial) file validation INFO:PyJobTransforms.trfValidation:Starting legacy (serial) file validation PyJobTransforms.trfValidation.performStandardFileValidation 2017-03-03 16:02:16,657 INFO Stopping legacy (serial) file validation INFO:PyJobTransforms.trfValidation:Stopping legacy (serial) file validation PyJobTransforms.trfExe._detectAthenaMP 2017-03-03 16:02:16,657 INFO No AthenaMP options found - assuming normal athena run INFO:PyJobTransforms.trfExe:No AthenaMP options found - assuming normal athena run PyJobTransforms.trfJobOptions.writeRunArgs 2017-03-03 16:02:16,657 INFO Writing runArgs to file "runargs.EVNTtoHITS.py" INFO:PyJobTransforms.trfJobOptions:Writing runArgs to file "runargs.EVNTtoHITS.py" PyJobTransforms.trfJobOptions.writeRunArgs 2017-03-03 16:02:16,657 INFO Successfully wrote runargs file runargs.EVNTtoHITS.py INFO:PyJobTransforms.trfJobOptions:Successfully wrote runargs file runargs.EVNTtoHITS.py PyJobTransforms.trfExe.preExecute 2017-03-03 16:02:16,659 INFO Asetup report: AtlasBaseDir=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 AtlasProject=AtlasProduction AtlasVersion=19.2.4.9 AtlasPatch=AtlasProduction AtlasPatchVersion=19.2.4.9 CMTCONFIG=x86_64-slc6-gcc47-opt No readable patch area found INFO:PyJobTransforms.trfExe:Asetup report: AtlasBaseDir=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 AtlasProject=AtlasProduction AtlasVersion=19.2.4.9 AtlasPatch=AtlasProduction AtlasPatchVersion=19.2.4.9 CMTCONFIG=x86_64-slc6-gcc47-opt No readable patch area found PyJobTransforms.trfUtils.releaseIsOlderThan 2017-03-03 16:02:16,660 INFO Detected release major 19, minor 2 (.4.9) from environment INFO:PyJobTransforms.trfUtils:Detected release major 19, minor 2 (.4.9) from environment PyJobTransforms.trfEnv.probeIMFSettings 2017-03-03 16:02:16,660 INFO Enabling IMF by default for release INFO:PyJobTransforms.trfEnv:Enabling IMF by default for release PyJobTransforms.trfEnv.probeTcmallocSettings 2017-03-03 16:02:16,661 INFO Skipping inclusion of tcmalloc INFO:PyJobTransforms.trfEnv:Skipping inclusion of tcmalloc PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-03 16:02:16,662 INFO Setting athena preloadlibs to: /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so INFO:PyJobTransforms.trfExe:Setting athena preloadlibs to: /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-03 16:02:16,662 INFO Skipping test for "--drop-and-reload" in this executor INFO:PyJobTransforms.trfExe:Skipping test for "--drop-and-reload" in this executor PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-03 16:02:16,662 INFO Updated script arguments with topoptions: ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] INFO:PyJobTransforms.trfExe:Updated script arguments with topoptions: ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] PyJobTransforms.trfExe.preExecute 2017-03-03 16:02:16,663 INFO Will execute script as ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] INFO:PyJobTransforms.trfExe:Will execute script as ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] PyJobTransforms.trfExe.preExecute 2017-03-03 16:02:16,663 INFO Interactive environment detected (stdio or stdout is a tty) - enabling command echoing to stdout INFO:PyJobTransforms.trfExe:Interactive environment detected (stdio or stdout is a tty) - enabling command echoing to stdout PyJobTransforms.trfExe.preExecute 2017-03-03 16:02:16,664 INFO Now writing wrapper for substep executor EVNTtoHITS INFO:PyJobTransforms.trfExe:Now writing wrapper for substep executor EVNTtoHITS PyJobTransforms.trfExe._writeAthenaWrapper 2017-03-03 16:02:16,664 INFO Valgrind not engaged INFO:PyJobTransforms.trfExe:Valgrind not engaged PyJobTransforms.trfExe.preExecute 2017-03-03 16:02:16,664 INFO Athena will be executed in a subshell via ['./runwrapper.EVNTtoHITS.sh'] INFO:PyJobTransforms.trfExe:Athena will be executed in a subshell via ['./runwrapper.EVNTtoHITS.sh'] PyJobTransforms.trfExe.execute 2017-03-03 16:02:16,664 INFO Starting execution of EVNTtoHITS (['./runwrapper.EVNTtoHITS.sh']) INFO:PyJobTransforms.trfExe:Starting execution of EVNTtoHITS (['./runwrapper.EVNTtoHITS.sh']) EVNTtoHITS 16:02:16 Fri Mar 3 16:02:16 CET 2017 EVNTtoHITS 16:02:16 Preloading tcmalloc_minimal.so EVNTtoHITS 16:02:16 Preloading /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so EVNTtoHITS 16:02:17 ERROR:root:code for hash md5 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type md5 EVNTtoHITS 16:02:17 ERROR:root:code for hash sha1 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type sha1 EVNTtoHITS 16:02:17 ERROR:root:code for hash sha224 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type sha224 EVNTtoHITS 16:02:17 ERROR:root:code for hash sha256 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type sha256 EVNTtoHITS 16:02:17 ERROR:root:code for hash sha384 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type sha384 EVNTtoHITS 16:02:17 ERROR:root:code for hash sha512 was not found. EVNTtoHITS 16:02:17 Traceback (most recent call last): EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 139, in <module> EVNTtoHITS 16:02:17 globals()[__func_name] = __get_hash(__func_name) EVNTtoHITS 16:02:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/lcg/external/Python/2.7.3/x86_64-slc6-gcc47-opt/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor EVNTtoHITS 16:02:17 raise ValueError('unsupported hash type %s' % name) EVNTtoHITS 16:02:17 ValueError: unsupported hash type sha512 EVNTtoHITS 16:02:17 Py:Athena INFO including file "AthenaCommon/Preparation.py" EVNTtoHITS 16:02:17 Py:Athena INFO using release [AtlasProduction-19.2.4.9] [x86_64-slc6-gcc47-opt] [19.2.X.Y-VAL-Prod/rel_3] -- built on [2015-08-12 02:01] EVNTtoHITS 16:02:17 Py:Athena INFO including file "AthenaCommon/Bootstrap.py" EVNTtoHITS 16:02:18 Py:Athena INFO including file "AthenaCommon/Atlas.UnixStandardJob.py" EVNTtoHITS 16:02:18 Traceback (most recent call last): EVNTtoHITS 16:02:18 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasCore/19.2.4/InstallArea/jobOptions/AthenaCommon/Preparation.py", line 86, in <module> EVNTtoHITS 16:02:18 import cppyy EVNTtoHITS 16:02:18 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/atlas/offline/external/root-snapshot/atlas-5.34.13p1/x86_64-slc6-gcc47-opt/lib/cppyy.py", line 52, in <module> EVNTtoHITS 16:02:18 import libPyROOT as _backend EVNTtoHITS 16:02:18 ImportError: libcrypto.so.10: cannot open shared object file: No such file or directory, ROOT version or setup problem? PyJobTransforms.trfExe.execute 2017-03-03 16:02:18,230 INFO EVNTtoHITS executor returns 1 INFO:PyJobTransforms.trfExe:EVNTtoHITS executor returns 1 PyJobTransforms.trfExe.postExecute 2017-03-03 16:02:18,309 WARNING Failed to process expected perfMon stats file ntuple.pmon.gz: [Errno 2] No such file or directory: 'ntuple.pmon.gz' WARNING:PyJobTransforms.trfExe:Failed to process expected perfMon stats file ntuple.pmon.gz: [Errno 2] No such file or directory: 'ntuple.pmon.gz' PyJobTransforms.trfExe.validate 2017-03-03 16:02:18,310 ERROR Validation of return code failed: Non-zero return code from EVNTtoHITS (1) (Error code 65) ERROR:PyJobTransforms.trfExe:Validation of return code failed: Non-zero return code from EVNTtoHITS (1) (Error code 65) PyJobTransforms.trfExe.validate 2017-03-03 16:02:18,335 INFO Scanning logfile log.EVNTtoHITS for errors INFO:PyJobTransforms.trfExe:Scanning logfile log.EVNTtoHITS for errors PyJobTransforms.transform.execute 2017-03-03 16:02:18,338 CRITICAL Transform executor raised TransformValidationException: Non-zero return code from EVNTtoHITS (1) CRITICAL:PyJobTransforms.transform:Transform executor raised TransformValidationException: Non-zero return code from EVNTtoHITS (1) |
Send message Joined: 20 Mar 15 Posts: 243 Credit: 886,442 RAC: 0 |
David,
Where does your script expect to find these libs? Mine are in /lib/x86_64-linux-gnu which is where I put the symlinks. There are (32bit?) ones in /lib/i386-linux-gnu and there apparently are apps that expect them to be in /usr/lib or maybe /usr/lib/x86_64-linux-gnu . |
Send message Joined: 26 Feb 15 Posts: 26 Credit: 5,042,431 RAC: 11,954 |
Same here. On Ubuntu 14.04.5 $ ldd /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/x86_64-slc6-gcc47-opt/lib/libNet.so $ ls -l /lib/x86_64-linux-gnu/libcrypto.so.10 /lib/x86_64-linux-gnu/libssl.so.10 I also added /lib/x86_64-linux-gnu/ to the environment variable LD_LIBRARY_PATH to no avail. $ echo $LD_LIBRARY_PATH | awk '{ split($0,t,":"); for (i in t) print t[i] }' $ Sim_tf.py Reno, NV Team: SETI.USA |
Send message Joined: 20 Mar 15 Posts: 243 Credit: 886,442 RAC: 0 |
Just run a couple of jobs. Both failed. The second still downloaded >200MB. Looking at one of the jobs, the stderr that's included with the task result isn't all of the stderr.txt from the slot. In the bit that's missing, there is:- <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> Reason: You're speaking plain HTTP to an SSL-enabled server port.<br /> Instead use the HTTPS scheme to access this URL, please.<br /> <blockquote>Hint: <a href=\"https://pandaserver.cern.ch:25443/\"><b>https://pandaserver.cern.ch:25443/</b></a></blockquote></p> </body></html> Are we still having ssl problems? Have I put the symlinks in the right place? Different distributions put libs in different places. |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
Are we still having ssl problems? Have I put the symlinks in the right place? Different distributions put libs in different places. This message is unrelated to the ssl libraries. The task is trying to contact an ATLAS central service and failing but it doesn't affect the task. Last week I set up a Ubuntu VM with 16.04, added the ssl symlinks and it worked ok. There should be no need to set LD_LIBRARY_PATH if the symlink is in the same directory as the original library. But I'll try to investigate a bit more tomorrow. |
Send message Joined: 28 Jul 16 Posts: 482 Credit: 394,720 RAC: 0 |
boinc@xxxx:/> /home/boinc/BOINC/slots/6/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT 1 bash: /home/boinc/BOINC/slots/6/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT: Keine Berechtigung The scripts in this directory donĀ“t have the executable flag by default. boinc@xxxx:/> chmod +x /home/boinc/BOINC/slots/6/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT boinc@xxxx:/> /home/boinc/BOINC/slots/6/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT 1 manpath: Warnung: $MANPATH ist gesetzt, ignoriere /etc/manpath.config Using AtlasProduction/19.2.4.9 with platform x86_64-slc6-gcc47-opt at /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 manpath: warning: $MANPATH set, ignoring /etc/manpath.config boinc@xxxx:/> Sim_tf.py If 'Sim_tf.py' is not a typo you can use command-not-found to lookup the package that contains it, like this: cnf Sim_tf.py boinc@xxxx:/> which Sim_tf.py which: no Sim_tf.py in (whole path) Path is not set. Try a direct call. boinc@xxxx:/> /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py Traceback (most recent call last): File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py", line 14, in <module> from PyJobTransforms.trfLogger import msg ImportError: No module named PyJobTransforms.trfLogger Any comments/suggestions? |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
These scripts are not executables, they are sourced (with ".") to set up environment variables like PATH. . /home/boinc/BOINC/slots/6/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT 1 |
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 861,475 RAC: 15 |
I retried the ATLAS native testing this morning after the weekend silence. I was hopeful cause the job was running longer than the usual ~4 minutes. Now over 9 minutes, but did not get into the real processing part and was validated as an error. Result: https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=316070 Maybe you can find something informative in the stderr.txt. My Linux VM is running Ubuntu 12.04.5 LTS. Also the 200MB downloading with every task is not solved, but I'm not suffering from low bandwidth. |
Send message Joined: 28 Jul 16 Posts: 482 Credit: 394,720 RAC: 0 |
These scripts are not executables, they are sourced (with ".") to set up environment variables like PATH. Shame. I ignored the ".". Result of the next try: boinc@xxxx:~> . /home/boinc/BOINC/slots/4/APPS/HEP/ATLAS-19.2.4.9-X86_64-SLC6-GCC47-OPT 1 manpath: Warnung: $MANPATH ist gesetzt, ignoriere /etc/manpath.config Using AtlasProduction/19.2.4.9 with platform x86_64-slc6-gcc47-opt at /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 manpath: warning: $MANPATH set, ignoring /etc/manpath.config boinc@xxxx:~> Sim_tf.py PyJobTransforms.<module> 2017-03-09 10:04:15,359 INFO logging set in /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py PyJobTransforms.main 2017-03-09 10:04:16,008 INFO This is /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py PyJobTransforms.transform.parseCmdLineArgs 2017-03-09 10:04:16,016 INFO Transform command line was: '/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/share/bin/Sim_tf.py' PyJobTransforms.transform.execute 2017-03-09 10:04:16,017 INFO Starting to resolve execution graph PyJobTransforms.transform.execute 2017-03-09 10:04:16,019 INFO Execution graph resolved PyJobTransforms.transform.execute 2017-03-09 10:04:16,019 INFO Starting to trace execution path PyJobTransforms.transform.execute 2017-03-09 10:04:16,020 INFO Execution path found with 1 step(s): EVNTtoHITS PyJobTransforms.transform.validateInFiles 2017-03-09 10:04:16,020 INFO Validating input files PyJobTransforms.trfValidation.performStandardFileValidation 2017-03-09 10:04:16,020 INFO Starting legacy (serial) file validation PyJobTransforms.trfValidation.performStandardFileValidation 2017-03-09 10:04:16,020 INFO Stopping legacy (serial) file validation PyJobTransforms.trfExe._detectAthenaMP 2017-03-09 10:04:16,020 INFO No AthenaMP options found - assuming normal athena run PyJobTransforms.trfJobOptions.writeRunArgs 2017-03-09 10:04:16,020 INFO Writing runArgs to file "runargs.EVNTtoHITS.py" PyJobTransforms.trfJobOptions.writeRunArgs 2017-03-09 10:04:16,020 INFO Successfully wrote runargs file runargs.EVNTtoHITS.py PyJobTransforms.trfExe.preExecute 2017-03-09 10:04:16,021 INFO Asetup report: AtlasBaseDir=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4 AtlasProject=AtlasProduction AtlasVersion=19.2.4.9 AtlasPatch=AtlasProduction AtlasPatchVersion=19.2.4.9 CMTCONFIG=x86_64-slc6-gcc47-opt No readable patch area found PyJobTransforms.trfUtils.releaseIsOlderThan 2017-03-09 10:04:16,021 INFO Detected release major 19, minor 2 (.4.9) from environment PyJobTransforms.trfEnv.probeIMFSettings 2017-03-09 10:04:16,021 INFO Enabling IMF by default for release PyJobTransforms.trfEnv.probeTcmallocSettings 2017-03-09 10:04:16,022 INFO Skipping inclusion of tcmalloc PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-09 10:04:16,022 INFO Setting athena preloadlibs to: /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-09 10:04:16,023 INFO Skipping test for "--drop-and-reload" in this executor PyJobTransforms.trfExe._prepAthenaCommandLine 2017-03-09 10:04:16,023 INFO Updated script arguments with topoptions: ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] PyJobTransforms.trfExe.preExecute 2017-03-09 10:04:16,023 INFO Will execute script as ['athena.py', '--preloadlib=/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so', 'runargs.EVNTtoHITS.py', 'SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py'] PyJobTransforms.trfExe.preExecute 2017-03-09 10:04:16,023 INFO Interactive environment detected (stdio or stdout is a tty) - enabling command echoing to stdout PyJobTransforms.trfExe.preExecute 2017-03-09 10:04:16,023 INFO Now writing wrapper for substep executor EVNTtoHITS PyJobTransforms.trfExe._writeAthenaWrapper 2017-03-09 10:04:16,023 INFO Valgrind not engaged PyJobTransforms.trfExe.preExecute 2017-03-09 10:04:16,023 INFO Athena will be executed in a subshell via ['./runwrapper.EVNTtoHITS.sh'] PyJobTransforms.trfExe.execute 2017-03-09 10:04:16,023 INFO Starting execution of EVNTtoHITS (['./runwrapper.EVNTtoHITS.sh']) EVNTtoHITS 10:04:16 Thu Mar 9 10:04:16 CET 2017 EVNTtoHITS 10:04:16 Preloading tcmalloc_minimal.so EVNTtoHITS 10:04:16 Preloading /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libintlc.so.5:/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/sw/IntelSoftware/linux/x86_64/xe2013/composer_xe_2013.3.163/compiler/lib/intel64/libimf.so EVNTtoHITS 10:04:16 Py:Athena INFO including file "AthenaCommon/Preparation.py" EVNTtoHITS 10:04:16 Py:Athena INFO using release [AtlasProduction-19.2.4.9] [x86_64-slc6-gcc47-opt] [19.2.X.Y-VAL-Prod/rel_3] -- built on [2015-08-12 02:01] EVNTtoHITS 10:04:16 Py:Athena INFO including file "AthenaCommon/Bootstrap.py" EVNTtoHITS 10:04:16 Py:Athena INFO including file "AthenaCommon/Atlas.UnixStandardJob.py" EVNTtoHITS 10:04:17 Py:Athena INFO including file "AthenaCommon/Execution.py" EVNTtoHITS 10:04:17 Py:Athena INFO including file "runargs.EVNTtoHITS.py" EVNTtoHITS 10:04:17 Py:Athena INFO including file "SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py" EVNTtoHITS 10:04:17 Py:Athena INFO including file "SimuJobTransforms/skeleton.EVGENtoHIT.py" EVNTtoHITS 10:04:17 Py:Athena INFO including file "SimuJobTransforms/CommonSkeletonJobOptions.py" EVNTtoHITS 10:04:17 Py:ISF INFO ****************** STARTING ISF ****************** EVNTtoHITS 10:04:17 Py:JobPropertyContainer:: INFO SimFlags:: Loading ATLAS flags EVNTtoHITS 10:04:17 Shortened traceback (most recent user call last): EVNTtoHITS 10:04:17 File "/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-opt/19.2.4/AtlasProduction/19.2.4.9/InstallArea/jobOptions/SimuJobTransforms/skeleton.EVGENtoHIT_ISF.py", line 24, in <module> EVNTtoHITS 10:04:17 raise RuntimeError("No geometryVersion provided.") EVNTtoHITS 10:04:17 RuntimeError: No geometryVersion provided. EVNTtoHITS 10:04:17 Py:Athena INFO leaving with code 8: "an unknown exception occurred" PyJobTransforms.trfExe.execute 2017-03-09 10:04:17,719 INFO EVNTtoHITS executor returns 8 PyJobTransforms.trfExe.postExecute 2017-03-09 10:04:17,761 WARNING Failed to process expected perfMon stats file ntuple.pmon.gz: [Errno 2] No such file or directory: 'ntuple.pmon.gz' PyJobTransforms.trfExe.validate 2017-03-09 10:04:17,761 ERROR Validation of return code failed: Non-zero return code from EVNTtoHITS (8) (Error code 65) PyJobTransforms.trfExe.validate 2017-03-09 10:04:17,777 INFO Scanning logfile log.EVNTtoHITS for errors PyJobTransforms.trfValidation.scanLogFile 2017-03-09 10:04:17,779 WARNING Detected python exception - activating python exception grabber PyJobTransforms.transform.execute 2017-03-09 10:04:17,779 CRITICAL Transform executor raised TransformValidationException: Non-zero return code from EVNTtoHITS (8); Logfile error in log.EVNTtoHITS: "RuntimeError: No geometryVersion provided." boinc@xxxx:~> What about the fact that a dualcore job produces a 50 MB result file while a singlecore job (same input?) produces only 20 kB? |
Send message Joined: 20 Apr 16 Posts: 180 Credit: 1,355,327 RAC: 0 |
Hi all, Thanks a lot for your helpful testing in the last weeks. Unfortunately it seems like there are rather deep issues with running ATLAS code on Ubuntu which I was not able to solve so far. As you know we have been planning the consolidation of ATLAS@Home with LHC@Home for some time and now I would like to focus on completing this. So I'm going to leave the native linux app for now and come back to it later. I will let the current WU drain and not submit any more. Hopefully we can come back to this at some point soon. There is a lot of discussion in ATLAS computing in general about using containers so perhaps this could be a future direction to deal with the OS incompatibilities. David |
©2024 CERN