Message boards : LHCb Application : Ready For Production?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1021
Credit: 274,753
RAC: 0
Message 3525 - Posted: 1 Jun 2016, 10:40:43 UTC

LHCb would like to encourage their members to participate in this activity and as such we are considering to move this application as a beta app in the production project similar to what was done for CMS. Are there any opinions on this suggestions?
ID: 3525 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1010
Credit: 591,548
RAC: 0
Message 3526 - Posted: 1 Jun 2016, 12:17:12 UTC - in response to Message 3525.  

I'm not sure whether the application is doing useful work.
In the past I saw a python process using most of the cpu and my last LHCb task was ended very quickly with low cpu-usage.

http://lhcathomedev.cern.ch/vLHCathome-dev/result.php?resultid=193431

Incomplete running log:

Directories in PYTHONPATH: ['']
2016-06-01 11:36:45 UTC INFO [Pilot] Executing commands: ['LHCbGetPilotVersion', 'CheckWorkerNode', 'LHCbInstallDIRAC', 'LHCbConfigureBasics', 'LHCbCleanPilotEnv', 'LHCbConfigureSite', 'LHCbConfigureArchitecture', 'LHCbConfigureCPURequirements', 'LaunchAgent']
2016-06-01 11:36:45 UTC INFO [Pilot] Requested command extensions: ['LHCbPilot']
2016-06-01 11:36:45 UTC INFO [Pilot] Command LHCbGetPilotVersion instantiated from LHCbPilotCommands
2016-06-01 11:36:45 UTC INFO [LHCbGetPilotVersion] Pilot version not requested as pilot script option, going to find it
2016-06-01 11:36:45 UTC INFO [LHCbGetPilotVersion] Setting pilot version to v8r2p45
2016-06-01 11:36:45 UTC INFO [Pilot] Command CheckWorkerNode instantiated from pilotCommands
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] Uname = Linux 38-37-16958 3.10.64-85.cernvm.x86_64 #1 SMP Fri Jan 9 09:53:29 CET 2015 x86_64
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] Host Name = 38-37-16958
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] Host FQDN = localhost.localdomain
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] WorkingDir = /home/boinc/pilot
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] RedHat Release = Scientific Linux release 6.6 (Carbon)
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] Linux release:
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] CPU (model) = Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] CPU (MHz) = 1 x 3463.850
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] Memory (kB) = 2050972
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] FreeMem. (kB) = 1777788
2016-06-01 11:36:45 UTC INFO [CheckWorkerNode] DiskSpace (MB) = 17266
2016-06-01 11:36:45 UTC INFO [Pilot] Command LHCbInstallDIRAC instantiated from LHCbPilotCommands
********************************************************************************
* ---- LHCb Login v8r6p1 ---- *
* Building with gcc49 on slc6 x86_64 system (x86_64-slc6-gcc49-opt) *
********************************************************************************
--- User_release_area is set to /home/boinc/pilot/cmtuser
--- LHCBPROJECTPATH is set to:
/cvmfs/lhcb.cern.ch/lib/lhcb
/cvmfs/lhcb.cern.ch/lib/lcg/releases
/cvmfs/lhcb.cern.ch/lib/lcg/app/releases
/cvmfs/lhcb.cern.ch/lib/lcg/external
--------------------------------------------------------------------------------
Using CMTPROJECTPATH = '/cvmfs/lhcb.cern.ch/lib/lhcb:/cvmfs/lhcb.cern.ch/lib/lcg/releases:/cvmfs/lhcb.cern.ch/lib/lcg/app/releases:/cvmfs/lhcb.cern.ch/lib/lcg/external'
Environment for LbScripts v8r6p1 ready.
(Compat v1r19 from /cvmfs/lhcb.cern.ch/lib/lhcb/COMPAT/COMPAT_v1r19,
LbScripts v8r6p1 from /cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v8r6p1,
LCGCMT 84 from /cvmfs/lhcb.cern.ch/lib/lcg/releases/LCGCMT/LCGCMT_84,
Compat v1r19 from /cvmfs/lhcb.cern.ch/lib/lhcb/COMPAT/COMPAT_v1r19)
2016-06-01 11:37:12 UTC INFO [LHCbInstallDIRAC] lb-run DONE, for release v8r2p45
2016-06-01 11:37:12 UTC INFO [Pilot] Command LHCbConfigureBasics instantiated from LHCbPilotCommands
2016-06-01 11:37:12 UTC WARN [LHCbConfigureBasics] Can't find shared area, forcing it to /cvmfs/lhcb.cern.ch/lib
2016-06-01 11:37:12 UTC INFO [LHCbConfigureBasics] Executing command dirac-configure -S "LHCb-Production" -C "dips://lbvobox46.cern.ch:9135/Configuration/Server" -o /LocalSite/ReleaseProject=LHCb -o /LocalSite/ReleaseVersion=v8r2p45 -o /LocalSite/SharedArea=/cvmfs/lhcb.cern.ch/lib -DMH --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -O pilot.cfg
Executing: /cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31/scripts/dirac-configure -S LHCb-Production -C dips://lbvobox46.cern.ch:9135/Configuration/Server -o /LocalSite/ReleaseProject=LHCb -o /LocalSite/ReleaseVersion=v8r2p45 -o /LocalSite/SharedArea=/cvmfs/lhcb.cern.ch/lib -DMH --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -O pilot.cfg
Checking DIRAC installation at "/cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31"
URL banned dips://lhcb-conf2-dirac.cern.ch:9135/Configuration/Server

2016-06-01 11:37:28 UTC INFO [Pilot] Command LHCbCleanPilotEnv instantiated from LHCbPilotCommands
2016-06-01 11:37:28 UTC WARN [LHCbCleanPilotEnv] Can't find shared area, forcing it to /cvmfs/lhcb.cern.ch/lib
2016-06-01 11:37:28 UTC INFO [LHCbCleanPilotEnv] Executing command dirac-configure -S "LHCb-Production" -C "dips://lbvobox46.cern.ch:9135/Configuration/Server" -o /LocalSite/ReleaseProject=LHCb -o /LocalSite/ReleaseVersion=v8r2p45 -o /LocalSite/SharedArea=/cvmfs/lhcb.cern.ch/lib -o /DIRAC/Configuration/Servers=dips://lbvobox46.cern.ch:9135/Configuration/Server -DMH --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -O pilot.cfg
Executing: /cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31/scripts/dirac-configure -S LHCb-Production -C dips://lbvobox46.cern.ch:9135/Configuration/Server -o /LocalSite/ReleaseProject=LHCb -o /LocalSite/ReleaseVersion=v8r2p45 -o /LocalSite/SharedArea=/cvmfs/lhcb.cern.ch/lib -o /DIRAC/Configuration/Servers=dips://lbvobox46.cern.ch:9135/Configuration/Server -DMH --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -O pilot.cfg
Checking DIRAC installation at "/cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31"

2016-06-01 11:37:30 UTC INFO [Pilot] Command LHCbConfigureSite instantiated from LHCbPilotCommands
2016-06-01 11:37:30 UTC INFO [LHCbConfigureSite] Executing command dirac-configure -o /LocalSite/GridMiddleware=DIRAC -n "BOINC.World.org" -S "LHCb-Production" -N "Boinc-World-CE.org" -o /LocalSite/GridCE=Boinc-World-CE.org -o /LocalSite/CEQueue=Boinc.World.Queue --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -FDMH -O pilot.cfg pilot.cfg
Executing: /cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31/scripts/dirac-configure -o /LocalSite/GridMiddleware=DIRAC -n BOINC.World.org -S LHCb-Production -N Boinc-World-CE.org -o /LocalSite/GridCE=Boinc-World-CE.org -o /LocalSite/CEQueue=Boinc.World.Queue --UseServerCertificate -o /DIRAC/Security/CertFile=/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/etc/grid-security/hostkey.pem -FDMH -O pilot.cfg pilot.cfg
Checking DIRAC installation at "/cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31"
Will update the output file pilot.cfg
Setting /LocalSite/Site = BOINC.World.org
Setting /LocalSite/GridCE = Boinc-World-CE.org

2016-06-01 11:37:32 UTC INFO [Pilot] Command LHCbConfigureArchitecture instantiated from LHCbPilotCommands
2016-06-01 11:37:32 UTC INFO [LHCbConfigureArchitecture] Executing command dirac-architecture -o /DIRAC/Security/UseServerCertificate=yes pilot.cfg
x86_64-slc6

2016-06-01 11:37:36 UTC INFO [LHCbConfigureArchitecture] Executing command dirac-configure -FDMH --UseServerCertificate -O pilot.cfg pilot.cfg -S "LHCb-Production" -o /LocalSite/Architecture=x86_64-slc6
Executing: /cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31/scripts/dirac-configure -FDMH --UseServerCertificate -O pilot.cfg pilot.cfg -S LHCb-Production -o /LocalSite/Architecture=x86_64-slc6
Checking DIRAC installation at "/cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31"
Will update the output file pilot.cfg
Setting /LocalSite/Site = BOINC.World.org
Setting /LocalSite/GridCE = Boinc-World-CE.org

2016-06-01 11:37:38 UTC INFO [LHCbConfigureArchitecture] Setting variable CMTCONFIG=x86_64-slc6
2016-06-01 11:37:38 UTC INFO [Pilot] Command LHCbConfigureCPURequirements instantiated from LHCbPilotCommands
2016-06-01 11:37:38 UTC INFO [LHCbConfigureCPURequirements] Executing command dirac-wms-cpu-normalization -U -o /DIRAC/Security/UseServerCertificate=yes -R pilot.cfg pilot.cfg
Estimated CPU power is 5.9 HS06
MJF not available on this node

2016-06-01 11:39:10 UTC INFO [LHCbConfigureCPURequirements] Current normalized CPU as determined by 'dirac-wms-cpu-normalization' is 5.900000
2016-06-01 11:39:10 UTC INFO [LHCbConfigureCPURequirements] Executing command dirac-wms-get-queue-cpu-time -o /DIRAC/Security/UseServerCertificate=yes pilot.cfg
16949

2016-06-01 11:39:13 UTC INFO [LHCbConfigureCPURequirements] CPUTime left (in seconds) is 16949
2016-06-01 11:39:13 UTC INFO [LHCbConfigureCPURequirements] Queue length (which is also set as CPUTimeLeft) is 99999.100000
2016-06-01 11:39:13 UTC INFO [LHCbConfigureCPURequirements] Executing command dirac-configure -FDMH -o /DIRAC/Security/UseServerCertificate=yes -O pilot.cfg pilot.cfg -o /LocalSite/CPUTimeLeft=99999
Executing: /cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31/scripts/dirac-configure -FDMH -o /DIRAC/Security/UseServerCertificate=yes -O pilot.cfg pilot.cfg -o /LocalSite/CPUTimeLeft=99999
Checking DIRAC installation at "/cvmfs/lhcb.cern.ch/lib/lhcb/DIRAC/DIRAC_v6r14p31"
Will update the output file pilot.cfg
Setting /LocalSite/Site = BOINC.World.org
Setting /LocalSite/GridCE = Boinc-World-CE.org

2016-06-01 11:39:15 UTC INFO [Pilot] Command LaunchAgent instantiated from pilotCommands
2016-06-01 11:39:15 UTC INFO [LaunchAgent] User Name = boinc
2016-06-01 11:39:15 UTC INFO [LaunchAgent] User Id = 500
2016-06-01 11:39:15 UTC INFO [LaunchAgent] Starting JobAgent
2016-06-01 11:39:15 UTC INFO [LaunchAgent] Executing command dirac-agent WorkloadManagement/JobAgent -o MaxCycles=10 -s /Resources/Computing/CEDefaults -o WorkingDirectory=/home/boinc/pilot -o /LocalSite/MaxCPUTime=99999 -o /LocalSite/CPUTime=99999 -o MaxTotalJobs=10 -o /DIRAC/Security/UseServerCertificate=yes -o /LocalSite/InstancePath=/home/boinc/pilot -o /AgentJobRequirements/ExtraOptions=pilot.cfg pilot.cfg /home/boinc/pilot/pilot.cfg
ID: 3526 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Cinzia

Send message
Joined: 3 Mar 16
Posts: 10
Credit: 33,623
RAC: 0
Message 3530 - Posted: 2 Jun 2016, 7:36:39 UTC - in response to Message 3526.  

Hi Crystal,

the log is not incomplete. There you see the pilot doing all the necessary steps in order to configure the environment and start a JobAgent, that will then pick a job and execute it.
The task may be ending quickly if we do not have waiting jobs.
ID: 3530 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1021
Credit: 274,753
RAC: 0
Message 3543 - Posted: 6 Jun 2016, 9:25:42 UTC - in response to Message 3530.  

With the exception of there not always being jobs available, I don't see any other complaints so in the absence of further comments will assume tacit agreement.
ID: 3543 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : LHCb Application : Ready For Production?


©2020 CERN