x509 proxy error

Author	Message
PDW Send message Joined: 20 May 15 Posts: 217 Credit: 5,968,227 RAC: 0	Message 3297 - Posted: 10 May 2016, 7:16:11 UTC After trying 6 times (per project site) to request a credential tasks are failing... 2016-05-10 07:18:48 (12888): Guest Log: [ERROR] Could not get an x509 credential 2016-05-10 07:18:48 (12888): Guest Log: [ERROR] The x509 proxy creation failed. 2016-05-10 07:18:48 (12888): Guest Log: [INFO] Shutting Down. 2016-05-10 07:18:48 (12888): VM Completion File Detected. 2016-05-10 07:18:48 (12888): VM Completion Message: The x509 proxy creation failed. ID: 3297 · Rating: 0 · rate: / Reply Quote

Rasputin42 Volunteer tester Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0	Message 3298 - Posted: 10 May 2016, 7:54:14 UTC Same for CMS-tasks. ID: 3298 · Rating: 0 · rate: / Reply Quote

PDW Send message Joined: 20 May 15 Posts: 217 Credit: 5,968,227 RAC: 0	Message 3300 - Posted: 10 May 2016, 8:46:00 UTC The host is now told that it has completed its quota for the day... 10/05/2016 09:36:49 \| vLHCathome-dev \| Sending scheduler request: Requested by user. 10/05/2016 09:36:49 \| vLHCathome-dev \| Requesting new tasks for CPU 10/05/2016 09:36:50 \| vLHCathome-dev \| Scheduler request completed: got 0 new tasks 10/05/2016 09:36:50 \| vLHCathome-dev \| No tasks sent 10/05/2016 09:36:50 \| vLHCathome-dev \| No tasks are available for Theory Simulation 10/05/2016 09:36:50 \| vLHCathome-dev \| This computer has finished a daily quota of 1 tasks I assume this is a result of the invalid tasks being reported due to the system failure to get x509 credential. Is this what you expect to happen regarding host backoff ? Does this not mean if every user gets this system error all machines will be backed off for a day ? ID: 3300 · Rating: 0 · rate: / Reply Quote

PDW Send message Joined: 20 May 15 Posts: 217 Credit: 5,968,227 RAC: 0	Message 3301 - Posted: 10 May 2016, 8:59:28 UTC - in response to Message 3300. Gave LHCb a go and that fails also (as I expected) but did see this in the output after the 'Could not get an X509 credential' message... /usr/sbin/boinc-shutdown: line 31: [: too many arguments Perhaps some sort of anger management is required ? ID: 3301 · Rating: 0 · rate: / Reply Quote

PDW Send message Joined: 20 May 15 Posts: 217 Credit: 5,968,227 RAC: 0	Message 3306 - Posted: 10 May 2016, 21:44:41 UTC - in response to Message 3301. LHCb task running okay now, will be tomorrow before I can run a Theory task ! ID: 3306 · Rating: 0 · rate: / Reply Quote

Crystal Pellet Volunteer tester Send message Joined: 13 Feb 15 Posts: 1188 Credit: 857,561 RAC: 22	Message 3730 - Posted: 18 Jul 2016, 7:10:43 UTC Last modified: 18 Jul 2016, 7:11:20 UTC Started a new task and got no credentials for running a Theory mt-mcore VM - 206 (0xce) EXIT_INIT_FAILURE 2016-07-18 07:37:48 (5740): Guest Log: [INFO] Reading volunteer information 2016-07-18 07:37:48 (5740): Guest Log: [INFO] Volunteer: Crystal Pellet (38) Host: 37 2016-07-18 07:37:48 (5740): Guest Log: [INFO] VMID: f1b43d2f-577d-4de1-aea7-eac1787823b8 2016-07-18 07:37:48 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:37:48 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:38:19 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:38:19 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:38:49 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:38:49 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:39:19 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:39:19 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:39:49 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:39:49 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:40:29 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home 2016-07-18 07:40:29 (5740): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev 2016-07-18 07:40:59 (5740): Guest Log: [ERROR] Could not get an x509 credential 2016-07-18 07:40:59 (5740): Guest Log: [ERROR] The x509 proxy creation failed. 2016-07-18 07:40:59 (5740): Guest Log: [INFO] Shutting Down. ID: 3730 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1067 Credit: 334,882 RAC: 0	Message 3731 - Posted: 18 Jul 2016, 8:09:16 UTC - in response to Message 3730. I confirm there is a problem with the authentication server and am working to fix it. ID: 3731 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1067 Credit: 334,882 RAC: 0	Message 3732 - Posted: 18 Jul 2016, 8:56:35 UTC - in response to Message 3731. Last modified: 18 Jul 2016, 9:59:28 UTC The online CA that provides the user certificates to the proxy server is down. We are working to bring it back up ASAP. ID: 3732 · Rating: 0 · rate: / Reply Quote

ivan Volunteer moderator Project administrator Project developer Project tester Project scientist Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,134,656 RAC: 697	Message 3733 - Posted: 18 Jul 2016, 9:41:10 UTC - in response to Message 3300. The host is now told that it has completed its quota for the day... 10/05/2016 09:36:49 \| vLHCathome-dev \| Sending scheduler request: Requested by user. 10/05/2016 09:36:49 \| vLHCathome-dev \| Requesting new tasks for CPU 10/05/2016 09:36:50 \| vLHCathome-dev \| Scheduler request completed: got 0 new tasks 10/05/2016 09:36:50 \| vLHCathome-dev \| No tasks sent 10/05/2016 09:36:50 \| vLHCathome-dev \| No tasks are available for Theory Simulation 10/05/2016 09:36:50 \| vLHCathome-dev \| This computer has finished a daily quota of 1 tasks I assume this is a result of the invalid tasks being reported due to the system failure to get x509 credential. Is this what you expect to happen regarding host backoff ? Does this not mean if every user gets this system error all machines will be backed off for a day ? Sorry for not spotting that sooner, but luckily Laurence was ahead of me... Yes, that is what BOINC does when it spots errored-out tasks. Unfortunately, since we have a small limit anyhow the quota bombs out quickly. And, yes, a lot of hosts will be without tasks for up to a day if they quotaed-out. I've got one host that's dry and another running one task where it normally has two (one -dev, one production). ID: 3733 · Rating: 0 · rate: / Reply Quote

Rasputin42 Volunteer tester Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0	Message 3734 - Posted: 18 Jul 2016, 9:49:23 UTC anyhow the quota bombs out quickly I thought, if you produce a number of valid tasks, this builds up to counter the quota.Apparently, that does notwork or i am wrong. ID: 3734 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1067 Credit: 334,882 RAC: 0	Message 3735 - Posted: 18 Jul 2016, 10:03:19 UTC - in response to Message 3734. I have just increased the limit. Will decrease it again tomorrow. ID: 3735 · Rating: 0 · rate: / Reply Quote

ivan Volunteer moderator Project administrator Project developer Project tester Project scientist Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,134,656 RAC: 697	Message 3736 - Posted: 18 Jul 2016, 10:34:41 UTC - in response to Message 3734. Last modified: 18 Jul 2016, 10:36:21 UTC anyhow the quota bombs out quickly I thought, if you produce a number of valid tasks, this builds up to counter the quota.Apparently, that does notwork or i am wrong. Yes, but if you are at the point that your quota has run out, you need to wait up to one day before you can get another task, to start building up your quota again. Anyway, the graph of running jobs is starting to rise again ID: 3736 · Rating: 0 · rate: / Reply Quote

Rasputin42 Volunteer tester Send message Joined: 16 Aug 15 Posts: 966 Credit: 1,211,816 RAC: 0	Message 3737 - Posted: 18 Jul 2016, 10:44:45 UTC - in response to Message 3736. Well, if you have done 20 consecutive valid task before, you should be able to produce 20 failed tasks+ quota, before you are not allowed any more tasks. Credentials are working, but i cannot get any job (CMS task). ID: 3737 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1067 Credit: 334,882 RAC: 0	Message 3738 - Posted: 18 Jul 2016, 11:01:03 UTC - in response to Message 3737. Yes, but if 20 tasks have failed, then the no more tasks will be given. I haven't found the magic flag in the DB to reset this. ID: 3738 · Rating: 0 · rate: / Reply Quote

Development for LHC@home