Disruption on Wednesday

Author	Message
Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3523 - Posted: 31 May 2016, 21:47:24 UTC On Wednesday we will be migrating the workloads to a new Condor server so that we are ready for the release on Monday. The migration will hopefully be transparent but don't be surprised if it isn't. ID: 3523 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3524 - Posted: 1 Jun 2016, 9:01:22 UTC - in response to Message 3523. The switch has been made so new tasks from now on will get jobs from the new server. ID: 3524 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3528 - Posted: 1 Jun 2016, 15:16:42 UTC - in response to Message 3524. It seems that nobody has noticed but we have just realized that the opening the external firewall for the new servers hasn't been done. It is being done now and they should be open within an hour or so. ID: 3528 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3529 - Posted: 1 Jun 2016, 21:46:53 UTC - in response to Message 3528. The firewall is open but we are still having connectivity issues. It seems to be working internally but not at home. ID: 3529 · Rating: 0 · rate: / Reply Quote

Ben Segal Volunteer moderator Volunteer developer Volunteer tester Send message Joined: 12 Sep 14 Posts: 65 Credit: 544 RAC: 0	Message 3531 - Posted: 2 Jun 2016, 8:17:57 UTC - in response to Message 3529. Last modified: 2 Jun 2016, 8:22:58 UTC The firewall is open but we are still having connectivity issues. It seems to be working internally but not at home. You are right… I'm getting from home: 06/02/16 10:18:47 Changing activity: Benchmarking -> Idle 06/02/16 10:20:41 attempt to connect to <128.142.141.53:9618> failed: Connection timed out (connect errno = 110). Will keep trying for 300 total seconds (172 to go). ID: 3531 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3532 - Posted: 2 Jun 2016, 10:22:43 UTC - in response to Message 3531. Last modified: 2 Jun 2016, 10:22:58 UTC Thanks, We are still fighting. I am working from home to try and debug the issue. ID: 3532 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3533 - Posted: 2 Jun 2016, 16:53:02 UTC - in response to Message 3532. The migration has been abandoned for now and we have reverted back to the old server. There seems to be an issue with the new setup that will require further investigation. ID: 3533 · Rating: 0 · rate: / Reply Quote

Laurence Project administrator Project developer Project tester Send message Joined: 12 Sep 14 Posts: 1069 Credit: 334,882 RAC: 0	Message 3535 - Posted: 3 Jun 2016, 22:12:32 UTC - in response to Message 3533. For those of you who are curious about what we were fighting with, here is an example. The following is some output from running various commands on a CentOS7 machine with Condor v8.3.8. [root@condor-test-cc7 ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28 [root@condor-test-cc7 ~]# condor_status Error: communication error CEDAR:6001:Failed to connect to <128.142.201.216:9618> [root@condor-test-cc7 ~]# setenforce 0 [root@condor-test-cc7 ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28 [root@condor-test-cc7 ~]# condor_status [root@condor-test-cc7 ~]# [root@condor-test-cc7 ~]# sed -i 's/enforcing/disabled/g' /etc/selinux/config && reboot [root@condor-test-cc7 ~]# sestatus SELinux status: disabled [root@condor-test-cc7 ~]# condor_status Error: communication error CEDAR:6001:Failed to connect to <128.142.201.216:9618> ID: 3535 · Rating: 0 · rate: / Reply Quote

Development for LHC@home