WHEN: 07/29/2016 Duration: Beginning at 12:32pm
WHERE: Campuswide
Current Status: Resolved via workaround
Description:
*** Update #3 - 9:10pm*** All servers and services should be restored to full access through the SSO system. A workaround has been implemented to resolve the outstanding issue with the authentication system. No additional work on this is planned through the weekend. The I&TS System team will monitor services the rest of the weekend and will convene a review and assessment effort on Monday morning to determine next-steps to remove the workaround and ensure continued service.
*** Update #2 - 5:30pm*** All servers have received preliminary assessment and file system issues have been resolved wherever identified. Unfortunately, the Single-Sign-On system that provides access to many of our services (e.g. Sakai, Google Apps, Banner SSB, etc.) is failing to operate properly. The I&TS Systems team is continuing to actively troubleshoot this issue including restoring system copies from the nightly backups and using every other available tool to restore service as soon as possible.
***Update #1 - 1:20pm*** Storage system access is restored. Assessment of affected servers is underway to determine if restarts will be necessary to restore services.
***Original Post*** It appears a support technician from one of our equipment vendors made a mistake in working on system configuration troubleshooting. A configuration change made caused many of our servers to lose access to their storage, thereby disrupting operations. Emergency work is underway to reestablish connections and return all services to full operation.
Service(s) Impacted: Potentially all campus server services
Next Update: Possible followup information to be shared following review meeting on Monday..
Root Cause / Continuous Improvement: To be determined.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.