Friday, July 29, 2016

***Updated #3 - Resolved - 9:10pm*** PLU Server Systems Offline due to storage disruption - Friday, July 29, 2016 12:32pm

WHAT: Several PLU Server Systems have gone offline unexpectedly

WHEN: 07/29/2016  Duration: Beginning at 12:32pm

WHERE: Campuswide

Current Status: Resolved via workaround

Description:
*** Update #3 - 9:10pm***  All servers and services should be restored to full access through the SSO system.  A workaround has been implemented to resolve the outstanding issue with the authentication system.  No additional work on this is planned through the weekend.  The I&TS System team will monitor services the rest of the weekend and will convene a review and assessment effort on Monday morning to determine next-steps to remove the workaround and ensure continued service.

*** Update #2 - 5:30pm***  All servers have received preliminary assessment and file system issues have been resolved wherever identified.  Unfortunately, the Single-Sign-On system that provides access to many of our services (e.g. Sakai, Google Apps, Banner SSB, etc.) is failing to operate properly.  The I&TS Systems team is continuing to actively troubleshoot this issue including restoring system copies from the nightly backups and using every other available tool to restore service as soon as possible.

***Update #1 - 1:20pm*** Storage system access is restored.  Assessment of affected servers is underway to determine if restarts will be necessary to restore services.

***Original Post***  It appears a support technician from one of our equipment vendors made a mistake in working on system configuration troubleshooting.  A configuration change made caused many of our servers to lose access to their storage, thereby disrupting operations.  Emergency work is underway to reestablish connections and return all services to full operation.

Service(s) Impacted:  Potentially all campus server services

Next Update: Possible followup information to be shared following review meeting on Monday..

Root Cause / Continuous Improvement: To be determined.

Check http://www.plu.edu/status/ for periodic updates.

Monday, July 18, 2016

***COMPLETED*** Planned Campus-wide Network Service Maintenance - Saturday, July 23rd - 7:00am-7:00pm

WHAT: ***COMPLETED***Planned Campus-wide Network Service Maintenance - Saturday, July 23rd - 7:00am-7:00pm

WHEN: 07/23/2016 Duration: 7:00AM to 7:00PM (estimated)

WHERE: PLU Campus Network

Current Status:

Description: *Update - July 23rd, 8:00pm*
Network maintenance work has been completed.  As of 8pm all services appear to be up and running properly.  No additional service impact is anticipated at this time.

*Original Post*
On Saturday, July 23rd Information & Technology Services will be performing maintenance on core network and server equipment.  During this time there will be a series of intermittent outages affecting most of the services we offer, including Sakai, Banner, PLU web (www.plu.edu), Library, single-sign-on authentication services (for Google Apps, Salesforce, etc.), and many others.

The work will begin as early as 7:00AM on Saturday morning with the primary disruption occurring earlier in this maintenance period.  Most services will be restored by 2:00pm and all work is anticipated to be completed by 7:00PM on Saturday evening.

If you have any questions, please call the I&TS Help Desk at (253) 535-7525 or send an email to helpdesk@plu.edu

Thank you for your patience during this essential maintenance work.

David P. Allen
Director/I&TS Enterprise Systems

Service(s) Impacted: Campus Network & Internet Access and all PLU hosted services.

Next Update: No additional posts are anticipated.

Check http://www.plu.edu/status/ for periodic updates.

Tuesday, July 12, 2016

Sakai Unavailable for Maintenance on Friday, July 15, 6AM PT

*** UPDATE *** July 15, 6:15AM PT - Sakai is back online. The planned maintenance is complete.

*** UPDATE *** July 15, 6:00AM PT - Sakai is currently offline for planned maintenance. The Sakai Support team is working to restore this service by 6:30AM (if not before).

WHAT: Sakai Unavailable for Maintenance

WHEN: 07/15/16 Duration: 06:00AM - 06:30 AM PT

Description: On Friday, July 15th, Sakai will be out of service from 6:00AM to 6:30AM PT for maintenance. Please plan accordingly. If you have any questions or concerns please email sakai@plu.edu.

Service(s) Impacted: Sakai

Estimated Time to Resolution or Next Update: 07/15/16 06:00 AM



Check http://www.plu.edu/status/ for periodic updates.

Monday, July 11, 2016

***Completed*** WorldCat and Classic Catalog Outage

WHAT: Ex Libris security patching, causing an outage for a few affected services

WHEN: 07/12/16, 7:00AM  Duration: One Hour

Current Status: Open

Description: Ex Libris, our Voyager library vendor, has notified us of a security patch that needs to be installed. While the security patch is being installed the following with be unavailable:

  • Voyager clients (e.g., circulation, cataloging, acquisitions) 
  • Classic catalog 
  • WorldCat Local can be searched but our local holdings will not be displayed
Service(s) Impacted: Library

Check http://www.plu.edu/status/ for periodic updates.

Tuesday, July 5, 2016

*Complete* Planned Internet Service Disruption - July 8th - Lasting approximately 30 Minutes between 1-5am

WHAT: Primary ISP for PLU will be performing maintenance on our connection.

WHEN: 7/8/16 Duration: Approximately 30 minutes between 1:00AM-5:00AM

WHERE: PLU Campus

Current Status: Closed

Description: The primary ISP for the PLU campus has notified us that their circuit provider will be performing "service-impacting maintenance" early the morning of July 8th.  The downtime is expect to be 30 minutes and will occur between 1:00AM and 5:00AM that morning.

Service(s) Impacted: Campus Network

Workarounds: Our firewalls may be able to re-route traffic to our backup connection, but some intermittent disruption would still occur at the beginning and end of the downtime.

Next Update: Following completion and confirmation of ISP work.

Check http://www.plu.edu/status/ for periodic updates.