Over the last weekend, on November 16th and 17th, the University of Redlands experienced a major internet failure that left many students unable to go online for some time.
“It was a catastrophic failure of some systems in our virtual computing environment,” said Director of Information Security (ITS) at the University of Redlands, Terry Reed.
He explained that it was not the university’s internet service providers that went offline, but the server systems responsible for providing IP addresses to devices. Without an IP address, wireless devices cannot access the internet.
According to Reed, the issue has been resolved over the weekend, with some functions such as the University’s Student Information System, Moodle and Public Safety being prioritized to get back up first.
All the server systems have now been restored, but he stated that there were “residual problems” that were still being fixed. One such problem is links not showing up or not directing users to the correct sites. These issues have since been corrected. At the moment, the root cause of the failure is still being investigated. ITS is working with their vendors to determine the cause.
Regarding the measures being taken to prevent a future occurrence, Reed referred to the University’s ITS “Incident Response Plan,” which outlines “Identification, investigation, recovery, and post-incident review.”
“We try to improve whatever deficiency that we have so that it doesn’t happen again in the future,” Reed said.
“We are taking a look at our virtual networks and how all our systems are set up,” Reed said. “They were redundant (duplicated) but both of them failed. We’re going to go back and see how we can do a better job with that redundancy, possibly putting them in two different virtual environments.”
The university community was notified of the system failure through a university-wide email on Monday. However, some students said they did not receive this email.
Commenting on this matter, Reed explained that the distribution group used to send the email may not have been up-to-date, which may have caused some people to be missed. ITS is reviewing this process to insure everyone receives University communications.
He also expressed his concerns about how students can be better informed.
“I’m not sure how students would communicate [any issues] back to us,” Reed said.
Reed hopes for “a way for [ITS] to make the process [of finding out] how we can communicate with students better.”
[hr gap=””]Photograph by Luis Chavez.
