Skip to main content

ACCRE networked storage connectivity

Posted by on Friday, December 11, 2020 in Website.

Update, 12/12: We were able to identify the source of the problem that was preventing the export services from operating normally. An update was applied just afternoon today and that has cleared the errors that were causing the NFS service to crash. The workstations of one of the groups that use the service got into a state that caused them to flood the export servers with bad requests.

The service is restored and operating on one of the three servers. Several user groups have already confirmed that they are able to connect normally. ACCRE is working on adding the other servers to the rotation and exploring ways to prevent this type of disruption.

Update, 12/10 3pm: We have re-established SMB connectivity to the storage system using one of the 3 export service servers, but not NFS. Enabling NFS causes all three of the servers to lose their network settings. This in turn breaks SMB connectivity and prevents the export service servers from working as a cluster.

For now, SMB connections will continue to be handled by a single server. ACCRE sysadmins and the vendor are actively troubleshooting the NFS issue. When the solution is identified, we expect all user sessions will be disconnected during its implementation.

For users with ACCRE accounts, all the storage is still accessible via ssh connections to the gateways and can be mounted using sshfs.

Update, 12/10 12pm: We have restored connectivity on one of the 3 servers in the export services cluster. We are currently working on getting the other two servers to sync in order to recover load balancing capabilities. So while you may experience fewer connectivity issues, performance will be limited.

Original post: Connectivity to ACCRE’s networked storage is very spotty due to the export service component refusing connections. We are actively troubleshooting to stabilize the connectivity. Access to all ACCRE storage via ssh is working normally. We will provide an update at 11:30am.