GPFS data migration to run most weekends, starting tonight
Beginning tonight we will kick off a data migration between different pools of storage in our GPFS23 filesystem (/scratch and /data). Hot data (files that have been accessed more recently) will be moved to faster, more expensive storage while cold data (files that have not been accessed recently) will be moved to more economical storage hardware.
This migration actually runs most weekends, and can make doing interactive work on the cluster more sluggish than normal, so please plan accordingly. The migration generally takes 1-2 days to complete, depending on the amount of data to be moved.
On the subject of storage, we are also excited to share that we have recently purchased a large amount of new hardware that we expect to greatly improve the overall performance of our GPFS-based filesystems. It will be several months before the new hardware is deployed in our environment, but we think it will make a big difference.
In the meantime, we will continue working to ensure that filesystem performance is as reliable and performant as possible, while also helping users modify their workflows to ensure that interactive usage of the cluster remains snappy.