5m of downtime for about a dozen customers

We experienced about 8 minutes of downtime about 30 minutes ago when a network problem killed NFS shares on all 12 servers in our clusters 7, 8, and 9.

We were notified of the problem immediately and got it fixed quickly.

We’re still not sure exactly why that network problem caused the failure; we’ll investigate it to see whether there’s something we can do it prevent this problem in future.