User News

Stay up to date with up to the minute news from XSEDE and XSEDE User Portal. Subscribe for email notifications.

Key Points
Newsfeed
Breaking user information
Contact Information

SDSC Comet maintenance 10AM-5PM(PT), Tue, 01/16/2018 (update)

Outage start
01/16/2018 10:00 PST
Anticipated end
01/17/2018 01:00 PST
Outage type
Full outage

Update 4

Posted by Mahidhar Tatineni on 01/17/2018 20:35 UTC

The Lustre OSS issue has been fixed and the /oasis/projects/nsf filesystem is now back in production. We have released jobs that were held last night and Comet is running jobs as scheduled. Please email help@xsede.org if you have any questions.

Update 3

Posted by Mahidhar Tatineni on 01/17/2018 08:56 UTC

The /oasis/scratch/comet filesystem is now back in production. One of the object storage servers (OSSs) in the /oasis/projects/nsf filesystem is still unavailable and we continue to work on it. At the moment we have deactivated the object storage targets (OSTs) associated with this OSS. Files in /oasis/projects/nsf that are associated with these OSTs are temporarily unavailable. We will update once this OSS is fixed and the files become available again.

The maintenance reservation has been released to allow jobs that use /oasis/scratch/comet or the NFS based directories to run. Jobs with /oasis/projects/nsf dependencies are currently being held and will be released once we fix the OSS issue. If you have jobs with /oasis/projects/nsf requirements, please hold off on submitting new jobs. We will update once the filesystem is back to full availability and release the jobs. Please email help@xsede.org if you have any questions.

Update 2

Posted by Mahidhar Tatineni on 01/17/2018 04:17 UTC

We are extending the maintenance window on Comet to fix an issue with one of the Lustre projects (/oasis/projects/nsf) filesystem object storage servers (OSSs). The reservation to prevent jobs from running has also been extended. We will update via a news post once we return Comet to production. Please email help@xsede.org if you have any questions.

Update 1

Posted by Mahidhar Tatineni on 01/16/2018 22:36 UTC

The ongoing Lustre filesystem maintenance on Comet is taking slightly longer than expected. We are extending the maintenance window by a few hours and will update via a news post once it is complete. Please email help@xsede.org if you have any questions.

Original post

Posted by Mahidhar Tatineni on 01/12/2018 15:42 UTC

We will be performing maintenance on the Comet Lustre scratch filesystem 10AM-2PM (PT) on Tue 01/16/2018. We have a reservation in place to prevent jobs from running during the maintenance. Any jobs that don’t fit in the time window before the maintenance will run after the maintenance is complete. For jobs that don’t fit in the window before the maintenance, the squeue command will report “ReqNodeNotAvailable”. These jobs will run fine after the maintenance is complete. Please email help@xsede.org if you have any questions.