XCRI's Mission

The XSEDE Cyberinfrastructure Resource Integration group is dedicated to providing tools that will help campus cyberinfrastructure staff more easily manage their local computing and storage resources.

On the following three points, there is nearly universal agreement

  • There are not enough people on campuses to meet all the demands for supporting researcher use of advanced cyberinfrastructure
  • There is simply not enough computational power in the US to support all of the meaningful and important research
  • The people on campuses managing local campus resources have too much to do and too little time in which to do it.

The XSEDE Cyberinfrastructure Resource Integration group is dedicated to providing tools that will help campus cyberinfrastructure staff more easily manage their local computing and storage resources.

To dig a bit deeper into the challenges we face as a research community, let's look at some of the causes of the situation we have today (beyond the issue of simply not enough funding to meet demand):

  • Many sysadmins are doing tasks locally, by hand, in ways that are often re-invented over and over again, when those tasks could largely be made more consistent across campuses and automated
  • When users switch between local computing environments and national resources such as those supported by XSEDE, it often feels very disconcerting – while systems are advertised as being similar
  • Things like writing documentation are hard, often the last things done when building a system, and because of lack of staff it is often the case that busy sysadmins are asked to write documentation for end users
  • The state of software for managing and using our many and diverse cyberinfrastructure resources is not good enough and does not effectively support resource sharing

XCRI's goal is to provide as much aid to the US research community on the above four challenges as possible.

XCRI Toolkits and services include

  • The XSEDE-Compatible Basic Cluster (XCBC) software toolkit enables campus CI resource administrators to build a local cluster from scratch, which is then easily interoperable with XSEDE-supported CI resources. XCBC is very simple in concept: pull the lever, have a cluster built for you complete with an open source resource manager / scheduler and all of the essential tools needed to run a cluster, and have those tools set in place in ways that mimic the basic setup on an XSEDE-supported cluster. The XCBC is based on the OpenHPC project, and consists of XSEDE-developed Ansible playbooks and templates designed to ease the work required to build a cluster. Consult the XSEDE Knowledge Base for complete information about how to use XCBC to set up a cluster.
  • The XSEDE National Integration Toolkit (XNIT). Suppose you already have a cluster that you are happy with and you want to add too it software tools that will allow users to use open sources software like that on XSEDE, or other particular pieces of software that you think are important, but you don't want to blow up your cluster to add that capability? XNIT is for you. You can add all of the basic software that is in SCBC, as relocatable RPMs (Resource Package Manger), via a YUM repo. (YUM Stands for Yellowdog Updater, Modified). The RPMs in XNIT allow you to expand the functionality of your cluster, in ways that mimic the setup on an XSEDE cluster. XNIT packages include specific scientific, mathematical, and visualization applications that have been useful on XSEDE systems. Systems administrators may pick and choose what they want to add to their local cluster; updates may be configured to run automatically or manually. Currently the XNIT repository is available for x86_64 systems running CentOS 6 or 7. Consult the XSEDE Knowledge Base for more information.
  • Optional software that can be added to clusters running XCBC or XNIT is available. This includes a toolkit for installing a local Globus connect server. Globus transfer is the recommended method for transferring data to any XSEDE system.
  • XCRI staff will travel in person to your campus to help implement XNIT, XCBC, or any other XCRI tools on your campus. After an initial phone consultation, we can assist onsite with configuration and ensure that you have the knowledge that you need to maintain your system. That's right…. XSEDE will pay to fly XSEDE staff to your campus and help you with your campus cluster, even if you have no particular relationship to XSEDE. You can see the list of places we have gone to give talks or help people set up clusters. You can read detailed descriptions of past visits to campuses to help with local clusters in the XSEDE 2016 paper Implementation of Simple XSEDE-Like Clusters: Science Enabled and Lessons Learned. These site visits are funded by XSEDE, including staff travel and lodgings.
  • XCRI offers a Cluster Monitoring toolkit - compatible with the XCBC or any OpenHPC cluster running Warewulf, that allows sites to install tools that monitor both cluster health and job statistics. The toolkit allows administrators to generate fine-grained reports of usage based on users, projects, and job types, which can be a great aid in keeping track of ROI or justifying future funding.

XCRI Impact

We're having an impact on the national CI ecosystem, as evidenced by the following statistics from the first five years of XSEDE:

  • Total aggregate TeraFLOPS of systems running XCBC and / or XNIT: 732
  • Total number of CI resource on which one or more XCI tools are used: 594
  • Total PetaBytes of files moved via Globus transfer since the start of XSEDE: 148
  • Total number of partnership interactions between XCRI and SPs, campus and national CI providers since the start of XSEDE: 72

A YouTube library of videos about XCI and its organizational forerunner the XSEDE1 Campus Bridging Group is available.

For a look at how XCRI's toolkits have helped campuses put resources in researchers' hands, see the YouTube video XSEDE Helps Small University Connect With Nation's Cyberinfrastructure.

Key Points
Shortage of time and resources to meet research demands
Causes of the challenges we face
XCRI's goal is to help the US research community adapt and leverage resources in response to these challenges
Contact Information