Science Gateways for Principal Investigators
This section brings together information required by principal investigators (PIs) of XSEDE allocations to plan, organize, and oversee an XSEDE Science Gateway. Content on this page focuses on decision-making and management. For information on developing and operating a science gateway, see Science Gateways for Developers and Administrators.
Science gateways provide communities of users with simplified mechanisms for accessing scientific applications, tools, and data. By using XSEDE, gateway providers can obtain access to a wide range of computational resources professionally managed by XSEDE service providers. XSEDE resources may allow gateway providers to bring their applications and data to a wider community, to free up their own departmental or campus resources that have become overwhelmed by community usage, or to acquire access to fundamentally different systems than are available at their home institutions or through commercial cloud providers.
Are You Eligible?
XSEDE Science Gateways are community provided resources. PIs must have an active XSEDE allocation in order to provide a functioning science gateway. Eligibility of a PI for sponsoring an XSEDE Science Gateway is the same as for an XSEDE allocation; in brief, you must be a U.S. researcher, or collaborating with a U.S. researcher. For details on eligibility guidelines, see NSF Resource Allocations Policies: Eligibility. For more eligibility details please view the Allocations section. PIs should also take responsibility for managing renewals, supplements, and extensions for their gateways to ensure continuous access to XSEDE resources.
Access Diverse Resources
XSEDE maintains a diverse set of resources through its constituent service providers. These include some of the largest academically operated supercomputers in the country, high throughput resources optimal for running large numbers of serial jobs, high memory and I/O optimized resources, storage-optimized resources, cloud computing resources, and hybrid CPU/co-processor machines. To support science gateways, XSEDE service providers offer both cloud computing resources and novel, hybrid clusters that support the direct hosting of gateways and other persistent services that are co-located with more conventional, batch scheduling clusters. Thus a gateway's web server, database and other services could be located at the PI's home institution and access XSEDE resources remotely, or it could run entirely in an XSEDE allocated academic cloud, or could it could run in a virtual machine or virtual cluster that has direct access to a co-located standard cluster with a shared file system for closer integration.
Get Support for Your Gateway
XSEDE supports science gateways directly through its Extended Collaborative Support Services (ECSS) program. PIs can include ECSS support in their startup and XRAC allocation requests, and they can add ECSS support to existing allocations through a supplemental request. See XRAS - Submit Allocation Requests. XSEDE recommends that all gateway requests include ECSS support. ECSS consultants will help you comply with XSEDE policies and make the best use of XSEDE resources as well as help with technical integration.
Join the Science Gateway Community
The XSEDE gateway program fosters the gateway community. PIs, their developers, and their gateway operators join the XSEDE gateway community mailing list. This list can be used to request community help or input on best practices. It can also be used to ensure that everyone on the PI's team receives important XSEDE announcements.
To subscribe to the mailing list, send an email with "subscribe gateways" in the message body.
Participate in Workshops and Conferences
PIs and developers can participate in attendance at Gateway-related workshops and meetings and annual PEARC conferences to publicize their work and stay abreast of the current state of practice. Gateway- and XSEDE-related conferences are announced in XSEDE User News under the Science Gateway category. PIs and their team members can manage their subscriptions at to this service at Manage XSEDE User News Email Subscription.
Find Third Party Service Providers and Consultants
XSEDE provides infrastructure for its users, including gateways, but relies in large part on community-provided middleware. Thus, there are many ways to build gateways. The PI may choose an entirely do-it-yourself approach, especially if integrating an already successful gateway with XSEDE. At the other end of the spectrum, there a number of third party science gateway framework and middleware providers that the PI can work with to build the gateway; these may provide solutions for a wide range of of problems, including user management, integration with diverse XSEDE resources, data management, and so on. The PI can also choose a middle path, selecting services from third party providers for reliable high performance data movement and authentication. In all these cases, XSEDE ECSS consultants are available to the PI to help choose and implement the right solutions. In addition to middleware services, the Science Gateways Community Institute (SGCI) is an NSF-funded project that acts as an XSEDE Service Provider. The PI may work with the SGCI to develop additional aspects of the Science Gateway that go beyond the scope of XSEDE integration. The SGCI is also able to help the PI with complementary services such as sustainability planning.
Is a Science Gateway Right for Your Community?
Before taking the first steps to start building a gateway, PIs need to assess if a gateway will add value for their research communities. A number of factors may define the usefulness of a gateway or portal. For example, you may intend to grow your user community significantly, or you may intend to develop complex workflows for only a few advanced scientists, but provide them with tremendous capabilities. The expected size of your user community and the anticipated scientific impact should justify the investment in building a gateway.
Successful Gateway Examples
XSEDE supports a diverse set of science gateways, and more users run jobs on XSEDE-provided resources by using science gateways than by logging in. For a list of active XSEDE gateways with current allocations, see Science Gateways Listing. Note that XSEDE science gateways are community-provided. XSEDE does not preclude multiple gateways from supporting the same fields of research or even the same applications. Likewise, XSEDE is very interested in expanding the gateway community offerings to include fields not represented by the current listings.
Computational Expertise and Culture of Your Community
Lack of expertise or infrequency of use of high-performance computing (HPC) resources can present obstacles to researchers trying to accomplish their computational goals in a timely manner. Your research audience may not be familiar with the Unix command line or with the complexity of optimization parameters of analytical or domain-specific computational software. Field or bench researchers may spend months obtaining experimental or observational data and only a few weeks during a grant cycle analyzing their data on computational resources. Others may be accustomed to working only on local clusters. For any of these potential users, a portal may facilitate their computational objectives.
Data and Storage
A portal can meet the need to stage, store, and share large quantities of data or to bring data together from diverse storage systems.
Goals or Problems to Be Solved
Develop a clear idea of what, specifically, would constitute success in your gateway. Many of the qualitative criteria that you define to help you decide whether a gateway is the right solution for your community's needs may turn into quantitative success metrics later. This would also be something you describe in an allocation request for XSEDE resources. To learn more about success metrics, see the section below.
Role of XSEDE in Your Gateway
When evaluating whether to develop or extend a portal to connect to XSEDE resources, consider the advantages of using high-end resources, since extra labor is required for XSEDE integration. What scientific questions can be answered through the use of XSEDE resources that couldn't be answered otherwise? How do XSEDE capabilities add value to the scientific community? What specific bottlenecks or problems will be addressed through the use of XSEDE resources? Once you've decided to create a portal, answers to these questions will help your development team to design an effective gateway that meets the specific needs of the scientists who will use it.
Measuring the Success of Your Science Gateway
XSEDE measures its success through a number of metrics, including the number of users, the amount of resources requested and delivered, and the number of supported publications. It in turn asks its users, including science gateway providers, to help provide these metrics. It required that science gateway projects use scripts that count the users running jobs on XSEDE resources; see Gateways for Developers for more information. As a gateway provider, accurately measuring the impact of your gateway will assist in the justification for resources in allocation requests. Supported scientific publications are also important to track. XSEDE encourages all PIs to upload publication information to its publications system; Gateway PIs should do the same. Keeping this up to date in the XSEDE publication system simplifies the allocation renewal process and helps with successful allocation renewals. Collecting publications supported by gateways is not required but highly encouraged; many successful gateways maintain their own databases of publications. XSEDE recommends that gateways adopt a simple citation and acknowledgement policy for publications by the gateway's users. This gives the gateway mechanisms for reliably and efficiently collecting this information
Once a PI obtains an XSEDE allocation for XSEDE resources, he or she can create and connect a gateway to XSEDE. Ultimately, it is the PI's responsibility to make sure that the gateway development team follows XSEDE requirements and practices. There are a few specific obligations defined below.
Obtain an Allocation and Monitor Usage
Using XSEDE resources requires an allocation; allocation policies are described at XSEDE Allocation Policies. PIs who are new to this process are recommended to initially request a startup allocation. Larger allocation requests require XRAC proposals; see Write a Winning Gateway Proposal and Successful Research Allocation Requests for guidelines on writing a successful proposal. Note also that the XSEDE gateway program staff are available to advise; contact them through the XSEDE Help Desk. The PI should monitor usage of the allocation or delegate this responsibility to her/his gateway administrator by adding the administrator to the allocation using the "allocation administrator" role. See Manage Your Projects and Allocations. Monitoring usage can be done through the XSEDE User Portal. The PI can request an extension on unused allocations that are expiring. See Renew or Extend Allocation Duration. PIs who wish to add new resources to their allocations can request a supplemental allocation. A supplemental request is also the mechanism for adding an ECSS consultant to the allocation. PIs may transfer allocations between XSEDE resources. All these actions are done through the XSEDE User Portal.
Report Usage Quarterly
Gateways must report the number of unique users who accessed XSEDE resources through the gateway each quarter. As described above, XSEDE provides scripts that automate this.
Display the XSEDE Logo
Please display the XSEDE logo on your site. Many gateways place it in the footer. Please look on the Logo page for XSEDE Logos.
Cite XSEDE in Publications
For guidelines on acknowledging XSEDE allocations, citing XSEDE, and citing the XSEDE ECSS program, see How to Acknowledge XSEDE. Gateways may also encourage their users to use one of these mechanisms.
Security and Accounting
PIs are responsible for securely operating their gateways and notifying XSEDE staff about any security incidents which have the potential to impact XSEDE. Proper accounting is required for fulfilling security and reporting requirements, as well as for justifying subsequent allocation renewals. Because the gateway maintains control of community accounts, the gateway PI must ensure that NSF computational resources are used in a manner consistent with the award and that reasonable attempts have been made and tools installed to ensure appropriate usage. This includes monitoring of all gateway usage by the community. Specifics are detailed in the user responsibility form; see XSEDE Usage Policy. Our experience in working with gateways has led to a list of recommendations for best practices to adopt in the implementation and management of your gateway to ensure appropriate use. These recommended practices and a full listing of security and accounting requirements are outlined in the Gateways for Developers section. For community accounts, risks may be greater and usage patterns less certain, so additional information is required to help prevent security incidents. This information must be submitted before jobs can be run externally via community accounts. Additionally, XSEDE Service Providers may choose to impose their own limitations on community accounts through operating system tools, restricted shells or other means.
Stay Informed Through XSEDE User News
Subscription to the Gateway news category in XSEDE User News by at least one person on your gateway team is strongly encouraged. XSEDE staff post information about XSEDE activities and resources that impact gateways in this category. News postings can be received by email, by RSS feed, or by monitoring the News Web site. Your developers should subscribe to news categories for the XSEDE resources you intend to use. Additional methods for staying informed and communicating with the gateway community are listed below. These opportunities are optional.