Write a Winning Gateway Proposal
What is a Science Gateway?
The NSF supports community services as an appropriate use of resources. This usage scenario is described in the Resource Allocations Policies and is employed by the majority of today's science gateways.
Science Gateways provide access to national computation and data resources through interfaces that are customized to meet the needs of specific communities. Gateways interfaces are usually web portal or desktop client-server applications. Scalability to handle hundreds to thousands of users is enabled for many gateways through the use of a community account. Gateway users typically run their jobs under a single Unix username.
Although, some gateways may serve the needs of close collaborators, others may provide services for broad science communities. PIs may not be closely associated with those using the service. Because the gateway maintains control of the allocation, it is the gateway PI who is responsible for ensuring that the NSF computational resources are used in a manner consistent with the award and that reasonable attempts and tools have been installed to ensure appropriate usage, including monitoring for all usage of the gateway by the community. For more information on gateways, see the Gateways for PIs page.
The principal investigator (PI) who sponsors a gateway must write a request for the use of the resources and the induction of the interface into the Science Gateway program. The material on this page provides important tips for writing a successful gateway allocation proposal.
Proposal Review Criteria
The Allocations Policy describes the review criteria employed by the review committee and include:
- Computational Methodology: The choice of applications, methods, algorithms and techniques to be employed to accomplish the stated objectives should be reasonably justified. While the accomplishment of the stated objectives in support of the science is important, it is incumbent on proposers to consider the methods available to them and to use that which is best suited.
- Appropriate Use of Resources: The resources chosen must be an appropriate match for the applications and methodologies to be used and must be in accordance with the recommended use guidelines for those resources.
- Efficient Use of Resources: The resources selected must be used as efficiently as is reasonably possible. To meet this criterion, performance and parallel scaling data should be provided for all applications to be used along with a discussion of optimization and/or parallelization work to be done to improve the applications.
Qualities of a Winning Request
The best request documentation will:
- Summarize the research in the context of the current state-of-the-art, outline the computational algorithms to be used, and relate these algorithms to the research subsections.
- Provide sufficient information to reviewers, but not overwhelm with details.
- Provide a justification for the service unit (SU) request that is clear and closely coupled to the computational experiments. If resources become scarce and the committee needs to reduce the original request, with sufficient information, this reduction may be done fairly and rationally with minimum disruption to the investigator.
- Summarize results from previous allocations and relate those to the request in the current proposal.
I. Research Objectives and Metrics of Success
Reviewers will not reconsider the scientific merit of a funded scientific activity and therefore do not need to read the proposal which funded the science. However, they do want to ensure that funding exists to pursue these activities. Research activities should be described to the extent necessary to provide context for the request of resources and to explain how the use of these resources relates to the research objectives.
A Science Gateway may best be able to describe the classes of research activities that the proposed effort will support and how the capabilities of the Science Gateway will support these classes of research activities. Descriptions should be short, but provide enough detail to support the rest of the proposal. Descriptions of gateway research activities might include:
- Fundamental tools and capabilities being provided
- Types of investigations that are made possible by these capabilities
- Target audience for different capabilities
A gateway resource request should also include a definition of what constitutes success. For PIs with their own research group, this is easier since goals can be detailed in advance and the success of the investigations more easily measured. Reviewers need metrics to determine if a gateway has been successful with its use of resources when renewing proposals. Your proposal should contain those metrics that will be used to define success for your particular gateway, which might include any of the following:
- Publications citing the use of the gateway (This metric is very important.)
- New scientific discoveries or results enabled by the tools, software, and resources of the gateway
- Membership consisting of a large user population and increasing trends in new users
- Number of distinct institutions accessing the gateway
- Measurable value to education and outreach shown through number of classes or courses using a gateway
- Number of visits to the gateway and increasing use trends, such as page view statistics from Web logs
- Actions performed through the portal, such as number of data downloads or simulations that have been run
- Total and distribution of resource consumption
- Increased number of collaborations facilitated by the gateway
- Advancement of state-of-the-art capabilities for scientists
II. Codes and Methods
Many successful PIs start with a development allocation on platforms of interest and obtain performance data themselves. This type of data referenced in the full proposal gives reviewers additional confidence that a PI is ready to deploy an application in production on a larger scale.
The choice of applications, methods, algorithms, and techniques that will be employed to accomplish the stated objectives should be reasonably justified. Although the accomplishment of the stated objectives in support of the science is important, it is incumbent on proposers to consider the methods available to them and to use that which is best suited to their objectives. This is more significant if using "home-grown" codes. If using widely known, third-party codes (e.g., NAMD, CHARMM, AMBER), an explanation of the choice of code should be provided.
Describe why this code is a good fit for the resource(s) requested and/or list acceptable alternatives. For example a proposed usage of a computational resource with an expensive interconnect is not appropriate for single processor jobs. Successful proposals will include performance and scaling data on problems and test cases similar to those expected to be analyzed by the community.
III. Computational Plan
Explicitly describe the typical use cases that the gateway supports and the type of runs that individual users are expected to make. These can vary among targetted user audiences. For instance, a gateway might make small demonstration capabilities available to an undergraduate class and also make more sophisticated capabilities that consume larger amounts of resources available to identified high-end scientists. Describe very specifically how the gateway will help ensure that the community will make scientifically meaningful runs. Include codes used, problem sizes, and types of analysis capabilities to be provided through the gateway. For example, the gateway may store analyses so that computationally intensive simulations are not run multiple times.
Usage goals can vary substantially from one gateway to the next. For example, one gateway may provide introductory access to high-end capabilities for students. A measure of success of this gateway might be many end users, each using a small amount of resources. Another gateway might be providing advanced workflow capabilities to experienced scientists. In this scenario, a small number of scientists might be expected to consume most of the resources.
IV. Justification for SUs Requested
In traditional resource requests this is usually a straightforward calculation once the codes, methods, and computational plan are described. A clear presentation of the resources required for each problem set allows reviewers to allocate resources fairly to deserving proposals when resources are scarce.
However, because of the fluid access to gateways, developers may not be able to predict usage with a very high degree of accuracy. Because unused cycles mean that some other deserving proposal cannot obtain access to resources and, therefore, cannot obtain significant scientific results, large gateway requests should be based on realistic expectations. Estimates based on previous experience preferred. Conservative requests are preferred, if there is no way to make a realistic prediction based on experience; if a gateway uses more resources than anticipated during the allocation period, a supplemental request can be used to augment a conservative initial request.
In the first year of a gateway allocation request, a PI will want to conservatively estimate the size and type of the community, then calculate the anticipated runs by this community and tabulate SU estimates. A PI may expect a particular distribution between educational and high end users for example. Start modestly. For a newly deployed gateway, projected user community growth into the hundreds or thousands in the next year may not seem realistic to the review committee without significant justification. For subsequent year allocations, a gateway PI can start with an estimate of total usage based on prior year's usage patterns and estimate for the coming year's usage patterns.
Gateway developers have an additional responsibility to include load balancing plans in gateway development plans if very high usage is anticipated. While millions of jobs may constitute a successful gateway, care must be taken to show responsible use of the resources and efforts to minimize impact on other users. A successful proposal will include a description of how load balancing will be handled among sites and may also include a description of scaling issues that have been explored. For example, a PI expecting a very large number of jobs will want to show that the gateway infrastructure, as well as TeraGrid software such as Globus and Condor, are able to handle such a load.
V. Additional Considerations
Prior progress: A successful proposal will report on work done both locally and with prior allocations prior to proposal submission. For a gateway this may include a description of how a user community has expanded from a single institution to a nationwide community. It may describe how some researchers have moved from simple simulations to more complex studies, or what outstanding achievements have been accomplished through use of the gateway. A PI can describe how the metrics of success identified in section I of a previous allocation request have been met. This section is an opportunity to demonstrate to reviewers that previous awards did lead to scientific advances.
Availability of local or other computing resource: A gateway may describe how smaller jobs are sent to local resources while more computationally intensive jobs are sent to TeraGrid resources. In general, the proposal must explain how local resources are leveraged and explain why they are not leveraged if that is the case.
Community Support and Management Plan: A successful proposal will include a brief description of the gateway interface, as well as staff expertise in developing and maintaining this interface. A PI may include a description of how the gateway has been used for production work, relevant development efforts or a description of how the gateway helps the community use their SU allocation. A description of funding for the gateway development would be relevant.
User identification, usage tracking and restriction: Because gateway usage is typically collected under a single username on NSF resources, responsibility for tracking and restricting individual usage rests with the gateway developer. Gateways need a variety of capabilities to serve their individual users. These include registration, accounting, access control, activity monitoring and a mechanism for restricting abuse. These mechanisms should all be clearly described in the proposal. Tools to facilitate individual usage tracking through Globus GRAM audit extensions and other methods are described in the TeraGrid Science Gateway Primer, http://www. teragridforum.org/mediawiki/index.php?title=TeraGrid_Science_Gateways_Primer .
It may be appropriate to collect varying levels of information at registration depending upon the capabilities granted. For example, a training class with a restricted amount of SUs or resource access time would require a minimal amount of information from each student. Even less information might be required for K-12 demonstration accounts.
Gateways that grant or provide a significant amount of their allocation to individuals (or groups) should have in place internal peer-review and allocation procedures for awarding and distributing SUs and for managing and reporting SU usage. The Allocations Officers and/or an NSF representative will review these processes to determine acceptability in satisfying NSF merit-review criteria. For example, a gateway may want to request the following before granting larger fractions of gateway resources to senior researchers
- Resume or short bio
- Research abstract or proposal summary
- Outstanding achievements in past year
- Number of grants or sources of funding
- List of publications
- List of presentations
Some gateways may also allow researchers to access their own individual allocations through a gateway. Such capabilities should be described in the gateway proposal, however the SU justification would be contained in that scientist's individual allocation request.