Genesis II/GFFS (Global Federated File System) Training
University of Virginia
Host site URL:
This class will cover both the use and administration of the Genesis II/GFFS stack on both XSEDE and local resources, though the administration portion will be towards the end of the session.
The class will run from 11 am – 4 pm EDT.
In order to join the class, please use the following web interface:
Also, you can use the following call in coordinates:
+1 217 332 6338 (Site1-Dialin-region) English (United States)
+1 312 994 8410 (Site1-Dialin-region) English (United States)
+1 888 983 3631 (Site1-Dialin-region) English (United States)
Conference ID: 8104049
Simple Secure Resource Sharing with the XSEDE Global Federated File System (GFFS) and Execution Management Services (EMS)
The XSEDE GFFS and EMS offers scientists and institutions a standards-based means through which they can interact with and securely share geographically distributed resources, e.g., file systems, storage, and compute resources that exist in different administrative domains. The resources may be in a lab, a regional compute center, or on a home computer.
The first principle of the GFFS is simplicity. Every researcher is familiar with the directory-based paradigm of interaction; the GFFS exploits this familiarity by providing a global shared namespace. The namespace appears to the user as files and directories so that the scientist can easily organize and interact with a variety of resources types. Once mapped into the shared namespace, resources can be accessed in a location-transparent fashion as if they were local.
In this tutorial we present the GFFS/EMS in the context of the GFFS.eu, a European rooted namespace that parallels the US XSEDE namespace, and describe functionality, motivation, as well as typical use cases. Participants will install and use the GFFS client and server, including: 1) securely sharing/accessing data, storage, and computational resources; 2) managing access control to resources; and 3) creating shared compute queues.
Topic area: Grids, campus bridging, secure compute and data sharing
Keywords: XSEDE, resource sharing, collaboration
Tutorial Goals. The goal of the tutorial is to familiarize the attendee with the capabilities offered by the XSEDE Global Federated File System and Execution Management Services (GFFS/EMS). This consists of two types of familiarity: first, familiarity with the typical use cases and sharing model in an abstract sense so that they can understand how the GFFS might be used in their research, and second, familiarity with installing and using the GFFS software.
Relevance to XSEDE users. The GFFS/EMS is one of the major components of the XSEDE architecture and one of the building blocks of the XSEDE Campus Bridging strategy. The GFFS will be used as the glue to integrate data and compute resources at national centers, campus centers, individual research labs, as well as desktops and laptops of individual researchers.
Target audience. The intended audience includes application developers, scientists that want to share resources with geographically separated partners, systems administrators in research labs and university computing centers, and computational support personnel who work with end users.
Audience Prerequisites. To install a container we assume familiarity with Unix. To install the client we assume familiarity with the notion of files and folders. We assume that each participant will have a suitable system (Linux, MacOS, Windows), and that the system has suitable bandwidth (minimum 1 Mb/sec, preferably more). We assume that users already have an XSEDE identity that has been validated, i.e., they are on an allocation and have a MyProxy identity.
• To perform the client exercises only, the user’s computer must be able to make outgoing https requests, i.e., no firewall blocking outgoing requests. Client-only means access and use GFFS/EMS resources that others have made available.
• To start a GFFS/EMS server the client machine must have 2 GB of memory, a publically routable IP address or DNS name, and 10 GB of local attached storage, NOT network storage, to hold a database. Local firewalls, if any, must permit incoming connections on port 18443.
General Description of Content. The tutorial consists of five modules. Many users may not be interested in the last module, server installation, and may want to drop off.
A. Overview, use-cases, demonstrations of the use cases.
Goals: • Understand the canonical use cases and how they apply in every day science
• Understand the federation model used by the GFFS
B. Client installation, GUI and shell usage, file-system access to the GFFS.
Goals: • Learn how to install an XSEDE GFFS client
• Login and navigate namespace
• Use GUI
• Manage access control
• Setup FUSE mount
C. Defining and running jobs, including parameter sweeps and workflows.
Goal: • Define and run jobs using GFFS
D. Access control including identity and group creation/management.
Goals: • Create users and groups
• Add users to groups
E. Server installation and configuration, sharing data and compute resources.
Goals: • Learn how to install an XSEDE GFFS container
• Learn how to create exports
• Create and manage Basic Execution Services (BES)
• Create and manage Grid queues
3. Detailed outline
As mentioned above there are five modules. For each module there are a set of topics and learning goals.
1. GFFS Overview
a. GFFS and EMS (Execution Management Services) as part of the XSEDE architecture
b. Five canonical uses cases
c. Accessing the GFFS with the GUI
i. Authentication, access control
ii. Accessing files and directories, editing files, creating files, copying, etc.
iii. Creating exports
iv. Job definition and management
d. Accessing the GFFS with a FUSE driver
2. Client installation basics hands on
a. Running the installer (we will come with the installer on memory sticks)
b. Accessing the GFFS with the GUI (see above)
3. Defining and running a jobs hands on
a. Defining a job with JSDL, JSDL tool
b. Running a job with a Basic Execution Service or Grid Queue
c. Managing and interacting with jobs, including running jobs
d. Parameter sweep jobs
4. Access Control and Authentication
a. Defining identities
b. Defining and managing groups
c. Access control management
5. Container installation and sharing resources
a. Container requirements
b. Installing and starting a container
c. Linking the container into the namespace
d. Setting up access control
e. Sharing data and compute resources
URL to sample material (including videos): http://genesis2.virginia.edu/wiki/Main/Tutorials
More information: http://genesis2.virginia.edu/wiki/Main/GFFS
05/12/2015 11:00 - 05/12/2015 16:00 EDT (SESSION HAS ENDED)View Session Details →
- Registration open date
- 05/05/2015 09:00 EDT
- Registration close date
- 05/12/2015 09:00 EDT
- Class size restriction
- 100 registrants
(94 spots left)
- Andrew Grimshaw
- Contact phone
- (434) 982-2204
- Contact email
- grimshaw, vbetro
- University of Virginia
- (434) 982-2204