Network storage

This storage is designed to be used as permanent storage. Network storage is a space containing users home and group storage directories.

• Implemented on a reliable Lustre file system (current capacity: 1.2 PB).
• Network-mounted and accessible from all BioHPC Cloud machines.
• Not backed up by default. Backup storage needs to be purchased independently.
• Access via Linux shell, scp/sftp clients, samba (mountable on Windows), or Globus Online. No machine reservation required.

Note: Network storage is not a scratch space for running jobs. Use local storage for this.

Users' home directories

• 200GB free for users with active credit
• Additional space available for purchase (in 1 TB-year increments)

Group storage directories (purchase in 1 TB-year increments)

• Intended as shared storage space for group members
• May host group members' home directories

Learn more about network storage

 

Local storage

This storage is designed as a temporary scratch space for running jobs.

• Located on disk arrays directly (i.e., not over the network) attached to each BioHPC Cloud machine.

• Capacity is machine-dependent (1TB - 12 TB), and specified on reservations pages

Use on rental machines

• Available as /workdir (same directory name on all machines, but physically different storage)
• Volatile - cleaned up automatically after end of reservation
• Accessible via Linux shell and scp/sftp (while reservation active)

Use on hosted servers

• Note that local storage is not backed up by default. Backup storage needs to be purchased independently.
• No automatic clean-up of either scratch or permanent local storage
• Access via Linux shell, scp/sftp, Globus Online (via mounts on login nodes), samba (on request)

Learn more about local storage

Backup storage

This storage is designed to maintain an additional, independent copy of your current data. It creates a 'current snapshot', automatically updated every night. It also retain previous versions of changed or deleted files for a specified amount of time.

• Located on an off-site cluster of dedicated file servers (in Weill Hall)

• Backup possible (but not default) for directories located on network storage (home and group directories) and/or on local storage on hosted server
• Backed up files retrievable through mounts on login nodes

Configuration options

• Purchase backup storage in 1 TB-year increments
• Directories to be included or excluded from back up
• Backup frequency
• Number of days a previous version of a file will be retained since the time the file was changed or deleted.

 Set up a backup storage

Data transfer and sharing

Data can be transferred and shared between the three storage types listed above.

Data transfer/sharing tools

• Linux shell, sftp, scp: Requires a BioHPC user account or a temporary guest account, plus an scp/sftp client at remote location.
• Samba (Windows network drive mapping): Available to BioHPC users on campus network (including VPN).
• Globus Online: Requires a BioHPC account or being added to access list for a Globus share defined at BioHPC, plus a Globus endpoint (server or personal) at remote location.

Sharing network storage options

1. Group storage directories: Convenient for sharing between lab or group members. Requires BioHPC user accounts for all members.
2. Temporary BioHPC accounts: Requested by BioHPC users for occasional sharing with external collaborators.
3. Globus Online: May be configured by any BioHPC user for sharing their files with specified Globus users (who do not have to have BioHPC accounts).

Sharing local storage on hosted servers

1. For intra-group access: Set appropriate file group ownership and permissions for each user of your group.
2. For external collaborators: Mount local storage on a login node, then

   • Share via Globus Online

   • Create and share with a temporary guest account (file permission adjustment may be required)

 See FAQs related to file transfer and sharing