Empire AI Alpha Storage

TABLE OF CONTENTS

• Home Filesystem
• Scratch Filesystem
• /dev/shm Storage

Home Filesystem

The home file system is a Vast storage appliance mounted over NFS. This filesystem is best used to hold software installations, job scripts, and other files that users need to store long term and use frequently. This is not a good location to store data sets or for use as scratch storage.

The home system is mounted at /mnt/home/{username} on all Alpha servers.

Scratch Filesystem

The scratch system for Alpha is a Data Direct Networks (DDN) Lustre appliance. This storage is best used for parallel access to files during jobs, bulk storage for job data, and data sets that can be replaced when needed. Currently there is no purge system, but one will likely be implemented in the future to remove files that have not been accessed for some period of time. In HPC parlance, this is a scratch filesystem and should be treated as transient, high-performance storage for files used by your compute jobs.

The scratch system is mounted at /mnt/lustre/{institution}/{username}

/dev/shm Storage

On Linux servers the /dev/shm location is a Shared Memory device. This is an in-memory filesystem, and as such it is extremely fast... far faster than either the home or Scratch systems. If you have code that either uses large numbers of very small files, or performs a lot of metadata operations (such as opening, reading, and closing files quickly) then you will get far better performance by doing those operations in /dev/shm then moving the files to Lustre or home storage at the end of your job.

If you plan to use /dev/shm you need to be aware that data placed there will count toward the memory usage of your job. If you need 10GB of storage in /dev/shm then be sure to add that 10GB to your jobs memory request. The memory you request should be the sum of (memory used by your code + storage requirements in /dev/shm).