Help:Cluster

If you are using a cluster with an older configuration, you may want to see Help:Rocks cluster.

Newer clusters will be using different tools than what comes with Rocks, but some components are shared. The biggest change is the use of slurm instead of SGE, but also new configurations now have the choice of Ubuntu or CentOS (or a mix).

You can view the status of all known clusters in CECS at http://newton.i2lab.ucf.edu/ganglia2/

See also the general cluster notes.

Additional documentation

The cluster software is composed of various components, each of which has their own documentation.

• Help:module load and unload software packages from your environment
• Help:MPI and Qsub and MPI example
• Help:Ganglia status monitor
• help:slurm which replaces the SGE batch queue system on the old clusters
• Help:Performance monitoring
• Gcc
• Access the cluster remotely through SSH
• Unix help
• Help:Data archival
• help:vpn Virtual Private Network to connect from off campus

See also Steve's cluster research page.

tutorial

This information is used for a 10 minute (or longer as necessary) short course on cluster use, which can be taylored to fit your needs and expected use.

general info

• Monitor cluster status with ganglia
• use ssh to enter the cluster
• use ssh or rsh to get between nodes if necessary
• use MPI to program and start your job if appropriate
• use MobaXterm or similar to access the system from windows if you need graphics

specific info

(These should probably be expanded--ask if you need details.)

• Use the batch queue systems appropriate for your cluster to schedule your jobs, such as slurm
• Watch out for hidden performance penalties if hyperthreading is enabled.
• your job can be optimized in various ways.
• please use ganglia to make sure your job is not left running accidentally and does not interfere with others' jobs.
• Be aware of disk quotas, move files to an appropriate storage location (or delete) when you are done with them.

External clusters and resources

• Globus file transfer on the research network
• UCF ARCC

Contact ARCC for access to the following external clusters:

• UF HiPerGator: training / price sheets
• XSEDE

Cluster programming guides

• Introduction to Parallel Computing
• Google: Introduction to Parallel Programming and MapReduce
• wikipedia:Message Passing Interface
• Wikipedia: instruction level parallelism, parallel computing, cluster computing