Cluster

This is the local cluster research page, linking to information about managing, setting up, and using clusters. You may want to look at one of these instead:

See also:

• Help:Cluster
• Category:Clusters
• Category:Cluster software

• Each cluster has a blog as well

"If you were plowing a field, which would you rather use? Two strong oxen or 1024 chickens?"

- Seymour Cray (1925-1996), father of supercomputing

External resources

If any of these become widely used in any cluster,, they will be installed and documentation links moved to help:cluster.

Amdahl's law is a good starting point when optimizing the hardware in a cluster.

Libraries and cluster projects

• ganglia
• OpenMPI
• Berkeley Lab Checkpoint/Restart (BLCR)
• Heirarchal Data Format
• ipython interactive computing and parallel processing library
• Chapel parallel programming language (see also: wikipedia:Chapel (programming language)

Sparse libraries

• CSparse
• GotoBlas
• SparseLib++
• SuperLU
• TAUCS
• UMFpack

performance monitoring

• IPM
  • perfctr

remote management hardware

• Dell hardware monitoring home page
• Dell: omconfig options

(most of these are PDF files)

• Managing Dell PowerEdge Servers Using IPMItool
• Dell PowerEdge IPMI
• Configuring the BMC and BIOS
• Remote Management with the Baseboard Management Controller in Eighth-Generation Dell PowerEdge Servers
• Managing and Monitoring High-Performance Computing Clusters with IPMI
• IPMI on linux

Unevaluated links

• power saving: [1] gridinfo wiki: PowerSaving
• module system
  • http://modules.sourceforge.net/
  • http://www.nersc.gov/nusers/resources/software/os/modules.php
• rocks-solid power saving roll
• Sun Clustering
  • sun grid engine
• Beowulf
• Global Layer Unix
• LJ article on Gnu Que
• Kerrighed
• OpenSSI
• webcom (university of cork)
• condor
• experimental multiple heads on rocks

Sun Grid engine

• http://gridengine.sunsource.net/project/gridengine/howto/howto.html
• http://gridengine.sunsource.net/project/gridengine/howto/qmon/qmon_custom.html
• http://gridengine.info/2005/09/19/parallel-environments-pes-loose-vs-tight-integration
• http://gridengine.sunsource.net/howto/mpich-integration.html
• http://gridengine.sunsource.net/howto/mpich2-integration/mpich2-integration.html
• http://wiki.gridengine.info/wiki/index.php/Main_Page

Cloud

• google: cloud
• Apache CloudStack
• OpenStack
• Amazon Elastic Compute Cloud
• Sun: Cloud webinar
• wikipedia:Cloud computing
• wikipedia:Category:Cloud platforms

• http://ostatic.com/blog/eucalyptus-an-unsung-open-source-infrastructure-for-cloud-computing
• http://www.eucalyptus.com/
• http://auser.github.com/poolparty/

cluster suites

• rocks
  • help:rocks command reference (local)
  • Users guide
  • paper on Visualization with Rocks
• Platform OCS commercial rocks fork
• Oscar (cluster suite)
• xcpu cluster cloud system
• VDT
• perceus

See also How to successfully build a Linux Compute Cluster

management systems

• Dell OMSA
• Ganglia (CECS grid)
• Argus

job queueing and distribution

See also wikipedia:Category:Job scheduling

• PBS queueing system (see wikipedia:Portable Batch System for a list of forks)
  • Torque
• Sun Grid Engine (see also GridWiki, wikipedia:Sun Grid Engine)
• slurm
• drqueue (for render farms?)
• redhat batch command
• pdsh
• http://freshmeat.net/projects/clustercontrol/
• gridder

Torque

• overview
• MOM
• scheduler (FIFO, Maui, Moab)
• server config
• torque admin guide (quickstart in appendix)

Virtualization

• Dell TOE
• iScsi
• Xen
• VirtualBox
• KVM

Visualization tools

• EMAN
• VisIt

Hardware

• APC papers on high density cluster deployments
• Amdahl's Other Law: To be balanced, the following characteristics of a computing system should all be the same:
  • the hertz of CPU speed
  • the bytes of main (RAM) memory
  • the bits per second of (I/O) bandwidth
• Dell PE 1950
• infiniband
• mirrornet

GPU clusters

• wikipedia:GPU cluster
• http://www.cs.sunysb.edu/~vislab/papers/GPUcluster_SC2004.pdf
• http://www.ncsa.uiuc.edu/Projects/GPUcluster/projects.html
• http://www.cs.unc.edu/~tgamblin/gpgpu/

Fair use policies of clusters

• http://euler.cs.ucf.edu/
• http://www.nersc.gov/nusers/systems/bassi/running_jobs/classes.php

Tomcat

• http://tomcat.apache.org
• Securing Linux for Java services
• Top Ten Tomcat Configuration Tips

Libraries, optimizations, and performance tools

libraries

• matrix libraries
  • ATLAS
  • Building and running HPL with GOTO BLAS

performance tools

• Integrated Performance Monitoring (IPM) or sourceforge

benchmarks

• benchmark collections and papers
  • BenchWeb
  • Cluster Benchmarks From The Cluster Agenda
• benchmark software
  • HPC Challenge (suite of 7)

optimizers

• http://brnorris03.github.io/Orio/ (RCE podcast)
• pypy JIT optimizer
• numba specialized JIT optimizer

UCF links

• https://arcc.ist.ucf.edu/

News articles

• 2007-Jan-30 Linux Cluster designed by Orlando’s Tsunamic Technologies to Power Gene-sequencing Research

Cloud computing tools

• wikipedia:Cloud computing
• wikipedia:scalr / http://scalr.net/
• http://www.ovirt.org/

cloud infrastructure projects

• [http://www.eucalyptus.com/
• wikipedia:OpenStack / http://www.openstack.org/

External links

• http://www.clustermonkey.net/
• The Distributed Computing Group at Daresbury Laboratory
• http://distrowatch.com/table.php?distribution=rockscluster
• http://www.schneertz.com/subcomputing.html (joke)
• Supercomputing with a stack of PlayStations (compares PS3 to 25 Blue Gene cpus)
• wikipedia
  • grid computing
  • cloud computing
• Google and the wisdom of clouds
• HP, Yahoo, Intel Launch Cloud Computing Test Bed