Link Search Menu Expand Document

HPC and computationally intensive workloads

Cloud platforms can be set up to run a job scheduling system similar to Xsede or an on-campus super computing cluster. One of the major benefits to this is that these systems can automate the allocation and release of virtual machines, minimizing the cost and time spent crunching data.

In general, jobs are packaged in the form of Docker container images. For an introduction to Docker, check out CloudBank’s Containerization Solution.

Turnkey job submission

A turn-key job submission service requires little to no setup on your end. These are usually called “batch” services, and are a good place to start if you don’t need fine-grain tuning of your setup.

Under the hood: setting up a workload scheduler (eg, Slurm)

If you’re running applications that require more fine tuning than just a job submission interface or powerful CPUs, most cloud platforms offer “compute cluster” services attempting to be drop-in replacements for on-campus HPC clusters.

Case Studies

  • Computing at Scale using Rosetta: Dr. Parisa Hosseinzadeh and her team were able to utilize the massively parallel nature of the cloud to simulate protein folding at an astounding rate using AWS Batch and cost-effective spot instances.
  • Wildfire Modeling with TPUs: A team at Google used GCP’s TPU processing clusters to perform large-scale CFD simulations of recent wildfire disasters, helping to improve model accuracy and improve predictions for future events.