Self Service Cloud HPC

LAUNCH YOUR OWN HPC CLOUD COMPLETE WITH SCHEDULER, COMPUTE AND STORAGE BASED ON ORANGEFS, EFS AND S3. CLOUDYCLUSTER COMES COMPLETE, READY TO COMPUTE WITH A WIDE VARIETY OF POPULAR OPEN HPC SOFTWARE

CloudyCluster

  • Begins with a Management Instance that is launched in your own AWS account.
  • A simple but powerful Web-UI is then used to create a base HPC environment with Scheduler, Login instance and storage that you choose.
  • After the HPC environment is created you can submit Torque(PBS) or Slurm jobs through CCQ, a meta-scheduler that is delivered with CloudyCluster.
  • Once submitted, CCQ inspects the job and instantiates the instances dynamically and once ready releases the job to Torque(PBS) or Slurm to process the job.
  • When all the jobs for that instance type are complete, CCQ tears down the instances before the billing cycle is complete, so you only pay for the runtime you need.
Try Cloudycluster
feature-image

Architecture

CloudyCluster is designed to look and feel similar to standard HPC environments, with login instances, scheduler, scratch storage and with the added capabilities of elastic compute instance creation. A group of researchers from Clemson University achieved a remarkable milestone while studying topic modeling, an important component of machine learning associated with natural language processing, breaking the record for creating the largest high-performance cluster by using more than 1,100,000 vCPUs on Amazon EC2 Spot Instances running in a single AWS region, accessed via CloudyCluster.

CLOUDYCLUSTER MILLION+ CORE RUN ON AWS

Read more about it in Jeff Barrs AWS Blog

1.1m vCPU on AWS Blog

Facts & Tweets

  • 6397

    CloudyClusters Deployed

  • 5000000

    + People working with HPC Globally

  • 49925

    Instances in Largest CloudyCluster

  • 1119196

    vCPUs in Largest CloudyCluster

Recent Blog Posts

Keep up with CloudyCluster Happenings!!

post-thumb AWS Spot Pricing Jupyter Notebook

CloudyCluster and CCQ enable easy usage of Spot Instances and Spot Fleet for HPC, saving the cost of computation for compatible workloads (short runtime, restartable, etc…) All you have to do is add directives directly to the job script and …

Read More
post-thumb Scaling HPC in AWS

We have been approached by many people wanting to scale their science or engineering to the scale of the 1.1m vCPU run recently completed by the DICE Lab at Clemson. In addition, others have wanted to automate their workflows in AWS as well. We have …

Read More