AWS Spot Pricing Jupyter Notebook

---

CloudyCluster and CCQ enable easy usage of Spot Instances and Spot Fleet for HPC, saving the cost of computation for compatible workloads (short runtime, restartable, etc…) All you have to do is add directives directly to the job script and CCQ will take care of creating the spot and spot fleet requests. To help determine the appropriate spot instance price and what region / availability zone is optimal for the run we have created a jupyter notebook that can pull down the pricing for an instance type in a region across as many AZs in that region you select. As part of the notebook we have leveraged FBProphet to do time series analysis to help create a model for the spot price for the instance selected.  
 

Project Jupyter

Jupyter is a project that spun off of the iPython interactive computing environment. The Project Jupyter goal is to extend interactve computing to all programming environments. To install jupyter or jupyterlab (the latest version of the jupyter interface) you can follow the Jupyter Instructions, JupyterLab Instructions  
 

AWS Spot Pricing Notebook

To use the notebok, complete the prerequsite setup items

Start jupyter then navigate to where the Spot_Price_Forecasting.ipynb is located on your system.

 

Next select Run -> Run all cells and scroll down and follow the instructions and fill in the Region and Availability Zone information and select Start Forecasting.**

 

The first graph, shows the spot pricing for the instance type, region and AZ for the time range specified

 

The next seet of graphs, shows the spot pricing forecast for the individual AZs for the forecast time range specified

 

Now you will have a simple tool to help determine what pricing to run your HPC jobs on Spot or Spot Fleet instances.

Enjoy.