Money is always important whether you are developing for yourself, your employer, or a client.
One way to save money in Azure is to shut down resources at the end of the day. As an Azure data engineer, it can be a tedious task that is easy to forget. There can be a lot of clicking through resources in the portal to get things shut down, especially if you have a lot of resources. Why not make our lives easier by scripting a solution?
In this post, I’m going to talk about two resources that I usually shut down at the end of the workday while they are in active development: VMs and Synapse Analytic SQL Pools.
VMs are often used as development platforms during the day and are left idle throughout the night. While idle, the cost still accumulates. There are a few options available to shut down VMs. VMs can be shut down one by one manually via the portal.
There is also a great Start/Stop feature that can automatically shutdown your VMs at specific times. This is effective, but only if you work during fixed hours. The other option is shutting down via the Azure CLI which I will explain here.
Before going any further it is important to understand 2 VM states:
One caveat is that you will likely lose your dynamically assigned IP address when deallocating.
The portal Stop button moves the VM into the “Stopped (Deallocated)” state.
Shutting down a VM via the Azure CLI gives the most control and flexibility. It is also an easy way to become familiar with the Azure CLI.
The commands covered today are
If you have a couple of static VMs you can just hard code these two commands and save them in a shell script and run them at the end of your workday. I recommend using the Azure Cloud Shell.
az vm stop --resource-group MyResourceGroup --name MyVm
az vm deallocate --name MyVm --no-wait --resource-group MyResourceGroup
If you have a few VMs in a resource group you can shut them all down with a commands like these:
az vm stop --ids $(az vm list -g MyResourceGroup --query "[].id" -o tsv)
az vm deallocate --ids $(az vm list -g MyResourceGroup --query "[].id" -o tsv)
Synapse Analytics Workspaces (note: at the time of writing this is a preview feature and subject to change) have SQL and Spark Pools associated with them. If there are no workloads for your fixed performance level SQL pool you can safely scale it down or pause it overnight.
Just like VMs, this can be accomplished via the portal or via script.
The Azure CLI command to pause your SQL pool is:
az synapse sql pool pause --name sqlpoolcli1 --workspace-name testsynapseworkspace --resource-group rg
Up to now, you’ve seen that we can use Azure CLI to shut down VMs and Synapse SQL Pools via static scripts. This is very effective if you have one project and resources don’t change frequently. If you are working in multiple projects/resource groups bash scripting (via a UNIX terminal with Azure CLI installed) can be leveraged to use these commands dynamically.
The below script is one I developed to make it easier to shut down VMs and Synapse SQL Pools within a resource group. It is presented here in its current form but will be improved and expanded in my repository. Please feel free to use it and modify it to suit your needs. It will prompt you to select the resource group and list all possible SQL Pools and VMs to shut down. You can then select which resources to shutdown/pause. It will show the commands it will run and prompt you if you want to continue.
https://gist.github.com/ZMon3y/6fc6493e0e2c714e5adec7c5f524302a
I hope this helps you save time and money as you develop in Azure! Please reach out to me via the repository or on Twitter @MattSzafirPro.
Cloud management is difficult to do manually, especially if you work with multiple cloud…
Azure’s scalable infrastructure is often cited as one of the primary reasons why it's the…
https://www.youtube.com/watch?v=wDzCN0d8SeA Watch our "Unlocking the Power of AI in your Software Development Life Cycle (SDLC)"…
FinOps is a strategic approach to managing cloud costs. It combines financial management best practices…
Using Kubernetes with Azure combines the power of Kubernetes container orchestration and the cloud capabilities…
In the intricate landscape of modern business, compliance is both a cornerstone of operational integrity…