Automate Creating Budgets for Resource Groups in Azure
So here at T1A (t1a.com), we continue our journey with cloud projects and recently we faced an issue with our internal Azure cloud environment.
The current approach for the internal stand is to provide wider access to our developers and consultants so they don’t struggle with any lack of access issues and can explore all available capabilities of the tools. We believe in our people, and we’ve never faced any issue that someone abused provided access with intentional or unintentional harm to the environment.
With an increasing number of projects and different teams working separately in our environment we faced that some Azure Resources’ Consumption might not be justified by the purposes it was created.
Our internal process says that for any new activity (project or internal research) you need to create a separate resource group (and we have a single subscription). With that been defined, we’re not interested in monitoring consumption on each Resource or on the Subscription level, we need to do it on the Resource Group (RG) level.
Azure capabilities allow you to explicitly create a Budget and associate list of alerts for RG, but it can’t be enforced by Policies, so in our case, a user can simply forget to create a corresponding budget for a new Resource Group, and if the RG overspends we will be able to see only in a monthly invoice. There are some ways how it can be avoided eg creating a budget on subscription, creating a report that shows RGs without associate budget or simply educating users, creating tags, and create budgets based on the tags
But for now, I decided to stick with budgets on RG level, here is an example of how it can be done by using Azure Automation, Azure Event Grid Subscription, and a Python script.
On the diagram below is a high-level architecture of the solution
Implementation Steps
- Create an Automation Account, in our case I named it T1A-Governance
2. Create a Run As Account under the Automation Account.
3. Create a Python 3 runbook. Examples of the scripts I used available here https://github.com/nbnm/AzureAutomation/. In this example, I refer to rb_createbudgetforrg.py which creates a budget with alerts for a newly created RG, there is also available rb_updatergwithbudget.py which pulls existing RGs and if the budget is not associated with the RG it creates it.
3. Publish the runbook in order to be able to create an associated webhook (don’t forget to copy and save the webhook address)
4. Create an Event Grid Subscription that will capture resource-writes-success events (basically whenever resource created/updated). As destination specify webhook and use the address from step 3.
By default, Event Grid Subscription captures all type of resources you need to specify an advanced filter to filter out resources other than Resource Group
5. Create a variable under Automation Account that will contain Budget Template in JSON format (not an ARM template). Example that I used BudgetTemplate.json
That was the last step, so once you publish the runbook it’s all should work.
- For any new/update on resource, the corresponding event will land in Event Grid
- Based on the created Event Grid Subscription the events for a resource other than the Resource group will be filtered and only the Resource Grop event will be forwarded to the webhook
- Webhook call will trigger the runbook and passes information about created Resource Group
- The runbook checks whether budget exists for the Resource Group and if it’s not there will create a budget using the template specified in the variable
Conclusion
Originally I thought I would be able to complete this assignment by using Python SDK only without explicit API calls, but due to the issue with SDK (https://github.com/Azure/azure-powershell/issues/11642) it didn’t work, the same issue appears for Python SDK.
I’m not sure whether it’s the best to approach to track resource consumption in consulting shops like ours, maybe we will adjust it in the future.
If you have questions or want to share your approach please feel free:
oleg.mikhov@t1a.com
