The architecture of enterpriseTerraform configuration building blocks for Kubernetes on Azure
In this article, I defined some of the best practices & Terraform configuration building blocks architecture to be followed in an enterprise application, especially for a Microservice application. The microservice architecture consists of resources such as Azure Kubernetes Service (AKS), Kubernetes cluster, Kubernetes node pools, Azure load balancer, container registry for docker image push/pull, azure monitor for monitoring of diagnostic setting logs & Azure Key vault for secrets management.
The sample reference architecture is available on Azure architecture center in this following link:
https://arch-center.azureedge.net/aks-reference-architecture.vsdx
Now, coming to the infrastructure provisioning tools part, writing terraform configuration for this architecture. To follow, there’s a rule of thumb to maintain a separate folder structure for each resource with having the following basic terraform configurations like :
- azservice.tf
- variables.tf
- providers.tf
- outputs.tf
- terraform.tfvars
while we can always define the terraform resource configuration building blocks tree enabled for different environments like
- Development
- UAT
- Preprod
- Production etc. with specific master directories configured for each environment consists of each resource configuration block.
Azure Kubernetes cluster node pools configuration with Terraform
The current edition of Kubernetes (1.19.0) on Azure doesn’t support the deployment of kube dashboard through Terraform configurations & the Kubernetes cluster automatic channel upgrade in “Stable” mode is in preview mode as of writing this article.
Here goes the sample terraform configuration of the Azure Kubernetes cluster enabling auto-scaling mode with node pool.
# Create Azure Kubernetes Cluster
resource “azurerm_kubernetes_cluster” “example” {
#name = “${random_pet.prefix.id}-aks” // for terraform version >= 0.14+
name = “${var.az_aks_cluster_name}” // for terraform version <= 0.13
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
#dns_prefix = “${random_pet.prefix.id}-k8s”
dns_prefix = “az-aks-nodepools-dns”
default_node_pool {
name = “nodepool”
node_count = 2
vm_size = “Standard_D2_V2”
os_disk_size_gb = 30
enable_auto_scaling = true
max_count = 4
min_count = 1
upgrade_settings {
max_surge = 70 // maximum no of or % of nodes added to the node pool during upgrade
}
}
service_principal {
client_id = var.appId
client_secret = var.appsecret
}
role_based_access_control {
enabled = true
}
addon_profile {
kube_dashboard {
enabled = false
}
}
tags = {
environment = “Production”
costcenter = “1005623487”
}
}
while provisioning Kubernetes deployment on Azure/Cloud, it’s absolutely necessary, for monitoring tools like Azure monitor, adding the capabilities like diagnostics settings logs, sign-in logs, failed response error logs, activity logs, etc post-collection sending to Azure log analytics workspace.
Here goes the sample terraform configuration of the Azure monitor provisioning .tf file with diagnostics logs, AD user sign-in logs, etc. being sent to the Azure Log Analytics workspace.
# Create Azure Monitor with Azure AD diagnostic settings
resource “azurerm_monitor_aad_diagnostic_setting” “example” {
name = var.azmonitor_diag_setting_name
storage_account_id = azurerm_storage_account.example.id
log_analytics_workspace_id = azurerm_log_analytics_workspace.example.id // Sent monitor diag logs to Azure Log Analytics workspace
log {
category = “SignInLogs”
enabled = true
retention_policy {
enabled = true
days = 1
}
}
log {
category = “AuditLogs”
enabled = true
retention_policy {
enabled = true
days = 1
}
}
log {
category = “NonInteractiveUserSignInLogs”
enabled = true
retention_policy {
enable = true
days = 1
}
}
log {
category = “ServicePrincipalSignInLogs”
enabled = true
retention_policy {
enabled = true
days = 1
}
}
log {
category = “ProvisoningLogs”
enabled = false
retention_policy {}
}
log {
category = “ADFSSignInLogs”
enabled = false
retention_policy {}
}
}
# Azure Log Analytics Workspace ID
data “azurerm_log_analytics_workspace” “example” {
name = azurerm_log_analytics_wsname
resource_group_name = azurerm_resource_group.example.name
}
The end-to-end infra as code (IAC) terraform configuration building blocks on this sample Azure Kubernetes microservice architecture is available in the following repo of Github.
#Happy Terraforming!
