The architecture of enterpriseTerraform configuration building blocks for Kubernetes on Azure

Kubernetes with Terraform

In this article, I defined some of the best practices & Terraform configuration building blocks architecture to be followed in an enterprise application, especially for a Microservice application. The microservice architecture consists of resources such as Azure Kubernetes Service (AKS), Kubernetes cluster, Kubernetes node pools, Azure load balancer, container registry for docker image push/pull, azure monitor for monitoring of diagnostic setting logs & Azure Key vault for secrets management.

The sample reference architecture is available on Azure architecture center in this following link:

https://arch-center.azureedge.net/aks-reference-architecture.vsdx

Now, coming to the infrastructure provisioning tools part, writing terraform configuration for this architecture. To follow, there’s a rule of thumb to maintain a separate folder structure for each resource with having the following basic terraform configurations like :

Terraform resource configuration

while we can always define the terraform resource configuration building blocks tree enabled for different environments like

Azure Kubernetes cluster node pools configuration with Terraform

The current edition of Kubernetes (1.19.0) on Azure doesn’t support the deployment of kube dashboard through Terraform configurations & the Kubernetes cluster automatic channel upgrade in “Stable” mode is in preview mode as of writing this article.

Here goes the sample terraform configuration of the Azure Kubernetes cluster enabling auto-scaling mode with node pool.

# Create Azure Kubernetes Cluster

resource “azurerm_kubernetes_cluster” “example” {

#name = “${random_pet.prefix.id}-aks” // for terraform version >= 0.14+

name = “${var.az_aks_cluster_name}” // for terraform version <= 0.13

location = azurerm_resource_group.example.location

resource_group_name = azurerm_resource_group.example.name

#dns_prefix = “${random_pet.prefix.id}-k8s”

dns_prefix = “az-aks-nodepools-dns”

default_node_pool {

name = “nodepool”

node_count = 2

vm_size = “Standard_D2_V2”

os_disk_size_gb = 30

enable_auto_scaling = true

max_count = 4

min_count = 1

upgrade_settings {

max_surge = 70 // maximum no of or % of nodes added to the node pool during upgrade

}

}

service_principal {

client_id = var.appId

client_secret = var.appsecret

}

role_based_access_control {

enabled = true

}

addon_profile {

kube_dashboard {

enabled = false

}

}

tags = {

environment = “Production”

costcenter = “1005623487”

}

}

while provisioning Kubernetes deployment on Azure/Cloud, it’s absolutely necessary, for monitoring tools like Azure monitor, adding the capabilities like diagnostics settings logs, sign-in logs, failed response error logs, activity logs, etc post-collection sending to Azure log analytics workspace.

Here goes the sample terraform configuration of the Azure monitor provisioning .tf file with diagnostics logs, AD user sign-in logs, etc. being sent to the Azure Log Analytics workspace.

# Create Azure Monitor with Azure AD diagnostic settings

resource “azurerm_monitor_aad_diagnostic_setting” “example” {

name = var.azmonitor_diag_setting_name

storage_account_id = azurerm_storage_account.example.id

log_analytics_workspace_id = azurerm_log_analytics_workspace.example.id // Sent monitor diag logs to Azure Log Analytics workspace

log {

category = “SignInLogs”

enabled = true

retention_policy {

enabled = true

days = 1

}

}

log {

category = “AuditLogs”

enabled = true

retention_policy {

enabled = true

days = 1

}

}

log {

category = “NonInteractiveUserSignInLogs”

enabled = true

retention_policy {

enable = true

days = 1

}

}

log {

category = “ServicePrincipalSignInLogs”

enabled = true

retention_policy {

enabled = true

days = 1

}

}

log {

category = “ProvisoningLogs”

enabled = false

retention_policy {}

}

log {

category = “ADFSSignInLogs”

enabled = false

retention_policy {}

}

}

# Azure Log Analytics Workspace ID

data “azurerm_log_analytics_workspace” “example” {

name = azurerm_log_analytics_wsname

resource_group_name = azurerm_resource_group.example.name

}

https://github.com/imcuteani/az-kubernetes-architecture/blob/ebd3fdeceff7b4e0a99cfe86185372b25475dea8/Azure_Monitor/azmonitor.tf

The end-to-end infra as code (IAC) terraform configuration building blocks on this sample Azure Kubernetes microservice architecture is available in the following repo of Github.

#Happy Terraforming!

Cloud Architect. Azure, AWS certified. Terraform expert.