当前位置： 首页 > news >正文

Terraform Helm：微服务基础设施即代码

news 2025/7/2 8:33:44

🚀 Terraform & Helm：微服务基础设施即代码

📚 目录

  • 🚀 Terraform & Helm：微服务基础设施即代码
    • 1. 引言 🚀
    • 2. 环境与依赖 🧰
    • 3. 架构示意 🏗️
    • 4. Terraform 定义云资源 🛠️
      • 4.1 Provider 与 Backend
      • 4.2 公共变量与 Tag
      • 4.3 Resource Group 模块
      • 4.4 VNet 模块
      • 4.5 Key Vault 模块
      • 4.6 AKS 模块
      • 4.7 RDS 模块
      • 4.8 根模块调用
    • 5. Helm Chart 打包与校验 📦
      • 5.1 Chart.yaml & Chart.lock
      • 5.2 values.schema.json
      • 5.3 Subchart 示例（gateway）
      • 5.4 Lint & Test
    • 6. CI/CD 流水线 🔄
      • 6.1 Infra Workflow
      • 6.2 Deploy Workflow
    • 7. 可观测性与告警 🔔
    • 8. 附录 📂

1. 引言 🚀

✨ TL;DR

  • 🛠️ 精准声明：Terraform 管理 Resource Group、VNet（多子网）、AKS（跨可用区 + Azure Monitor）、Key Vault、PostgreSQL 等生产资源，统一 Tag 并锁定 Provider 版本
  • 📦 Umbrella Chart：Helm Umbrella Chart 支持多微服务（Gateway、Identity…），含 Probes、PodDisruptionBudget、NetworkPolicy、HPA、Secret 引用、values.schema.json 与 Chart.lock
  • 🔄 端到端 CI/CD：GitHub Actions 流水线集成 OIDC 登录、Terraform fmt/validate/lint、Checkov、Infracost、Azure 登录、ACR 鉴权、Helm lint/test/package/push、自动回滚，并发控制与环境审批
  • 🌟 企业级要素：高性能（Azure CNI + HPA + PDB）、高可用（多 AZ + Monitoring）、安全可复现（Key Vault + Terraform Backend + Sensitive + OIDC + Cost Scan）

📚 背景与动机
在多微服务架构下，“在我机器上没问题”往往难以复现🧩。通过 Terraform + Helm Chart + GitOps/CI-CD，可实现基础设施与应用部署一体化自动化、一致化、可审计、可回滚，大幅提升交付速度与可靠性💪。

2. 环境与依赖 🧰

terraform version      # >=1.4
az version             # 最新 Azure CLI
kubectl version --client
helm version           # >=3.8+

仓库结构

.
├─ infra/
│   ├─ terraform/
│   │    ├─ backend.tf
│   │    ├─ required_providers.tf
│   │    ├─ variables.tf
│   │    ├─ main.tf
│   │    ├─ outputs.tf
│   │    └─ modules/
│   │         ├─ resource_group/
│   │         ├─ vnet/
│   │         ├─ keyvault/
│   │         ├─ aks/
│   │         └─ rds/
│   └─ helm-charts/
│        └─ abp-vnext/
│             ├─ Chart.yaml
│             ├─ Chart.lock
│             ├─ values.schema.json
│             ├─ values.yaml
│             ├─ values-dev.yaml
│             ├─ values-prod.yaml
│             ├─ charts/           # Gateway、Identity 等 Subcharts
│             └─ templates/        # Umbrella 公共资源（Ingress 可选）
└─ src/└─ MyAbpSolution.sln

3. 架构示意 🏗️

Azure
Resource Group
VNet
Subnet: aks
Subnet: db
Key Vault
AKS 集群
PostgreSQL
Helm 部署多服务
开发者
CI/CD 流水线
IaC: Terraform
Helm 部署

4. Terraform 定义云资源 🛠️

4.1 Provider 与 Backend

# required_providers.tf
terraform {required_version = ">= 1.4"required_providers {azurerm = {source  = "hashicorp/azurerm"version = "~> 3.0"}}backend "azurerm" {resource_group_name  = "tfstate-rg"storage_account_name = "tfstateacct"container_name       = "tfstate"key                  = "${terraform.workspace}.tfstate"}
}

4.2 公共变量与 Tag

# variables.tf
variable "location"    { type = string }
variable "environment" { type = string }
variable "owner"       { type = string }
variable "cost_center" { type = string }
variable "db_admin"    { type = string }
variable "db_password" { type = string }locals {common_tags = {environment = var.environmentowner       = var.ownercost_center = var.cost_center}
}

4.3 Resource Group 模块

# modules/resource_group/main.tf
resource "azurerm_resource_group" "this" {name     = var.namelocation = var.locationtags     = var.tags
}
output "rg_name" { value = azurerm_resource_group.this.name }

# modules/resource_group/variables.tf
variable "name"     { type = string }
variable "location" { type = string }
variable "tags"     { type = map(string) }

4.4 VNet 模块

# modules/vnet/main.tf
resource "azurerm_virtual_network" "this" {name                = var.nameaddress_space       = var.address_spacelocation            = var.locationresource_group_name = var.rg_nametags                = var.tags
}resource "azurerm_subnet" "this" {for_each            = var.subnetsname                = each.keyresource_group_name = var.rg_namevirtual_network_name= azurerm_virtual_network.this.nameaddress_prefixes    = [each.value]
}output "subnet_ids_map" {value = { for s in azurerm_subnet.this : s.name => s.id }
}

# modules/vnet/variables.tf
variable "name"          { type = string }
variable "location"      { type = string }
variable "rg_name"       { type = string }
variable "address_space" { type = list(string) }
variable "subnets"       { type = map(string) }
variable "tags"          { type = map(string) }

4.5 Key Vault 模块

# modules/keyvault/main.tf
data "azurerm_client_config" "current" {}resource "azurerm_key_vault" "this" {name                        = var.namelocation                    = var.locationresource_group_name         = var.rg_nametenant_id                   = data.azurerm_client_config.current.tenant_idsku_name                    = "standard"purge_protection_enabled    = truesoft_delete_enabled         = truetags                        = var.tags
}resource "azurerm_key_vault_secret" "db_password" {name         = "db-password"value        = var.admin_passwordkey_vault_id = azurerm_key_vault.this.id
}

# modules/keyvault/variables.tf
variable "name"           { type = string }
variable "location"       { type = string }
variable "rg_name"        { type = string }
variable "admin_password" { type = string }
variable "tags"           { type = map(string) }

4.6 AKS 模块

# modules/aks/main.tf
resource "azurerm_kubernetes_cluster" "this" {name                = var.namelocation            = var.locationresource_group_name = var.rg_namedns_prefix          = var.dns_prefixtags                = var.tagsdefault_node_pool {name                = "agentpool"vm_size             = var.node_sizeavailability_zones  = var.availability_zonesenable_auto_scaling = var.enable_auto_scalermin_count           = var.min_countmax_count           = var.max_countos_disk_size_gb     = 50}network_profile {network_plugin    = "azure"network_policy    = "calico"load_balancer_sku = "standard"subnet_id         = var.aks_subnet_id}identity { type = "SystemAssigned" }addon_profile { oms_agent { enabled = true } }  # Azure Monitor
}output "kube_config" {value     = azurerm_kubernetes_cluster.this.kube_admin_config_rawsensitive = true
}

# modules/aks/variables.tf
variable "name"               { type = string }
variable "location"           { type = string }
variable "rg_name"            { type = string }
variable "dns_prefix"         { type = string }
variable "aks_subnet_id"      { type = string }
variable "node_size"          { type = string }
variable "availability_zones"{ type = list(string), default = ["1","2","3"] }
variable "enable_auto_scaler" { type = bool, default = true }
variable "min_count"          { type = number, default = 2 }
variable "max_count"          { type = number, default = 5 }
variable "tags"               { type = map(string) }

4.7 RDS 模块

# modules/rds/main.tf
data "azurerm_key_vault_secret" "db_pwd" {name         = "db-password"key_vault_id = var.keyvault_id
}resource "azurerm_postgresql_flexible_server" "this" {name                         = var.namelocation                     = var.locationresource_group_name          = var.rg_nameversion                      = var.pg_versionsku_name                     = var.sku_namestorage_mb                   = var.storage_mbdelegated_subnet_id          = var.subnet_idadministrator_login          = var.admin_useradministrator_login_password = data.azurerm_key_vault_secret.db_pwd.valuetags                         = var.tags
}output "connection_string" {value     = format("Host=%s;Port=5432;Username=%s;Password=%s;Database=%s",azurerm_postgresql_flexible_server.this.fqdn,var.admin_user,data.azurerm_key_vault_secret.db_pwd.value,var.db_name)sensitive = true
}

# modules/rds/variables.tf
variable "name"        { type = string }
variable "location"    { type = string }
variable "rg_name"     { type = string }
variable "pg_version"  { type = string }
variable "sku_name"    { type = string }
variable "storage_mb"  { type = number }
variable "admin_user"  { type = string }
variable "subnet_id"   { type = string }
variable "db_name"     { type = string }
variable "keyvault_id" { type = string }
variable "tags"        { type = map(string) }

4.8 根模块调用

# main.tf
provider "azurerm" { features {} }module "rg" {source   = "./modules/resource_group"name     = "${var.environment}-rg"location = var.locationtags     = local.common_tags
}module "vnet" {source        = "./modules/vnet"name          = "${var.environment}-vnet"location      = var.locationrg_name       = module.rg.rg_nameaddress_space = ["10.0.0.0/16"]subnets       = { aks = "10.0.1.0/24", db = "10.0.2.0/24" }tags          = local.common_tags
}module "keyvault" {source         = "./modules/keyvault"name           = "${var.environment}-kv"location       = var.locationrg_name        = module.rg.rg_nameadmin_password = var.db_passwordtags           = local.common_tags
}module "aks" {source            = "./modules/aks"name              = "${var.environment}-aks"location          = var.locationrg_name           = module.rg.rg_namedns_prefix        = var.environmentaks_subnet_id     = module.vnet.subnet_ids_map["aks"]node_size         = "Standard_DS2_v2"availability_zones= ["1","2","3"]enable_auto_scaler= truemin_count         = 2max_count         = 5tags              = local.common_tags
}module "rds" {source      = "./modules/rds"name        = "${var.environment}-pg"location    = var.locationrg_name     = module.rg.rg_namepg_version  = "13"sku_name    = "GP_Gen5_2"storage_mb  = 5120admin_user  = var.db_adminsubnet_id   = module.vnet.subnet_ids_map["db"]keyvault_id = module.keyvault.azurerm_key_vault.this.iddb_name     = "abpdb"tags        = local.common_tags
}

# outputs.tf
output "kubeconfig"     { value = module.aks.kube_config      sensitive = true }
output "db_conn_string" { value = module.rds.connection_string sensitive = true }

5. Helm Chart 打包与校验 📦

5.1 Chart.yaml & Chart.lock

# Chart.yaml
apiVersion: v2
name: abp-vnext
version: 0.4.0
appVersion: "1.0.0"
description: "ABP VNext 多服务 Kubernetes Umbrella Chart"
dependencies:- name: gatewayversion: "0.2.0"repository: file://charts/gateway- name: identityversion: "0.2.0"repository: file://charts/identity

helm dependency update infra/terraform/helm-charts/abp-vnext
helm dependency build  infra/terraform/helm-charts/abp-vnext

5.2 values.schema.json

{"$schema": "https://json-schema.org/draft/2020-12/schema","type": "object","properties": {"replicaCount": { "type": "integer" },"image": {"type": "object","properties": {"repository": { "type": "string" },"tag": { "type": "string" }},"required": ["repository","tag"]},"service": {"type": "object","properties": {"type": { "type": "string" },"port": { "type": "integer" }},"required": ["type","port"]}},"required": ["replicaCount","image","service"]
}

5.3 Subchart 示例（gateway）

# charts/gateway/values.yaml
replicaCount: 2
image:repository: myacr.azurecr.io/abp-gatewaytag: "1.0.0"
service:type: ClusterIPport: 80
resources:limits:cpu: "500m"memory: "512Mi"requests:cpu: "250m"memory: "256Mi"

# charts/gateway/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: {{ include "gateway.fullname" . }}
spec:replicas: {{ .Values.replicaCount }}selector:matchLabels:app: {{ include "gateway.name" . }}template:metadata:labels:app: {{ include "gateway.name" . }}spec:containers:- name: gatewayimage: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"ports:- containerPort: 80readinessProbe:httpGet:path: /health/readyport: 80initialDelaySeconds: 20periodSeconds: 10livenessProbe:httpGet:path: /health/liveport: 80initialDelaySeconds: 30periodSeconds: 15resources:{{ toYaml .Values.resources | indent 12 }}

其余 Service、Ingress、PDB、NetworkPolicy、HPA 与前文一致。

5.4 Lint & Test

helm lint infra/terraform/helm-charts/abp-vnext --strict
helm template abp-vnext infra/terraform/helm-charts/abp-vnext
helm package  infra/terraform/helm-charts/abp-vnext -d charts-packages

6. CI/CD 流水线 🔄

6.1 Infra Workflow

# .github/workflows/infra.yml
name: Infra – Terraformon:push:paths: ["infra/terraform/**"]concurrency:group: infra-${{ github.ref }}cancel-in-progress: truepermissions:id-token: writecontents: readjobs:terraform:runs-on: ubuntu-latestoutputs:kubeconfig: ${{ steps.apply.outputs.kubeconfig }}db_conn_string: ${{ steps.apply.outputs.db_conn_string }}steps:- uses: actions/checkout@v3- name: Setup Terraformuses: hashicorp/setup-terraform@v2- name: Terraform Fmt Checkrun: terraform fmt -checkworking-directory: infra/terraform- name: Terraform Validaterun: terraform validateworking-directory: infra/terraform- name: Terraform Initrun: terraform init -input=falseworking-directory: infra/terraform- name: Checkov Scanuses: bridgecrewio/checkov-action@masterwith:directory: infra/terraform- name: Infracost Estimateuses: infracost/actions@v2with:path: infra/terraformenv:INFRACOST_TOKEN: ${{ secrets.INFRACOST_TOKEN }}- name: Terraform Planid: planrun: terraform plan -out=tfplanworking-directory: infra/terraform- name: Terraform Applyid: applyrun: |terraform workspace select ${{ github.ref_name }} || terraform workspace new ${{ github.ref_name }}terraform apply -auto-approve tfplanecho "kubeconfig<<EOF" >> $GITHUB_OUTPUTterraform output -raw kubeconfig >> $GITHUB_OUTPUTecho "EOF" >> $GITHUB_OUTPUTecho "db_conn_string<<EOF" >> $GITHUB_OUTPUTterraform output -raw db_conn_string >> $GITHUB_OUTPUTecho "EOF" >> $GITHUB_OUTPUTworking-directory: infra/terraformenv:ARM_USE_MSI:          trueARM_SUBSCRIPTION_ID:  ${{ secrets.AZURE_SUBSCRIPTION_ID }}ARM_TENANT_ID:        ${{ secrets.AZURE_TENANT_ID }}

6.2 Deploy Workflow

# .github/workflows/deploy.yml
name: Deploy – Helmon:push:paths:- "src/**"- "infra/terraform/helm-charts/**"concurrency:group: deploy-${{ github.ref }}cancel-in-progress: truepermissions:id-token: writecontents: readenvironment:name: productionurl: https://abp.example.comreviewers:- alice- bobjobs:deploy:runs-on: ubuntu-latestneeds: terraformsteps:- uses: actions/checkout@v3- name: Azure Login via OIDCuses: azure/login@v1with:client-id:       ${{ secrets.AZURE_CLIENT_ID }}tenant-id:       ${{ secrets.AZURE_TENANT_ID }}subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}- name: AKS Set Contextuses: azure/aks-set-context@v1with:resource-group: production-rgcluster-name:   production-aks- name: Docker Login to ACRuses: docker/login-action@v2with:registry: myacr.azurecr.iousername: ${{ secrets.ACR_USERNAME }}password: ${{ secrets.ACR_PASSWORD }}- name: Build & Push Imagerun: |docker build -t myacr.azurecr.io/abp-vnext:${{ github.sha }} src/docker push myacr.azurecr.io/abp-vnext:${{ github.sha }}- name: Set DEPLOY_ENVrun: |if [ "${GITHUB_REF}" == "refs/heads/main" ]; thenecho "DEPLOY_ENV=prod" >> $GITHUB_ENVelseecho "DEPLOY_ENV=dev" >> $GITHUB_ENVfi- name: Helm Lint & Testrun: |helm lint infra/terraform/helm-charts/abp-vnext --stricthelm test --cleanup gateway- name: Helm Package & Pushrun: |helm package infra/terraform/helm-charts/abp-vnext -d charts-packageshelm push charts-packages/abp-vnext-*.tgz oci://myhelmrepo- name: Helm Upgrade with Rollbackrun: |set +ehelm repo add myrepo oci://myhelmrepohelm repo updatehelm upgrade --install abp-vnext myrepo/abp-vnext \--version 0.4.0 \-f infra/terraform/helm-charts/abp-vnext/values-${DEPLOY_ENV}.yaml \--set image.tag=${GITHUB_SHA} \--set-string env.DB_CONN="${{ needs.terraform.outputs.db_conn_string }}" \--wait --timeout 5mif [ $? -ne 0 ]; thenhelm rollback abp-vnext 1exit 1fiset -e- name: Verify Rolloutrun: kubectl rollout status deployment/gateway- name: Notify Slack on Successif: success()uses: 8398a7/action-slack@v3with:payload: '{"text":"✅ 部署成功：ABP VNext 微服务已更新到 '"${{ github.sha }}"'"}'channel: production-alertsenv:SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}- name: Notify Slack on Failureif: failure()uses: 8398a7/action-slack@v3with:payload: '{"text":"❌ 部署失败：请检查流水线日志！"}'channel: production-alertsenv:SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

7. 可观测性与告警 🔔

  • Azure Monitor：已开启 Container Insights
  • Prometheus/Grafana：可选部署，收集集群与业务指标
  • EFK/ELK：通过 DaemonSet 或 Sidecar 收集日志
  • Alertmanager：基于阈值触发告警，推送到 Slack/Teams

8. 附录 📂

参考资料：

  • Terraform Azure Provider 文档
  • Helm 官方文档
http://www.dtcms.com/a/263932.html

相关文章：

  • Kubernetes Pod 调度基础
  • 飞算JavaAI：重构软件开发范式的智能引擎
  • 【SQL知识】PDO 和 MySQLi 的区别
  • python __doc__
  • gateway白名单存储nacos，改成存储数据库
  • leetcode124-二叉树中的最大路径和
  • CPU缓存一致性
  • AI智能体在用户行为数据分析中有哪些应用？
  • 具身多模态大模型在感知与交互方面的综述
  • （十一）Spring WebSocket
  • Ansys Speos | Speos Camera 传感器机器视觉示例
  • vue-35（使用 Jest 和 Vue Test Utils 设置测试环境）
  • 列表元素滚动动画
  • LAN8720 寄存器概览和STM32 HAL库读写测试
  • CSS 安装使用教程
  • FreeRTOS任务切换
  • 力扣网C语言编程题：寻找两个正序数组的中位数
  • RIP 技术深度解析
  • 文心一言开源版测评：能力、易用性与价值的全面解析
  • [创业之路-457]：企业经营层 - 蓝海战略 - 价值创新不仅仅是技术创新
  • Java项目：基于SSM框架实现的智慧养老平台管理系统【ssm+B/S架构+源码+数据库+毕业论文】
  • 大麦基于HarmonyOS星盾安全架构，打造全链路安全抢票方案
  • 【机器学习深度学习】模型微调的基本概念与流程
  • 06会话管理
  • 前端计算机视觉：使用 OpenCV.js 在浏览器中实现图像处理
  • 量化选股策略 聚宽
  • 如何利用Charles中文版抓包工具提升API调试与网络性能
  • 二刷 苍穹外卖day10（含bug修改）
  • 如何使用StartUML绘制类图，用例图，时序图入门
  • 转录组分析流程(二)：差异分析