terraform/README.md

Terraform

Basic Proxmox setup

Add TerraformProv role

pveum role add TerraformProv -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Console VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"

Add terraform-prov user

pveum user add terraform-prov@pve --password <password>

Set terraform-prov user TerraformProv role

pveum aclmod / -user terraform-prov@pve -role TerraformProv

Create proxmox token for terraform API

pveum user token add terraform-prov@pve terraform -expire 0 -privsep 0 -comment "Terraform token"

Client Setup

Add environment variable

cp terraform.tfvars.example

fill with your secrets (do no push this file)

Usefull commands

opentofu.tofu init
opentofu.tofu plan
opentofu.tofu apply
opentofu.tofu destroy
tofu apply -target module.<module-name>

On WSL

ssh agent could be off if ssh-add -L gives

Could not open a connection to your authentication agent.

start and configure ssh agent

eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519

Add new service

Create backup folder on proxmox host

mkdir /main/backups/<service-name>

Create a module

mkdir modules/apps/<module-hostname>

Example tree

modules/apps/bookshelf/
├── cloud-init
│   └── service.yaml
├── lib
│   ├── scripts
│   │   ├── env.sh
│   └── services
├── main.tf
├── output.tf
├── variables.tf
├── .env.example
└── .env

modules/apps/<service-name>/main.tf

module "vm" {
  source      = "../../vm"
  name        = var.name
  hostname    = var.hostname
  domain      = var.domain
  vm_id       = var.vm_id
  node_name   = var.node_name
  vm_ip_address = var.vm_ip_address

  template_id = var.template_id

  cores       = var.cores
  memory      = var.memory
  disk_size   = var.disk_size

  ssh_public_key  = var.ssh_public_key
  proxmox_host_ip = var.proxmox_host_ip
  cloudinit_config = templatefile(
      "${path.module}/cloud-init/service.yaml",
      {
        hostname        = var.hostname
        domain          = var.domain
        ssh_key         = var.ssh_public_key
        proxmox_host_ip = var.proxmox_host_ip
        environment-setup-script = indent(6, file("${path.module}/../common/scripts/env.sh"))
        env-file-content = indent(6, file("${path.module}/.env"))
      }
    )
}

Add inside templatefile() object scripts content to upload with cloud-init :

• Backups scripts
• Backups services
• Install scripts
• Application services

modules/apps/<service-name>/variables.tf

variable "name" {
  description = "Virtual Machine name"
  type = string
}

variable "vm_id" {
  description = "Virtual Machine id"
  type = number
}

variable "node_name" {
  description = "Proxmox node name"
  type = string
  default = "mop"
}

variable "cores" {
  description = "Number of CPU cores for this virtual machine"
  type = number
  default = 2
}

variable "memory" {
  description = "Memory RAM for this virtual machine" 
  type = number
  default = 2048
}

variable "balloon" {
  description = "Minimum vm memory, using ballooning devide to reach Proxmox node memory target."
  type = number
  default = 1024
}

variable "template_id" {
  description = "Virtual machine template ID"
  type = number
}

variable "ssh_public_key" {
  description = "Public SSH key for cloud-init user"
  type = string
}

variable "hostname" {
  description = "Virtual Machine hostname"
  type = string
  default = "test"
}

variable "domain" {
  description = "Virtual Machine domain"
  type = string
  default = ""
}

variable "disk_size" {
  description = "Disk size for the virtual machine"
  type = number
  default = 10
}

variable "proxmox_host_ip" {
  description = "Proxmox host base ip"
  type = string
}

variable "vm_ip_address" {
  description = "Virtual machine ip"
  type = string
}

modules/apps/<service-name>/output.tf

output "traefik_service" {
    value = [{
        domain = var.domain
        name = var.name
        host = "${var.hostname}"
        ip = var.vm_ip_address
        port = 80
    }]
}

This traefik_serive variable output.tf supports multiple service for one VM.

cloud-init/service.yaml

Base users, groups and ssh-key

#cloud-config
hostname: ${hostname}
local-hostname: ${hostname}
fqdn: ${hostname}.${domain}
manage_etc_hosts: true

users:
  - default
  - name: ${hostname}
    groups: sudo
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ${ssh_key}

disable_root: true

package_update: true
package_upgrade: false

Backup setup

packages:
  - nfs-common

mounts:
  - [ "192.168.1.12:/main/backups", "/backups", "nfs", "defaults,_netdev,x-systemd.requires=network-online.target", "0", "0" ]

nfs-common: NFS mount package for /main/backups mount point. mounts: adds NFS mount point to /etc/fstab file.

Environment variables for scripts

write_files:
  - path: /opt/<service-name>/env.sh
    permissions: "0644"
    content: |
      ${environment-setup-script}
  - path: /opt/<service-name>/<service-name>.env
    permissions: "0644"
    content: |
      ${env-file-content}