This repository represents the configuration of the mop node on a Proxmox server. Using Terraform (opentofu), cloud-init and proxmox/bpg provider It defines all virtual machines on the node mop:

gateway: Traefik gateway that reloads its configuration automatically when new entry is created in terraform.
bookshelf: A self developped service that manage a book database.
fefan: A self developped website for the Event of Girasol association Le Fefan
gitea: A gitea instance where all repository and actions are uploaded.
keycloak: A keycloak instance for IAM purposes.
wiki: A wikimedia website for a friend project
vaultwarden: A password manager.

Basic Proxmox setup

Add TerraformProv role

pveum role add TerraformProv -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Console VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"

Add terraform-prov user

pveum user add terraform-prov@pve --password <password>

Set terraform-prov user TerraformProv role

pveum aclmod / -user terraform-prov@pve -role TerraformProv

Create proxmox token for terraform API

pveum user token add terraform-prov@pve terraform -expire 0 -privsep 0 -comment "Terraform token"

Client Setup

Add environment variable

cp terraform.tfvars.example terraform.tfvars

Warning

Do no push this file

Usefull commands

OpenTofu

tofu init # initialize module
tofu plan # test configuration
tofu apply # apply configuration
tofu destroy # destroy all vm
tofu apply -target module.<module-name> # apply a specific module

On WSL

ssh agent could be off if ssh-add -L gives

Could not open a connection to your authentication agent.

start and configure ssh agent

eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519

Add new service

Create backup folder on proxmox host

mkdir /main/backups/<service-name>

Create a module

mkdir modules/apps/<module-hostname>

Example tree

modules/apps/bookshelf/
├── cloud-init
│   └── service.yaml
├── lib
│   ├── scripts
│   │   ├── env.sh
│   └── services
├── main.tf
├── output.tf
├── variables.tf
├── .env.example
└── .env

`modules/apps/<service-name>/main.tf`

module "vm" {
  source      = "../../vm"
  name        = var.name
  hostname    = var.hostname
  domain      = var.domain
  vm_id       = var.vm_id
  node_name   = var.node_name
  vm_ip_address = var.vm_ip_address

  template_id = var.template_id

  cores       = var.cores
  memory      = var.memory
  disk_size   = var.disk_size

  ssh_public_key  = var.ssh_public_key
  proxmox_host_ip = var.proxmox_host_ip

  cloudinit_config = templatefile(
      "${path.module}/cloud-init/service.yaml",
      {
        hostname        = var.hostname
        domain          = var.domain
        ssh_key         = var.ssh_public_key
        proxmox_host_ip = var.proxmox_host_ip

        env-file-content = indent(6, file("${path.module}/.env"))
      }
    )
}

Add inside templatefile() object scripts content to upload with cloud-init :

Backups scripts
Backups services
Install scripts
Application services

`modules/apps/<service-name>/variables.tf`

variable "name" {
  description = "Virtual Machine name"
  type = string
}

variable "vm_id" {
  description = "Virtual Machine id"
  type = number
}

variable "node_name" {
  description = "Proxmox node name"
  type = string
  default = "mop"
}

variable "cores" {
  description = "Number of CPU cores for this virtual machine"
  type = number
  default = 2
}

variable "memory" {
  description = "Memory RAM for this virtual machine" 
  type = number
  default = 2048
}

variable "balloon" {
  description = "Minimum vm memory, using ballooning devide to reach Proxmox node memory target."
  type = number
  default = 1024
}

variable "template_id" {
  description = "Virtual machine template ID"
  type = number
}

variable "ssh_public_key" {
  description = "Public SSH key for cloud-init user"
  type = string
}

variable "hostname" {
  description = "Virtual Machine hostname"
  type = string
  default = "test"
}

variable "domain" {
  description = "Virtual Machine domain"
  type = string
  default = ""
}

variable "disk_size" {
  description = "Disk size for the virtual machine"
  type = number
  default = 10
}

variable "proxmox_host_ip" {
  description = "Proxmox host base ip"
  type = string
}

variable "vm_ip_address" {
  description = "Virtual machine ip"
  type = string
}

`modules/apps/<service-name>/output.tf`

output "traefik_service" {
    value = [{
        domain = var.domain
        name = var.name
        host = "${var.hostname}"
        ip = var.vm_ip_address
        port = 80
    }]
}

This traefik_serive variable output.tf supports multiple service for one VM.

`cloud-init/service.yaml`

Base users, groups and ssh-key

#cloud-config
hostname: ${hostname}
local-hostname: ${hostname}
fqdn: ${hostname}.${domain}
manage_etc_hosts: true

users:
  - default
  - name: ${hostname}
    groups: sudo
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ${ssh_key}

disable_root: true

package_update: true
package_upgrade: false

Environment variables for scripts

write_files:
  - path: /opt/<service-name>/<service-name>.env
    permissions: "0644"
    content: |
      ${env-file-content}

.env.example files are given in each modules describing mandatory variables used by scripts.

Backup setup

packages:
  - nfs-common

mounts:
  - [ "192.168.1.12:/main/backups", "/backups", "nfs", "defaults,_netdev,x-systemd.requires=network-online.target", "0", "0" ]

write_files:
   - path: /usr/local/bin/restore-backup.sh
    permissions: "0755"
    content: |
      ${restore-backup-script}
  - path: /etc/systemd/system/restore-backup.service
    permissions: "0644"
    content: |
      ${restore-backup-service}
  - path: /usr/local/bin/backup.sh
    permissions: "0755"
    content: | 
      ${create-backup-script}
  - path: /etc/systemd/system/create-backup.timer
    permissions: "0644"
    content: |
      ${create-backup-timer}
  - path: /etc/systemd/system/create-backup.service
    permissions: "0644"
    content: |
      ${create-backup-service}

restore-backup.sh: Restore backup script (specific per module).
restore-backup.service: Restore backup systemd service (common for all modules).
create-backup.service: Create backup systemd service (common for all modules).
create-backup.timer: Create backup systemd timer (common for all modules).
nfs-common: NFS mount package for /main/backups mount point.
mounts: adds NFS mount point to /etc/fstab file.