297 lines
7.3 KiB
Markdown
297 lines
7.3 KiB
Markdown
# Terraform
|
|
This repository represents the configuration of the `mop` node on a Proxmox server.
|
|
Using Terraform (opentofu), cloud-init and proxmox/bpg provider
|
|
It defines all virtual machines on the node `mop`:
|
|
- `gateway`: Traefik gateway that reloads its configuration automatically when new entry is created in terraform.
|
|
- `bookshelf`: A self developped service that manage a book database.
|
|
- `fefan`: A self developped website for the Event of Girasol association `Le Fefan`
|
|
- `gitea`: A gitea instance where all repository and actions are uploaded.
|
|
- `keycloak`: A keycloak instance for IAM purposes.
|
|
- `wiki`: A wikimedia website for a friend [project](https://benoit.mathieu.wiki)
|
|
- `vaultwarden`: A password manager.
|
|
|
|
## Basic Proxmox setup
|
|
Add `TerraformProv` role
|
|
```sh
|
|
pveum role add TerraformProv -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Console VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"
|
|
```
|
|
|
|
Add `terraform-prov` user
|
|
```sh
|
|
pveum user add terraform-prov@pve --password <password>
|
|
```
|
|
|
|
Set `terraform-prov` user `TerraformProv` role
|
|
```sh
|
|
pveum aclmod / -user terraform-prov@pve -role TerraformProv
|
|
```
|
|
|
|
Create proxmox token for terraform API
|
|
```sh
|
|
pveum user token add terraform-prov@pve terraform -expire 0 -privsep 0 -comment "Terraform token"
|
|
```
|
|
|
|
## Client Setup
|
|
Add environment variable
|
|
```sh
|
|
cp terraform.tfvars.example terraform.tfvars
|
|
```
|
|
|
|
---
|
|
> **WARNING**
|
|
> Do no push this file
|
|
---
|
|
|
|
## Usefull commands
|
|
### OpenTofu
|
|
```sh
|
|
tofu init # initialize module
|
|
tofu plan # test configuration
|
|
tofu apply # apply configuration
|
|
tofu destroy # destroy all vm
|
|
tofu apply -target module.<module-name> # apply a specific module
|
|
```
|
|
|
|
### On WSL
|
|
ssh agent could be off
|
|
if `ssh-add -L` gives
|
|
```sh
|
|
Could not open a connection to your authentication agent.
|
|
```
|
|
start and configure ssh agent
|
|
```sh
|
|
eval $(ssh-agent)
|
|
ssh-add ~/.ssh/id_ed25519
|
|
```
|
|
|
|
## Add new service
|
|
### Create backup folder on proxmox host
|
|
```sh
|
|
mkdir /main/backups/<service-name>
|
|
```
|
|
|
|
### Create a module
|
|
```sh
|
|
mkdir modules/apps/<module-hostname>
|
|
```
|
|
|
|
Example tree
|
|
```sh
|
|
modules/apps/bookshelf/
|
|
├── cloud-init
|
|
│ └── service.yaml
|
|
├── lib
|
|
│ ├── scripts
|
|
│ │ ├── env.sh
|
|
│ └── services
|
|
├── main.tf
|
|
├── output.tf
|
|
├── variables.tf
|
|
├── .env.example
|
|
└── .env
|
|
```
|
|
|
|
#### `modules/apps/<service-name>/main.tf`
|
|
|
|
```hcl
|
|
module "vm" {
|
|
source = "../../vm"
|
|
name = var.name
|
|
hostname = var.hostname
|
|
domain = var.domain
|
|
vm_id = var.vm_id
|
|
node_name = var.node_name
|
|
vm_ip_address = var.vm_ip_address
|
|
|
|
template_id = var.template_id
|
|
|
|
cores = var.cores
|
|
memory = var.memory
|
|
disk_size = var.disk_size
|
|
|
|
ssh_public_key = var.ssh_public_key
|
|
proxmox_host_ip = var.proxmox_host_ip
|
|
|
|
cloudinit_config = templatefile(
|
|
"${path.module}/cloud-init/service.yaml",
|
|
{
|
|
hostname = var.hostname
|
|
domain = var.domain
|
|
ssh_key = var.ssh_public_key
|
|
proxmox_host_ip = var.proxmox_host_ip
|
|
|
|
env-file-content = indent(6, file("${path.module}/.env"))
|
|
}
|
|
)
|
|
}
|
|
```
|
|
|
|
Add inside `templatefile()` object scripts content to upload with `cloud-init` :
|
|
- Backups scripts
|
|
- Backups services
|
|
- Install scripts
|
|
- Application services
|
|
|
|
#### `modules/apps/<service-name>/variables.tf`
|
|
```hcl
|
|
variable "name" {
|
|
description = "Virtual Machine name"
|
|
type = string
|
|
}
|
|
|
|
variable "vm_id" {
|
|
description = "Virtual Machine id"
|
|
type = number
|
|
}
|
|
|
|
variable "node_name" {
|
|
description = "Proxmox node name"
|
|
type = string
|
|
default = "mop"
|
|
}
|
|
|
|
variable "cores" {
|
|
description = "Number of CPU cores for this virtual machine"
|
|
type = number
|
|
default = 2
|
|
}
|
|
|
|
variable "memory" {
|
|
description = "Memory RAM for this virtual machine"
|
|
type = number
|
|
default = 2048
|
|
}
|
|
|
|
variable "balloon" {
|
|
description = "Minimum vm memory, using ballooning devide to reach Proxmox node memory target."
|
|
type = number
|
|
default = 1024
|
|
}
|
|
|
|
variable "template_id" {
|
|
description = "Virtual machine template ID"
|
|
type = number
|
|
}
|
|
|
|
variable "ssh_public_key" {
|
|
description = "Public SSH key for cloud-init user"
|
|
type = string
|
|
}
|
|
|
|
variable "hostname" {
|
|
description = "Virtual Machine hostname"
|
|
type = string
|
|
default = "test"
|
|
}
|
|
|
|
variable "domain" {
|
|
description = "Virtual Machine domain"
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "disk_size" {
|
|
description = "Disk size for the virtual machine"
|
|
type = number
|
|
default = 10
|
|
}
|
|
|
|
variable "proxmox_host_ip" {
|
|
description = "Proxmox host base ip"
|
|
type = string
|
|
}
|
|
|
|
variable "vm_ip_address" {
|
|
description = "Virtual machine ip"
|
|
type = string
|
|
}
|
|
|
|
|
|
```
|
|
|
|
#### `modules/apps/<service-name>/output.tf`
|
|
|
|
```hcl
|
|
output "traefik_service" {
|
|
value = [{
|
|
domain = var.domain
|
|
name = var.name
|
|
host = "${var.hostname}"
|
|
ip = var.vm_ip_address
|
|
port = 80
|
|
}]
|
|
}
|
|
```
|
|
This `traefik_serive` variable `output.tf` supports multiple service for one VM.
|
|
|
|
#### `cloud-init/service.yaml`
|
|
##### Base users, groups and ssh-key
|
|
```hcl
|
|
#cloud-config
|
|
hostname: ${hostname}
|
|
local-hostname: ${hostname}
|
|
fqdn: ${hostname}.${domain}
|
|
manage_etc_hosts: true
|
|
|
|
users:
|
|
- default
|
|
- name: ${hostname}
|
|
groups: sudo
|
|
shell: /bin/bash
|
|
sudo: ALL=(ALL) NOPASSWD:ALL
|
|
ssh_authorized_keys:
|
|
- ${ssh_key}
|
|
|
|
disable_root: true
|
|
|
|
package_update: true
|
|
package_upgrade: false
|
|
```
|
|
|
|
##### Environment variables for scripts
|
|
```hcl
|
|
write_files:
|
|
- path: /opt/<service-name>/<service-name>.env
|
|
permissions: "0644"
|
|
content: |
|
|
${env-file-content}
|
|
```
|
|
|
|
`.env.example` files are given in each modules describing mandatory variables used by scripts.
|
|
|
|
##### Backup setup
|
|
```hcl
|
|
packages:
|
|
- nfs-common
|
|
|
|
mounts:
|
|
- [ "192.168.1.12:/main/backups", "/backups", "nfs", "defaults,_netdev,x-systemd.requires=network-online.target", "0", "0" ]
|
|
|
|
write_files:
|
|
- path: /usr/local/bin/restore-backup.sh
|
|
permissions: "0755"
|
|
content: |
|
|
${restore-backup-script}
|
|
- path: /etc/systemd/system/restore-backup.service
|
|
permissions: "0644"
|
|
content: |
|
|
${restore-backup-service}
|
|
- path: /usr/local/bin/backup.sh
|
|
permissions: "0755"
|
|
content: |
|
|
${create-backup-script}
|
|
- path: /etc/systemd/system/create-backup.timer
|
|
permissions: "0644"
|
|
content: |
|
|
${create-backup-timer}
|
|
- path: /etc/systemd/system/create-backup.service
|
|
permissions: "0644"
|
|
content: |
|
|
${create-backup-service}
|
|
```
|
|
- `restore-backup.sh`: Restore backup script (specific per module).
|
|
- `restore-backup.service`: Restore backup `systemd` service (common for all modules).
|
|
- `create-backup.service`: Create backup `systemd` service (common for all modules).
|
|
- `create-backup.timer`: Create backup `systemd` timer (common for all modules).
|
|
- `nfs-common`: NFS mount package for `/main/backups` mount point.
|
|
- `mounts`: adds NFS mount point to `/etc/fstab` file. |