使用Terraform和Ansible在KVM上自动部署k3s

3 年 ago
科, 雅
3 minutes

目的 – 我只需要一种选择 :

当需要进行K8s验证时，创建K8s集群本身需要花费很多时间。在这次中，我们将使用Ansible和Terraform来解决时间上的瓶颈问题。

构成

terasible_page-0001.jpg

目录结构

    • main.tf

 

    • provider.tf

 

    • k3s-ansible

ansible.cfg
site.yml
inventory

my-cluster

hosts.ini
group_vars
– all.yml

Terraform的配置

提供者.tf

由于本次的VM供应商是基于KVM主机的，所以我们将使用libvirt提供者。

provider "libvirt" {
  uri = "qemu+ssh://shoma@172.24.20.3/system"
}

resource "libvirt_volume" "os-image" {
  count = 5
  name   = "${format("terraform-vm%02d.qcow2", count.index + 1)}"
  source = "https://cloud-images.ubuntu.com/releases/23.04/release-20230714/ubuntu-23.04-server-cloudimg-amd64.img"
  format = "qcow2"
}


resource "libvirt_cloudinit_disk" "cloudinit_terraform-vm" {
  count = 5
  name = "${format("terraform-vm%02d.iso", count.index + 1)}"
  pool = "default"

  user_data = <<EOF
    #cloud-config
    hostname: "${format("terraform-vm%02d.qcow2", count.index + 1)}"
    user: tmcit
    password: tmcit
    chpasswd: { expire: False }
    ssh_pwauth: True
    EOF

  network_config = <<EOF
    version: 2
    ethernets:
      ens3:
        addresses:
          - "${format("172.24.20.20%d/24", count.index + 1)}"
        gateway4: 172.24.20.254
        nameservers:
          addresses: [172.24.2.51]
    EOF
}



resource "libvirt_domain" "terraform-vm" {
  count = 5
  name = "${format("ubuntu-terraform%02d", count.index + 1)}"
  memory = 2048
  vcpu = 2

  disk { volume_id = libvirt_volume.os-image[count.index].id }

  cloudinit = libvirt_cloudinit_disk.cloudinit_terraform-vm[count.index].id

  cpu {
    mode = "host-passthrough"
  }

  network_interface {
    bridge = "bridge1"
  }

  console {
    type = "pty"
    target_port = "0"
  }

  graphics {
    type = "vnc"
    listen_type = "address"
    autoport = "true"
  }
}


resource "null_resource" "ssh-keydelete" {
  count = 5

  triggers = {
    vm_id = libvirt_domain.terraform-vm[count.index].id
  }

  provisioner "local-exec" {
    command = <<COMMAND
    sleep 45
    ssh-keygen -R ${format("172.24.20.20%d", count.index + 1)}
    COMMAND
  }
}


resource "null_resource" "ssh_key" {
  count = 5

  triggers = {
    vm_id = libvirt_domain.terraform-vm[count.index].id
  }

  provisioner "local-exec" {
    command = <<COMMAND
    sleep 60
    sshpass -p tmcit ssh-copy-id -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa.pub tmcit@${format("172.24.20.20%d", count.index + 1)}
    COMMAND
  }
}


resource "null_resource" "run_ansible" {
  count = length(libvirt_domain.terraform-vm)

  triggers = {
    vm_id = libvirt_domain.terraform-vm[count.index].id
  }

  provisioner "local-exec" {
    command = <<COMMAND
    sleep 75
    cd k3s-ansible
    ansible-playbook site.yml
    COMMAND
  }
}

Ansible的配置

k3s-ansible/inventory/my-cluster/hosts.ini的翻译如下：

[master]
172.24.20.201

[node]
172.24.20.202
172.24.20.203
172.24.20.204
172.24.20.205

[k3s_cluster:children]
master
node

k3s-ansible/site.yml的翻译如下：

---

- hosts: k3s_cluster
  gather_facts: yes
  become: yes
  roles:
    - role: prereq
    - role: download
    - role: raspberrypi

- hosts: master
  become: yes
  roles:
    - role: k3s/master

- hosts: node
  become: yes
  roles:
    - role: k3s/node

k3s-ansible/ansible.cfg：

修改库存部分

[defaults]
nocows = True
roles_path = ./roles
inventory  = inventory/my-cluster/hosts.ini

remote_tmp = $HOME/.ansible/tmp
local_tmp  = $HOME/.ansible/tmp
pipelining = True
become = True
host_key_checking = False
deprecation_warnings = False
callback_whitelist = profile_tasks

确认

$ terraform apply
$ ssh tmcit@172.24.20.201
$ kubectl get nodess
NAME             STATUS   ROLES                  AGE    VERSION
terraform-vm05   Ready    <none>                 57s    v1.22.3+k3s1
terraform-vm04   Ready    <none>                 57s    v1.22.3+k3s1
terraform-vm02   Ready    <none>                 57s    v1.22.3+k3s1
terraform-vm03   Ready    <none>                 57s    v1.22.3+k3s1
terraform-vm01   Ready    control-plane,master   106s   v1.22.3+k3s1
bannerAds