使用 AWS-S3 + Packer + Ansible + Vagrant 来构建可进行版本管理的开发环境
首先
Docker真不错啊哈哈
尽管大家普遍都知道容器虚拟化软件具有轻量级的优点，但在某些情况下，可能还是会选择主机虚拟化。
我想深入探讨一下使用VirtualBox + Vagrant配置并执行vagrant up的过程。
运行环境
主机
-
- macOS: 10.15.7
-
- Virtualbox: 6.0.22
-
- Vagrant: 2.2.9
vagrant-s3auth: 1.3.2
Packer: 1.4.5
packer-post-processor-vagrant-s3: 1.4.0
Ansible: 2.9.2
请用中文原生语言转述以下内容，只需提供一个选项：
访客机
-
- CentOS: 7.8
- Nginx: 1.18.0
最终的目录结构
operation
├── ansible
│ ├── development
│ │ ├── group_vars
│ │ │ └── all.yml
│ │ ├── host_vars
│ │ │ └── all.yml
│ │ └── hosts.yml
│ ├── production
│ │ └── .gitkeep
│ ├── roles
│ │ ├── nginx
│ │ │ ├── defaults
│ │ │ │ └── main.yml
│ │ │ ├── files
│ │ │ │ ├── development
│ │ │ │ │ └── .gitkeep
│ │ │ │ ├── production
│ │ │ │ │ └── .gitkeep
│ │ │ │ └── staging
│ │ │ │ └── .gitkeep
│ │ │ ├── handlers
│ │ │ │ └── main.yml
│ │ │ ├── meta
│ │ │ │ └── main.yml
│ │ │ ├── tasks
│ │ │ │ └── main.yml
│ │ │ ├── templates
│ │ │ │ ├── development
│ │ │ │ │ └── .gitkeep
│ │ │ │ ├── production
│ │ │ │ │ └── .gitkeep
│ │ │ │ └── staging
│ │ │ │ └── .gitkeep
│ │ │ ├── tests
│ │ │ │ ├── inventory
│ │ │ │ └── test.yml
│ │ │ └── vars
│ │ │ └── main.yml
│ ├── site.yml
│ └── staging
│ └── .gitkeep
├── packer
│ ├── development
│ │ ├── http
│ │ │ └── kickstart.ks
│ │ └── template.json
│ ├── production
│ │ └── .gitkeep
│ └── staging
│ └── .gitkeep
└── vagrant
└── Vagrantfile
平常的
$ mkdir operation/vagrant && cd operation/vagrant
$ vagrant init
$ echo -e 'Vagrant.configure("2") do |config|
config.vm.box = "bento/centos-7.8"
config.vm.network "forwarded_port", guest: 80, host: 8080, auto_correct: true
config.vm.network "private_network", ip: "192.168.33.10"
config.vm.synced_folder ".", "/vagrant", disabled: true
end
' > Vagrantfile
$ vagrant up
可以吸收VirtualBox复杂的启动选项，并且可以根据模板在一条命令下启动。
手动配置
$ vagrant ssh
$ touch /etc/yum.repos.d/nginx.repo && \
echo -e '[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
' > /etc/yum.repos.d/nginx.repo
$ yum install -y nginx
$ systemctl enable nginx && systemctl start nginx
连接到虚拟机，通过SSH手动执行命令
虽然简单且学习成本低，但可能会产生人为错误的空间余地。
自动配置(Ansible)
$ mkdir operation/ansible && \
mkdir operation/ansible/development && \
mkdir operation/ansible/development/hosts_va && \
touch operation/ansible/development/hosts_all/all.yml && \
mkdir operation/ansible/development/group_all && \
touch operation/ansible/development/group_all/all.yml && \
touch operation/ansible/development/hosts.yml && \
mkdir operation/ansible/staging && \
touch operation/ansible/staging/.gitkeep && \
mkdir operation/ansible/production && \
touch operation/ansible/production/.gitkeep && \
mkdir operation/ansible/roles
$ ansible-galaxy init --init-path=operation/ansible/roles nginx
$ mkdir operation/ansible/roles/nginx/files/development && \
touch operation/ansible/roles/nginx/files/development/.gitkeep && \
mkdir operation/ansible/roles/nginx/files/production && \
touch operation/ansible/roles/nginx/files/production/.gitkeep && \
mkdir operation/ansible/roles/nginx/files/staging && \
touch operation/ansible/roles/nginx/files/staging/.gitkeep && \
mkdir operation/ansible/roles/nginx/templates/development && \
touch operation/ansible/roles/nginx/templates/development/.gitkeep && \
mkdir operation/ansible/roles/nginx/templates/production && \
touch operation/ansible/roles/nginx/templates/production/.gitkeep && \
mkdir operation/ansible/roles/nginx/templates/staging && \
touch operation/ansible/roles/nginx/templates/staging/.gitkeep
$ echo -e 'all:
hosts:
localhost:
ansible_connection: local
' > operation/ansible/development/hosts.yml
$ echo -e '- hosts: all
become: yes
roles:
- nginx
' > operation/ansible/site.yml
$ echo -e '- name: add nginx repository
yum_repository:
file: nginx
name: nginx-stable
description: nginx stable repo
baseurl: http://nginx.org/packages/centos/$releasever/$basearch
gpgcheck: yes
gpgkey: https://nginx.org/keys/nginx_signing.key
enabled: yes
- name: install nginx
yum:
name: nginx
- name: service enable and start
service:
name: nginx
state: started
enabled: yes
' > operation/ansible/roles/nginx/tasks/main.yml
$ echo -e 'Vagrant.configure("2") do |config|
config.vm.box = "bento/centos-7.8"
config.vm.network "forwarded_port", guest: 80, host: 8080, auto_correct: true
config.vm.network "private_network", ip: "192.168.33.10"
config.vm.synced_folder "../ansible", "/home/vagrant/ansible"
config.vm.provision :ansible_local do |ansible|
ansible.install_mode = "pip"
ansible.playbook = "/home/vagrant/ansible/site.yml"
ansible.inventory_path = "/home/vagrant/ansible/development"
ansible.limit = "all"
end
end
' > operation/vagrant/Vagrantfile
$ cd operation/vagrant && \
vagrant halt && \
vagrant destroy -f && \
rm -rf .vagrant && \
vagrant up
安装配置管理工具到虚拟机，并执行自己的配置管理。这是一个常见的配置，但在配置过程中可能会出现故障排除的问题。
可供分发的开发环境存在的问题
-
- 由于每个开发者的环境都要执行供应，导致意外故障的可能性增加。
- 很难将故障排除的结果应用到整个系统中。
从分发可能到发布可能
尝试将模板分发给个人进行构建的流程，更改为分发预先构建好的开发环境并启动的流程。
创建和版本控制黄金镜像
$ mkdir operation/packer && \
mkdir operation/packer/development && \
mkdir operation/packer/development/http && \
touch operation/packer/development/http/kickstart.ks && \
touch operation/packer/development/template.json && \
mkdir operation/packer/production && \
touch operation/packer/production/.gitkeep && \
mkdir operation/packer/staging
touch operation/packer/staging/.gitkeep
$ echo -e '{
"variables": {
"environment": "development",
"vmName": "golden-image",
"vmVersion": "1.0.0",
"isoUrl": "http://ftp.riken.jp/Linux/centos/7.8.2003/isos/x86_64/CentOS-7-x86_64-Minimal-2003.iso",
"isoChecksum": "659691c28a0e672558b003d223f83938f254b39875ee7559d1a4a14c79173193",
"isoChecksumType": "sha256",
"guestAdditionsIsoFileName": "VBoxGuestAdditions.iso",
"diskSize": "10240",
"homeDirectory": "/home/vagrant",
"userName": "vagrant",
"passWord": "vagrant"
},
"builders": [
{
"type": "virtualbox-iso",
"vm_name": "{{user `vmName`}}",
"guest_os_type": "RedHat_64",
"iso_url": "{{user `isoUrl`}}",
"iso_checksum": "{{user `isoChecksum`}}",
"iso_checksum_type": "{{user `isoChecksumType`}}",
"http_directory": "http",
"guest_additions_path": "{{user `homeDirectory`}}/{{user `guestAdditionsIsoFileName`}}",
"boot_command": "<tab> text ks=http://{{.HTTPIP}}:{{.HTTPPort}}/kickstart.ks <enter>",
"shutdown_command": "echo 'vagrant' | sudo -S shutdown -h now",
"disk_size": "{{user `diskSize`}}",
"vboxmanage": [
["modifyvm", "{{ .Name }}", "--cpus", "2"],
["modifyvm", "{{ .Name }}", "--memory", "2048"],
["modifyvm", "{{ .Name }}", "--chipset", "ich9"],
["modifyvm", "{{ .Name }}", "--ioapic", "on"],
["modifyvm", "{{ .Name }}", "--rtcuseutc", "on"],
["modifyvm", "{{ .Name }}", "--pae", "on"],
["modifyvm", "{{ .Name }}", "--hwvirtex", "on"],
["modifyvm", "{{ .Name }}", "--nestedpaging", "on"],
["modifyvm", "{{ .Name }}", "--largepages", "on"],
["modifyvm", "{{ .Name }}", "--paravirtprovider", "kvm"],
["modifyvm", "{{ .Name }}", "--vram", "9"],
["modifyvm", "{{ .Name }}", "--vrde", "off"],
["modifyvm", "{{ .Name }}", "--graphicscontroller", "vboxsvga"],
["modifyvm", "{{ .Name }}", "--audio", "none"],
["storagectl", "{{ .Name }}", "--name", "IDE Controller", "--controller", "ICH6"]
],
"headless": true,
"ssh_wait_timeout": "10000s",
"ssh_username": "{{user `userName`}}",
"ssh_password": "{{user `passWord`}}"
}
],
"provisioners": [
{
"type": "ansible",
"playbook_file": "../../ansible/site.yml",
"inventory_directory": "../../ansible/{{user `environment`}}"
}
],
"post-processors": [
[
{
"type": "vagrant",
"compression_level": 9,
"output": "{{user `vmName`}}.box"
},
{
"type": "vagrant-s3",
"region": "ap-northeast-1",
"bucket": "{{user `environment`}}-virtual-machine-image",
"acl": "private",
"profile": "default",
"manifest": "{{user `vmName`}}/manifest.json",
"box_dir": "{{user `vmName`}}",
"box_name": "{{user `vmName`}}.box",
"version": "{{user `vmVersion`}}"
}
]
]
}
' > operation/packer/development/template.json
$ echo -e '# Install OS instead of upgrade
install
# Use CDROM installation media
cdrom
# System language
lang ja_JP.UTF-8
# Keyboard layouts
keyboard jp106
# network setting
network --bootproto=dhcp --onboot=on --device=eth0
# Firewall configuration
firewall --disabled
# SELinux configuration
selinux --disabled
# System Timezone
timezone Asia/Tokyo --utc
# System bootloader configuration
bootloader --location=mbr
# Use graphical install
text
# Do not configure the X Window System
skipx
# Clear the Master Boot Record
zerombr
# Partition clearing infomation
clearpart --all --initlabel
# Diskpartitioning information
autopart
# System authorization infomation
auth --useshadow --passalgo=sha512 --kickstart
# Root password
rootpw --iscrypted $1$we3.LxeP$ko6NFF4j8D02bbqgCS80r.
# Create user
user --name=vagrant --plaintext --password vagrant
# First boot
firstboot --disabled
# Machine reboot
reboot --eject
%post
echo "%vagrant ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vagrant
chmod 0440 /etc/sudoers.d/vagrant
yum update -y --exclude=centos*
%end
' > operation/packer/development/http/kickstart.ks
$ : > operation/ansible/development/hosts.yml
$ echo -e '- hosts: all
become: yes
roles:
- common
- {
role: guest_additions,
when: targetEnvironment == "development"
}
- nginx
' > operation/ansible/site.yml
$ echo -e 'targetEnvironment: development
userName: vagrant
userGroup: vagrant
homeDirectory: /home/vagrant
' > operation/ansible/development/group_vars/all.yml
$ ansible-galaxy init --init-path=operation/ansible/roles common
$ mkdir operation/ansible/roles/common/files/development && \
touch operation/ansible/roles/common/files/development/sshd_config && \
mkdir operation/ansible/roles/common/files/production && \
touch operation/ansible/roles/common/files/production/.gitkeep && \
mkdir operation/ansible/roles/common/files/staging && \
touch operation/ansible/roles/common/files/staging/.gitkeep && \
mkdir operation/ansible/roles/common/templates/development && \
touch operation/ansible/roles/common/templates/development/.gitkeep && \
mkdir operation/ansible/roles/common/templates/production && \
touch operation/ansible/roles/common/templates/production/.gitkeep && \
mkdir operation/ansible/roles/common/templates/staging && \
touch operation/ansible/roles/common/templates/staging/.gitkeep
$ echo -e '- name: create .ssh
file:
path: "{{ homeDirectory }}/.ssh"
state: directory
owner: "{{ userName }}"
group: "{{ userGroup }}"
mode: 0700
- name: set authorized_keys
copy:
src: "{{ targetEnvironment }}/authorized_keys"
dest: "{{ homeDirectory }}/.ssh"
owner: "{{ userName }}"
group: "{{ userGroup }}"
mode: 0600
- name: set sshd_config
copy:
src: "{{ targetEnvironment }}/sshd_config"
dest: /etc/ssh
owner: root
group: root
mode: 0600
' > operation/ansible/roles/common/tasks/main.yml
$ wget https://raw.githubusercontent.com/hashicorp/vagrant/master/keys/vagrant.pub -O operation/ansible/roles/common/files/development/authorized_keys
$ echo -e '# $OpenBSD: sshd_config,v 1.100 2016/08/15 12:32:04 naddy Exp $
# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.
# This sshd was compiled with PATH=/usr/local/bin:/usr/bin
# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options override the
# default value.
# If you want to change the port on a SELinux system, you have to tell
# SELinux about this change.
# semanage port -a -t ssh_port_t -p tcp #PORTNUMBER
#
#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::
HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
# Ciphers and keying
#RekeyLimit default none
# Logging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel INFO
# Authentication:
#LoginGraceTime 2m
#PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
PubkeyAuthentication yes
# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile .ssh/authorized_keys
#AuthorizedPrincipalsFile none
#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don"t trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don"t read the user"s ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication yes
# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no
# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
#KerberosUseKuserok yes
# GSSAPI options
GSSAPIAuthentication no
GSSAPICleanupCredentials no
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no
#GSSAPIEnablek5users no
# Set this to "yes" to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to "no".
# WARNING: "UsePAM no" is not supported in Red Hat Enterprise Linux and may cause several
# problems.
UsePAM yes
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation sandbox
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#ShowPatchLevel no
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none
# no default banner path
#Banner none
# Accept locale-related environment variables
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
# override default of no subsystems
Subsystem sftp /usr/libexec/openssh/sftp-server
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# PermitTTY no
# ForceCommand cvs server
UseDNS no
' > operation/ansible/roles/common/files/development/sshd_config
$ ansible-galaxy init --init-path=operation/ansible/roles guest_additions
$ mkdir operation/ansible/roles/guest_additions/files/development && \
touch operation/ansible/roles/guest_additions/files/development/.gitkeep && \
mkdir operation/ansible/roles/guest_additions/files/production && \
touch operation/ansible/roles/guest_additions/files/production/.gitkeep && \
mkdir operation/ansible/roles/guest_additions/files/staging && \
touch operation/ansible/roles/guest_additions/files/staging/.gitkeep && \
mkdir operation/ansible/roles/guest_additions/templates/development && \
touch operation/ansible/roles/guest_additions/templates/development/.gitkeep && \
mkdir operation/ansible/roles/guest_additions/templates/production && \
touch operation/ansible/roles/guest_additions/templates/production/.gitkeep && \
mkdir operation/ansible/roles/guest_additions/templates/staging && \
touch operation/ansible/roles/guest_additions/templates/staging/.gitkeep
$ echo -e '- name: get kernel version
shell: uname -r
register: kernelVersion
- name: install guest additions requirement
yum:
name: "{{ packages }}"
vars:
packages: "{{ guestAdditionsRequirementPackageList }}"
- name: create mount directory
file:
path: "{{ guestAdditionsPath }}"
state: directory
owner: root
group: root
mode: 0700
- name: mount guest additions
mount:
path: "{{ guestAdditionsPath }}"
src: "{{ homeDirectory }}/{{ guestAdditionsIsoFileName }}"
state: mounted
opts: ro,loop
fstype: iso9660
- name: install guest additions
shell: "sh {{ guestAdditionsInstallerFileName }}"
args:
chdir: "{{ guestAdditionsPath }}"
- name: unmount guest additions
mount:
path: "{{ guestAdditionsPath }}"
src: "{{ homeDirectory }}/{{ guestAdditionsIsoFileName }}"
state: absent
- name: service start and enable vboxadd
service:
name: "{{ item }}"
state: started
enabled: yes
with_items: "{{ guestAdditionsService }}"
' > operation/ansible/roles/guest_additions/tasks/main.yml
$ echo -e 'guestAdditionsRequirementPackageList:
- gcc
- make
- perl
- bzip2
- "kernel-devel-{{ kernelVersion.stdout }}"
guestAdditionsPath: /tmp/guest_additions
guestAdditionsIsoFileName: VBoxGuestAdditions.iso
guestAdditionsInstallerFileName: VBoxLinuxAdditions.run
guestAdditionsService:
- vboxadd
- vboxadd-service
' > operation/ansible/roles/guest_additions/vars/main.yml
$ cd operation/packer/development && packer build template.json
如果要根据自动配置（Ansible）进行调整，
使用ansible-local可能更好…orz
基于操作系统映像文件，使用Kickstart自动设置虚拟机。
启动后，使用配置管理工具进行配置，并将其作为vagrant的box文件输出，然后上传至S3。
启动已传送的开发环境。
$ cd operation/vagrant
$ vagrant halt && \
vagrant destroy -f && \
rm -rf .vagrant
$ echo -e 'ENV.delete_if { |name| name.start_with?("AWS_") }
ENV["AWS_PROFILE"] = "default"
Vagrant.configure("2") do |config|
config.vm.box = "golden-image.box"
config.vm.box_url = "https://s3-ap-northeast-1.amazonaws.com/development-virtual-machine-image/golden-image/manifest.json"
config.vm.network "forwarded_port", guest: 80, host: 8080, auto_correct: true
config.vm.network "private_network", ip: "192.168.33.10"
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.provider :virtualbox do |vbox|
vbox.name = "golden-image"
end
end
' > Vagrantfile
$ vagrant up
根据 S3 上的清单，正在下载并启动指定的盒子文件。如果未指定版本，则会下载最新版本的盒子文件；如果指定了版本，则会下载相应版本的盒子文件。
好处
-
- 只需要启动已经进行了配置管理的box文件，因此出现错误的可能性很小。
-
- 容易反映故障排除的结果。
- 能够明确定义Dev和Ops的角色，使他们可以在各自擅长的领域中战斗。
坏的地方 de
-
- 由于需要使用AWS-S3，会产生少量的费用。
- 由于依赖第三方插件，因此取决于提供方的开发状况。
附言
構建此配置時，我不知道可以在设置文件中自动启动操作系统，所以我遇到了很大的困难… 哈哈
順便提一下，在Ubuntu的情况下，您可以将preseed上传到虚拟机并进行设置。
既然使用 Docker 也能做类似的事情，所以无论选择哪个都可以尝试并享受乐趣。
