当我试图升级Terraform版本时，我遇到了一个令人困惑的差异问题
※ 这篇文章是关于Mixi Group Advent Calendar 2021第11天的贡献.
我最近将我们在项目中创建的 Terraform 代码升级到最新的版本1.0.11。
在尝试升级版本时，出现了一些奇怪的差异，于是我进行了一些调查并记录下来。
请做一个中文的总结。
-
- Terraform コードを 0.14.11 -> 1.0.11 にアップグレードしようと plan を実行したところ、謎の差分が発生
-
- 調べると、 AWS Security Manager を使用して設定した値で差分が発生している
-
- どうも 0.15 以降は provider_sensitive_attrs の機能があり、sensitive な値を変数に入れて使うと、使っている resource の値も sensitive 扱いになってくれるらしい
-
- sensitive の扱いが変更されると、物によっては Terraform はそれを resource の差分として認識する（っぽい）
- というわけでアップグレードに伴って変更される resource はなかったので、みんな幸せ
起源
当发生这件事时，仅仅是由于试图升级Terraform并更改Terraform的required_version而创建了一个简单的PR。
在使用 CodeBuild 运行 terraform plan 的结果中出现了莫名其妙的差异。
差异发生在配置了 AWS WAF 的规则中。
而关键差异的内容是：
~ rule {
# At least one attribute in this block is (or was) sensitive,
# so its contents will not be displayed.
}
因为敏感的原因，无法展示差异。
这下麻烦了。
调查
在搜索了一下有没有办法检查差异之后，我找到了以下论坛的帖子：
https://discuss.hashicorp.com/t/how-to-show-sensitive-values/24076
根据它所说，执行以下命令可以将详细的计划执行方案以 JSON 格式输出，并能够检查差异。
terraform plan -out=tfplan
terraform show -json tfplan
JSON的内容在以下官方文档中有详细说明。
https://www.terraform.io/docs/internals/json-format.html
根据这个，在执行 terraform plan 命令时输出的配置大致如下。
-
- 現在 Terraform で管理している resources の状態（Values Representation）
-
- 現在の Terraform の設定（Configuration Representation）
- 今回の変更点（Change Representation）
在“变更表现方式”部分中，有一个“before/after”选项，其中详细描述了每个变更点。
这次的差异可以通过比较这里的“before”和“after”来了解。
因此，我们可以提取“before”和“after”，进行差异比较。
14c14
< "before": {
---
> "after": {
37c37
< "custom_request_handling": []
---
> "custom_request_handling": null
116c116
< "before_sensitive": {
---
> "after_sensitive": {
132,134c132
< {
< "custom_request_handling": []
< }
---
> {}
187c185
< },
---
> }
203c201
< "before": {
---
> "after": {
305c303,304
< "before_sensitive": {
---
> "after_unknown": {},
> "after_sensitive": {
316,368c315
< "rule": [
< {
< "action": [
< {
< "allow": [
< {}
< ],
< "block": [],
< "count": []
< }
< ],
< "override_action": [],
< "statement": [
< {
< "and_statement": [],
< "byte_match_statement": [
< {
< "field_to_match": [
< {
< "all_query_arguments": [],
< "body": [],
< "method": [],
< "query_string": [],
< "single_header": [
< {}
< ],
< "single_query_argument": [],
< "uri_path": []
< }
< ],
< "text_transformation": [
< {}
< ]
< }
< ],
< "geo_match_statement": [],
< "ip_set_reference_statement": [],
< "managed_rule_group_statement": [],
< "not_statement": [],
< "or_statement": [],
< "rate_based_statement": [],
< "regex_pattern_set_reference_statement": [],
< "rule_group_reference_statement": [],
< "size_constraint_statement": [],
< "sqli_match_statement": [],
< "xss_match_statement": []
< }
< ],
< "visibility_config": [
< {}
< ]
< }
< ],
---
> "rule": true,
374c321
< },
---
> }
看到这个，可以看出 before_sensitive 中的 WAF 规则完全被替换为 after_sensitive 中的 true。关于 before/after_sensitive，
// "before_sensitive" and "after_sensitive" are object values with similar
// structure to "before" and "after", but with all sensitive leaf values
// replaced with true, and all non-sensitive leaf values omitted.
据说，所有可能被视为敏感的参数都被设置为了 true。换句话说，可以看出 AWS WAF 的规则被全部视为敏感处理了。
哎呀，我在想是否有一些对规则敏感的数据…？但在这里，我记起了将AWS Secret Manager中获取的值放入规则中。
这个系统设置了一个WAF规则，只允许带有固定值的请求头来自系统。然后我们通过密钥管理器Secret Manager来管理这个固定值。
新增了 provider_sensitive_attrs 特性的版本是 0.15。
如果这样的话，我会想知道升级后是否有关于敏感值的处理变化？查阅后发现在0.15版本中提供了provider_sensitive_attrs选项并已成为通用可用。
https://dev.classmethod.jp/articles/terraform-015/#toc-4
提供者敏感属性（provider_sensitive_attrs）是指用于处理敏感数据的资源时同样将其视为敏感处理的功能。最初，当将敏感数据存储在变量中并在terraform plan等操作中以明文形式显示时，存在问题。为了解决这个问题，引入了这个功能。
由于Security Manager的值本身就是敏感的，因此可以看出WAF的规则也被完全视为敏感处理。
资源的更改？
其实，通常情况下，只要是敏感处理的更改，似乎会显示为“The value is unchanged（数值未变）”类似的内容。
# Warning: this block will be marked as sensitive and will not
# display in UI output after applying this change. The value is unchanged. <= ココ！
~ rule {
# At least one attribute in this block is (or was) sensitive,
# so its contents will not be displayed.
}
如果这个字出现了，就可以知道值没有被改变，但是这次它没有被显示出来。
看起来似乎是由于资源的原因，可能有时候不会显示出”The value is unchanged.”这个字来？
最后怎么样了？
价值没有改变。由于没有任何标识，并且这次的环境是生产环境，所以对其进行立即应用有点恐怖。因此，我创建了仅包括问题WAF和Secret Manager的简单Terraform，以验证实际情况如何。
验证
首先，在0.14.11版本中创建。
terraform {
required_version = "~> 0.14.11"
}
provider "aws" {
profile = "test"
region = "ap-northeast-1"
}
provider "aws" {
profile = "test"
region = "us-east-1"
alias = "virginia"
}
data "aws_secretsmanager_secret" "secret" {
name = "test"
}
data "aws_secretsmanager_secret_version" "secret" {
secret_id = data.aws_secretsmanager_secret.secret.id
}
resource "aws_wafv2_web_acl" "waf" {
provider = aws.virginia
name = "test-waf-allow-header"
scope = "CLOUDFRONT"
description = "test"
tags = {}
visibility_config {
cloudwatch_metrics_enabled = false
metric_name = "test-waf-allow-header"
sampled_requests_enabled = false
}
default_action {
block {}
}
rule {
name = "test-waf-rule-allow-header"
priority = 1
visibility_config {
cloudwatch_metrics_enabled = false
metric_name = "test-waf-rule-allow-header"
sampled_requests_enabled = false
}
action {
allow {}
}
statement {
byte_match_statement {
field_to_match {
single_header {
name = "x-cdn-key"
}
}
positional_constraint = "EXACTLY"
search_string = jsondecode(data.aws_secretsmanager_secret_version.secret.secret_string)["x-cdn-key"]
text_transformation {
priority = 0
type = "NONE"
}
}
}
}
}
执行已创建的 main.tf 文件。
% terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_wafv2_web_acl.waf will be created
+ resource "aws_wafv2_web_acl" "waf" {
+ arn = (known after apply)
+ capacity = (known after apply)
+ description = "test"
+ id = (known after apply)
+ lock_token = (known after apply)
+ name = "test-waf-allow-header"
+ scope = "CLOUDFRONT"
+ tags_all = (known after apply)
+ default_action {
+ block {
}
}
+ rule {
+ name = "test-waf-rule-allow-header"
+ priority = 1
+ action {
+ allow {
}
}
+ statement {
+ byte_match_statement {
+ positional_constraint = "EXACTLY"
+ search_string = "hogehoge"
+ field_to_match {
+ single_header {
+ name = "x-cdn-key"
}
}
+ text_transformation {
+ priority = 0
+ type = "NONE"
}
}
}
+ visibility_config {
+ cloudwatch_metrics_enabled = false
+ metric_name = "test-waf-rule-allow-header"
+ sampled_requests_enabled = false
}
}
+ visibility_config {
+ cloudwatch_metrics_enabled = false
+ metric_name = "test-waf-allow-header"
+ sampled_requests_enabled = false
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_wafv2_web_acl.waf: Creating...
aws_wafv2_web_acl.waf: Creation complete after 2s [id=ebd18307-4665-4bfa-8807-a53ae9df4779]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
在这种情况下，设定在 WAF 中的敏感值如下。
+ search_string = "hogehoge"
更改所需版本。
diff --git a/main.tf b/main.tf
index 699ddc8..e8af984 100644
--- a/main.tf
+++ b/main.tf
@@ -1,5 +1,5 @@
terraform {
- required_version = "~> 0.14.11"
+ required_version = "~> 1.0.11"
}
provider "aws" {
然后进行terraform规划。
% terraform plan
aws_wafv2_web_acl.waf: Refreshing state... [id=ebd18307-4665-4bfa-8807-a53ae9df4779]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# aws_wafv2_web_acl.waf has been changed
~ resource "aws_wafv2_web_acl" "waf" {
id = "ebd18307-4665-4bfa-8807-a53ae9df4779"
name = "test-waf-allow-header"
+ tags = {}
# (6 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes
using ignore_changes, the following plan may include actions to undo or respond to these changes.
────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_wafv2_web_acl.waf will be updated in-place
~ resource "aws_wafv2_web_acl" "waf" {
id = "ebd18307-4665-4bfa-8807-a53ae9df4779"
name = "test-waf-allow-header"
tags = {}
# (6 unchanged attributes hidden)
# Warning: this block will be marked as sensitive and will not
# display in UI output after applying this change.
~ rule {
# At least one attribute in this block is (or was) sensitive,
# so its contents will not be displayed.
}
# (2 unchanged blocks hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
────────────────────────────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly
these actions if you run "terraform apply" now.
果然，数值未变。没有”やはり”的表示。
直接应用这样，确认数值会如何变化。
% terraform apply [ main ]
aws_wafv2_web_acl.waf: Refreshing state... [id=ebd18307-4665-4bfa-8807-a53ae9df4779]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# aws_wafv2_web_acl.waf has been changed
~ resource "aws_wafv2_web_acl" "waf" {
id = "ebd18307-4665-4bfa-8807-a53ae9df4779"
name = "test-waf-allow-header"
+ tags = {}
# (6 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes
using ignore_changes, the following plan may include actions to undo or respond to these changes.
────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_wafv2_web_acl.waf will be updated in-place
~ resource "aws_wafv2_web_acl" "waf" {
id = "ebd18307-4665-4bfa-8807-a53ae9df4779"
name = "test-waf-allow-header"
tags = {}
# (6 unchanged attributes hidden)
# Warning: this block will be marked as sensitive and will not
# display in UI output after applying this change.
~ rule {
# At least one attribute in this block is (or was) sensitive,
# so its contents will not be displayed.
}
# (2 unchanged blocks hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_wafv2_web_acl.waf: Modifying... [id=ebd18307-4665-4bfa-8807-a53ae9df4779]
aws_wafv2_web_acl.waf: Modifications complete after 0s [id=ebd18307-4665-4bfa-8807-a53ae9df4779]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
在CloudTrail上没有任何记录，也没有看到任何在AWS上的更改。
因此，我判断没有问题，进行了申请。由于没有出现任何问题，所以很好。
最后
这篇文章写的时候，Terraform 1.1.0 已经发布了，所以还需要升级一下。
