* provider/aws: Fixed DynamoDB GSI update when using multiple indexes
* provider/aws: Fixed DynamoDB GSI set hash function
* Added DynamoDB table state migration
Redshift was changed to not skip snapshots by default, so our configs
were out of date and causing errors in destroy (thus leaking redshifts)
This changes the configs to skip snapshots, which should at least fix:
- TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates
- TestAccAWSRedshiftCluster_loggingEnabled
* provider/aws: Fix KMS Key reading with Exists method
Fixes#13322 by checking if the key Exists and offering to recreate if
not found, or pending delete
* remove redundant code
Add getters for the AWSClient s3.S3 and dynamodb.DynamoDB clients so the
s3 remote-state backend can use all the same initialization code as the
aws provider.
* provider/aws: Add support for treat_missing_data to cloudwatch_metric_alarm
Fixes: #13263
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSCloudWatchMetricAlarm'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/05 08:51:06 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSCloudWatchMetricAlarm -timeout 120m
=== RUN TestAccAWSCloudWatchMetricAlarm_importBasic
--- PASS: TestAccAWSCloudWatchMetricAlarm_importBasic (23.93s)
=== RUN TestAccAWSCloudWatchMetricAlarm_basic
--- PASS: TestAccAWSCloudWatchMetricAlarm_basic (27.81s)
=== RUN TestAccAWSCloudWatchMetricAlarm_treatMissingData
--- PASS: TestAccAWSCloudWatchMetricAlarm_treatMissingData (43.39s)
=== RUN TestAccAWSCloudWatchMetricAlarm_extendedStatistic
--- PASS: TestAccAWSCloudWatchMetricAlarm_extendedStatistic (26.80s)
=== RUN TestAccAWSCloudWatchMetricAlarm_missingStatistic
--- PASS: TestAccAWSCloudWatchMetricAlarm_missingStatistic (5.95s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 127.899s
```
* provider/aws: Set cloudwatch_metric_alarm treamt_missing_data to missing
This follows what the AWS API does. We had to add a state migration for
this to make sure that the user doesn't see any unexpected activity on
their Terraform plans
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAWSCloudWatchMetricAlarmMigrateState'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/05 14:51:32 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAWSCloudWatchMetricAlarmMigrateState -timeout 120m
=== RUN TestAWSCloudWatchMetricAlarmMigrateState
2017/04/05 14:52:13 [INFO] Found AWS CloudWatch Metric Alarm State v0; migrating to v1
2017/04/05 14:52:13 [DEBUG] Attributes before migration: map[string]string{}
2017/04/05 14:52:13 [DEBUG] Attributes after migration: map[string]string{"treat_missing_data":"missing"}
--- PASS: TestAWSCloudWatchMetricAlarmMigrateState (0.00s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 0.018s
```
The AWS waiter package has changed location in the 1.8.0 version of the
SDK. DMS will need to mitigate a breaking change because of this
Between @radeksimko and myself, we think that we should migrate the DMS
resources to using the Terraform state refresh func pattern that is used
across the entire of the AWS provider. DMS is the *only* resource that
currently uses the AWS waiters, so the LOE to migrate is pretty low
For our ECS service definition we have this snippet at the
`load_balancer`.
The `target_group_arn` is being pupulated by an external service that
returns the arn based on a simple string from our microservices list.
If the arn changed, this would not cause a recreation of the service and
leaving a dangling pointer to an arn that does not exist anymore.
```
load_balancer {
target_group_arn = "${lookup(var.target_group_mapping, element(values(var.microservices), count.index))}"
container_name = "${element(values(var.microservices), count.index)}"
container_port = "${var.container_port}"
}
```
The fix is adding another field to the set that's creating the ELB/ALB
definition. From looking into the git history seems this code was
created prior to ALB thus not having this field available at the time.
Service is being recreated as expected, no other services are affected
(expected behavior)
The Default values set by AWS were breaking the AWS ALB Listener Rule
tests. The stickiness parameter needed to be set to be Computed: true to
remove this discrepancy
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSALBListenerRule_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/03 01:23:47 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSALBListenerRule_basic -timeout 120m
=== RUN TestAccAWSALBListenerRule_basic
--- PASS: TestAccAWSALBListenerRule_basic (235.36s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 235.397s
```
It appears that #13232 doesn't work properly if you actually try to set any `parameter`s. Specifically, I was getting the following error:
```
* aws_db_parameter_group.test: 1 error(s) occurred:
* aws_db_parameter_group.test: Error modifying DB Parameter Group: InvalidParameterValue: The parameter DBParameterGroupName must be provided and must not be blank.
status code: 400, request id: 5783e396-17ff-11e7-87d5-e3fd4c7025ce
```
* provider/aws: New resource: aws_ses_domain_identity
Provide a resource to manage domain identities in SES. Exports the
verification_code attribute which can be used to add the TXT record to
the domain to complete the domain verification.
* provider/aws: Acceptance tests for aws_ses_domain_identity
* Resource aws_ses_domain_identity: Documentation update
Provide documentation for the new resource type.
Fixes: #9959
When we specify that we want an opsworks_instance state of running, we
should wait for that the be the case. This will then allow us to use the
Computed values (e.g. private_ip) etc and allow us to use provisioners
as part of the terraform config
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSOpsworksInstance'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 20:55:21 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSOpsworksInstance -timeout 120m
=== RUN TestAccAWSOpsworksInstance_importBasic
--- PASS: TestAccAWSOpsworksInstance_importBasic (72.28s)
=== RUN TestAccAWSOpsworksInstance
--- PASS: TestAccAWSOpsworksInstance (110.17s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 182.479s
```
Fixes: #8847
We actually didn't get the list of tags from the API, therefore, any
manual changes were not actually showing up in subsequent plans
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSALBTargetGroup_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 15:45:53 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSALBTargetGroup_basic -timeout 120m
=== RUN TestAccAWSALBTargetGroup_basic
--- PASS: TestAccAWSALBTargetGroup_basic (62.76s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 62.787s
```
Fixes: #13167
When changes to the target group were made via CLI or AWS Console, they
were not being picked up by terraform. This is because we were not
catching an error setting the `stickiness` parameters:
```
Error refreshing state: 1 error(s) occurred:
* aws_alb_target_group.test: aws_alb_target_group.test: stickiness.0.enabled: '' expected type 'bool', got unconvertible type 'string'
```
This meant that changes were not picked up in the following plan. The changes mean the following now:
```
~ aws_alb_target_group.test
stickiness.0.cookie_duration: "10440" => "10000"
stickiness.0.enabled: "false" => "true"
Plan: 0 to add, 1 to change, 0 to destroy.
```
Fixes: #13216
Prior to Terraform 0.9.2, we always set placement_strategies to
lowercase. Therefore, people using it in Terraform 0.9.2 are getting
continual diffs:
```
-/+ aws_ecs_service.mongo
cluster: "arn:aws:ecs:us-west-2:187416307283:cluster/terraformecstest1" => "arn:aws:ecs:us-west-2:187416307283:cluster/terraformecstest1"
deployment_maximum_percent: "200" => "200"
deployment_minimum_healthy_percent: "100" => "100"
desired_count: "1" => "1"
name: "mongodb" => "mongodb"
placement_strategy.#: "1" => "1"
placement_strategy.1676812570.field: "instanceid" => "" (forces new resource)
placement_strategy.1676812570.type: "spread" => "" (forces new resource)
placement_strategy.3946258308.field: "" => "instanceId" (forces new resource)
placement_strategy.3946258308.type: "" => "spread" (forces new resource)
task_definition: "arn:aws:ecs:us-west-2:187416307283:task-definition/mongodb:1991" => "arn:aws:ecs:us-west-2:187416307283:task-definition/mongodb:1991"
Plan: 1 to add, 0 to change, 1 to destroy.
```
This adds a DiffSuppression func to make sure this doesn't trigger a
ForceNew resource:
```
% terraform plan ✹ ✭
[WARN] /Users/stacko/Code/go/bin/terraform-provider-aws overrides an internal plugin for aws-provider.
If you did not expect to see this message you will need to remove the old plugin.
See https://www.terraform.io/docs/internals/internal-plugins.html
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
aws_ecs_cluster.default: Refreshing state... (ID: arn:aws:e...ecstest1)
aws_ecs_task_definition.mongo: Refreshing state... (ID: mongodb)
aws_ecs_service.mongo: Refreshing state... (ID: arn:aws:e.../mongodb)
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, Terraform
doesn't need to do anything.
```
```
```
* provider/aws: Add Support for maintenance_window and back_window to rds_cluster_instance
Fixes: #9489
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSRDSClusterInstance_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/28 23:08:45 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSRDSClusterInstance_basic -timeout 120m
=== RUN TestAccAWSRDSClusterInstance_basic
--- PASS: TestAccAWSRDSClusterInstance_basic (1433.41s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 1433.438s
```
* Update rds_cluster_instance.html.markdown
* Update rds_cluster_instance.html.markdown
It turns out if you're trying to write a config to conditionally restore
an instance from a snapshot, you end up painted into a corner with the
combination of `snapshot_identifier` and `name`.
You need `name` to be specified for DBs you're creating, but when
`snapshot_identifier` is populated you need it to be blank or else the
AWS API throws an error.
The logical next step is to drop a ternary in:
```tf
resource "aws_db_instance" "foo" {
# ...
snapshot_identifier = "${var.snap}"
name = "${var.snap != "" ? "" : var.dbname}"
}
```
**BUT** the above config will _replace_ the DB on subsequent runs as the
config basically has `name = ""` which will trigger a ForceNew diff once
the `name` is populated from the snapshot restore.
**SO** we can get a workable solution by actively avoiding populating
DBName when we're using one of the engines the docs explicitly mention.
It does not look like there are any tests covering `snapshot_identifer`,
so I'll subject this to some manual tests and follow up with some
results.
* provider/aws: Deprecate roles in favour of role in iam_instance_profile
You can only specify a single role to an IAM Instance Profile. So having
a slice of roles in the provider makes no sense. Therefore, we are going
to deprecate this infavour of `role`
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSIAMInstanceProfile_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/28 21:24:20 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSIAMInstanceProfile_ -timeout 120m
=== RUN TestAccAWSIAMInstanceProfile_importBasic
--- PASS: TestAccAWSIAMInstanceProfile_importBasic (25.08s)
=== RUN TestAccAWSIAMInstanceProfile_basic
--- PASS: TestAccAWSIAMInstanceProfile_basic (22.40s)
=== RUN TestAccAWSIAMInstanceProfile_withRoleNotRoles
--- PASS: TestAccAWSIAMInstanceProfile_withRoleNotRoles (22.63s)
=== RUN TestAccAWSIAMInstanceProfile_missingRoleThrowsError
--- PASS: TestAccAWSIAMInstanceProfile_missingRoleThrowsError (4.02s)
=== RUN TestAccAWSIAMInstanceProfile_namePrefix
--- PASS: TestAccAWSIAMInstanceProfile_namePrefix (22.18s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 96.349s
```
* Update iam_instance_profile.html.markdown
Fix an issue when upgrading from Terraform < 0.9 to 0.9+, when we added
support for the regional endpoints in OpsWorks Stacks. OpsWorks Stacks
can only be managed via the endpoint with which they were created, not
where the stack resides.
* Adding import to resource_aws_iam_server_certificate.
* provider/aws: Update tests for import of aws_iam_server_certificate
Builds upon the work of @mrcopper in #12940
Resource:
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSIAMServerCertificate_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/25 00:08:48 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSIAMServerCertificate_ -timeout 120m
=== RUN TestAccAWSIAMServerCertificate_importBasic
--- PASS: TestAccAWSIAMServerCertificate_importBasic (22.81s)
=== RUN TestAccAWSIAMServerCertificate_basic
--- PASS: TestAccAWSIAMServerCertificate_basic (19.68s)
=== RUN TestAccAWSIAMServerCertificate_name_prefix
--- PASS: TestAccAWSIAMServerCertificate_name_prefix (19.88s)
=== RUN TestAccAWSIAMServerCertificate_disappears
--- PASS: TestAccAWSIAMServerCertificate_disappears (13.94s)
=== RUN TestAccAWSIAMServerCertificate_file
--- PASS: TestAccAWSIAMServerCertificate_file (32.67s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 109.062s
```
Data Source:
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSDataSourceIAMServerCertificate_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/25 13:07:10 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSDataSourceIAMServerCertificate_ -timeout 120m
=== RUN TestAccAWSDataSourceIAMServerCertificate_basic
--- PASS: TestAccAWSDataSourceIAMServerCertificate_basic (43.86s)
=== RUN TestAccAWSDataSourceIAMServerCertificate_matchNamePrefix
--- PASS: TestAccAWSDataSourceIAMServerCertificate_matchNamePrefix (2.68s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 46.569s
```
Updates `aws_caller_identity` data source to actually include the correct attributes from the `GetCallerIdentity` API function.
```
$ make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSCallerIdentity_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/27 09:26:13 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSCallerIdentity_basic -timeout 120m
=== RUN TestAccAWSCallerIdentity_basic
--- PASS: TestAccAWSCallerIdentity_basic (12.74s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 12.767s
```
In function `resourceAwsSesReceiptRuleRead` the position of the receipt
rules in the rule set was taken directly from the index of the rule's
position in the slice returned by the AWS API call. As the slice is
zero-based and the ruleset is one-based, this results in an incorrect
representation.
This manifests as `aws_ses_receipt_rule` resources always showing a
diff during plan or apply.
same
Fixes: #13012
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSNetworkAclRule_allProtocol'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/24 18:42:05 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSNetworkAclRule_allProtocol -timeout 120m
=== RUN TestAccAWSNetworkAclRule_allProtocol
--- PASS: TestAccAWSNetworkAclRule_allProtocol (53.95s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 53.974s
```
clint shryock