* provider/aws: Fixed DynamoDB GSI update when using multiple indexes
* provider/aws: Fixed DynamoDB GSI set hash function
* Added DynamoDB table state migration
Redshift was changed to not skip snapshots by default, so our configs
were out of date and causing errors in destroy (thus leaking redshifts)
This changes the configs to skip snapshots, which should at least fix:
- TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates
- TestAccAWSRedshiftCluster_loggingEnabled
Also doesnt use decode/encode anymore since those methods are more
intended for streams. So this goes to the more standed marshal/unmarshal
of data.
This also adds better error support and will try to give the user
better errors from the api if it can. Or any issues with the bitbucket
service.
* provider/aws: Fix KMS Key reading with Exists method
Fixes#13322 by checking if the key Exists and offering to recreate if
not found, or pending delete
* remove redundant code
Add getters for the AWSClient s3.S3 and dynamodb.DynamoDB clients so the
s3 remote-state backend can use all the same initialization code as the
aws provider.
* provider/aws: Add support for treat_missing_data to cloudwatch_metric_alarm
Fixes: #13263
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSCloudWatchMetricAlarm'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/05 08:51:06 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSCloudWatchMetricAlarm -timeout 120m
=== RUN TestAccAWSCloudWatchMetricAlarm_importBasic
--- PASS: TestAccAWSCloudWatchMetricAlarm_importBasic (23.93s)
=== RUN TestAccAWSCloudWatchMetricAlarm_basic
--- PASS: TestAccAWSCloudWatchMetricAlarm_basic (27.81s)
=== RUN TestAccAWSCloudWatchMetricAlarm_treatMissingData
--- PASS: TestAccAWSCloudWatchMetricAlarm_treatMissingData (43.39s)
=== RUN TestAccAWSCloudWatchMetricAlarm_extendedStatistic
--- PASS: TestAccAWSCloudWatchMetricAlarm_extendedStatistic (26.80s)
=== RUN TestAccAWSCloudWatchMetricAlarm_missingStatistic
--- PASS: TestAccAWSCloudWatchMetricAlarm_missingStatistic (5.95s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 127.899s
```
* provider/aws: Set cloudwatch_metric_alarm treamt_missing_data to missing
This follows what the AWS API does. We had to add a state migration for
this to make sure that the user doesn't see any unexpected activity on
their Terraform plans
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAWSCloudWatchMetricAlarmMigrateState'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/05 14:51:32 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAWSCloudWatchMetricAlarmMigrateState -timeout 120m
=== RUN TestAWSCloudWatchMetricAlarmMigrateState
2017/04/05 14:52:13 [INFO] Found AWS CloudWatch Metric Alarm State v0; migrating to v1
2017/04/05 14:52:13 [DEBUG] Attributes before migration: map[string]string{}
2017/04/05 14:52:13 [DEBUG] Attributes after migration: map[string]string{"treat_missing_data":"missing"}
--- PASS: TestAWSCloudWatchMetricAlarmMigrateState (0.00s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 0.018s
```
The AWS waiter package has changed location in the 1.8.0 version of the
SDK. DMS will need to mitigate a breaking change because of this
Between @radeksimko and myself, we think that we should migrate the DMS
resources to using the Terraform state refresh func pattern that is used
across the entire of the AWS provider. DMS is the *only* resource that
currently uses the AWS waiters, so the LOE to migrate is pretty low
For our ECS service definition we have this snippet at the
`load_balancer`.
The `target_group_arn` is being pupulated by an external service that
returns the arn based on a simple string from our microservices list.
If the arn changed, this would not cause a recreation of the service and
leaving a dangling pointer to an arn that does not exist anymore.
```
load_balancer {
target_group_arn = "${lookup(var.target_group_mapping, element(values(var.microservices), count.index))}"
container_name = "${element(values(var.microservices), count.index)}"
container_port = "${var.container_port}"
}
```
The fix is adding another field to the set that's creating the ELB/ALB
definition. From looking into the git history seems this code was
created prior to ALB thus not having this field available at the time.
Service is being recreated as expected, no other services are affected
(expected behavior)
The Default values set by AWS were breaking the AWS ALB Listener Rule
tests. The stickiness parameter needed to be set to be Computed: true to
remove this discrepancy
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSALBListenerRule_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/04/03 01:23:47 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSALBListenerRule_basic -timeout 120m
=== RUN TestAccAWSALBListenerRule_basic
--- PASS: TestAccAWSALBListenerRule_basic (235.36s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 235.397s
```
It appears that #13232 doesn't work properly if you actually try to set any `parameter`s. Specifically, I was getting the following error:
```
* aws_db_parameter_group.test: 1 error(s) occurred:
* aws_db_parameter_group.test: Error modifying DB Parameter Group: InvalidParameterValue: The parameter DBParameterGroupName must be provided and must not be blank.
status code: 400, request id: 5783e396-17ff-11e7-87d5-e3fd4c7025ce
```
* provider/aws: New resource: aws_ses_domain_identity
Provide a resource to manage domain identities in SES. Exports the
verification_code attribute which can be used to add the TXT record to
the domain to complete the domain verification.
* provider/aws: Acceptance tests for aws_ses_domain_identity
* Resource aws_ses_domain_identity: Documentation update
Provide documentation for the new resource type.
GOPATH is actually a list of path and doesn't necessarily have to be
set. If unset it will default to $GOPATH/go in go 1.9+.
Assume that go install will install to the first path in the list.
Fixes: #9959
When we specify that we want an opsworks_instance state of running, we
should wait for that the be the case. This will then allow us to use the
Computed values (e.g. private_ip) etc and allow us to use provisioners
as part of the terraform config
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSOpsworksInstance'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 20:55:21 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSOpsworksInstance -timeout 120m
=== RUN TestAccAWSOpsworksInstance_importBasic
--- PASS: TestAccAWSOpsworksInstance_importBasic (72.28s)
=== RUN TestAccAWSOpsworksInstance
--- PASS: TestAccAWSOpsworksInstance (110.17s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 182.479s
```
Fixes: #8847
We actually didn't get the list of tags from the API, therefore, any
manual changes were not actually showing up in subsequent plans
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSALBTargetGroup_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 15:45:53 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSALBTargetGroup_basic -timeout 120m
=== RUN TestAccAWSALBTargetGroup_basic
--- PASS: TestAccAWSALBTargetGroup_basic (62.76s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 62.787s
```
Fixes: #13167
When changes to the target group were made via CLI or AWS Console, they
were not being picked up by terraform. This is because we were not
catching an error setting the `stickiness` parameters:
```
Error refreshing state: 1 error(s) occurred:
* aws_alb_target_group.test: aws_alb_target_group.test: stickiness.0.enabled: '' expected type 'bool', got unconvertible type 'string'
```
This meant that changes were not picked up in the following plan. The changes mean the following now:
```
~ aws_alb_target_group.test
stickiness.0.cookie_duration: "10440" => "10000"
stickiness.0.enabled: "false" => "true"
Plan: 0 to add, 1 to change, 0 to destroy.
```
* provider/triton: Move to joyent/triton-go
This commit moves the Triton provider to the new joyent/triton-go
library from gosdc. This has a number of advantages - not least that
requests can be signed using an SSH agent without having to keep
unencrypted key material in memory.
Schema has been maintained for all resources, and several tests have
been added and acceptance tests repaired - in some cases by fixing bugs
in the underlying resources.
After applying this patch, all acceptance tests pass:
```
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 13:48:33 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/triton -v -timeout 120m
=== RUN TestProvider
--- PASS: TestProvider (0.00s)
=== RUN TestProvider_impl
--- PASS: TestProvider_impl (0.00s)
=== RUN TestAccTritonFabric_basic
--- PASS: TestAccTritonFabric_basic (15.11s)
=== RUN TestAccTritonFirewallRule_basic
--- PASS: TestAccTritonFirewallRule_basic (1.48s)
=== RUN TestAccTritonFirewallRule_update
--- PASS: TestAccTritonFirewallRule_update (1.55s)
=== RUN TestAccTritonFirewallRule_enable
--- PASS: TestAccTritonFirewallRule_enable (1.52s)
=== RUN TestAccTritonKey_basic
--- PASS: TestAccTritonKey_basic (11.76s)
=== RUN TestAccTritonKey_noKeyName
--- PASS: TestAccTritonKey_noKeyName (11.20s)
=== RUN TestAccTritonMachine_basic
--- PASS: TestAccTritonMachine_basic (82.19s)
=== RUN TestAccTritonMachine_dns
--- PASS: TestAccTritonMachine_dns (173.36s)
=== RUN TestAccTritonMachine_nic
--- PASS: TestAccTritonMachine_nic (167.82s)
=== RUN TestAccTritonMachine_addNIC
--- PASS: TestAccTritonMachine_addNIC (192.11s)
=== RUN TestAccTritonMachine_firewall
--- PASS: TestAccTritonMachine_firewall (188.53s)
=== RUN TestAccTritonMachine_metadata
--- PASS: TestAccTritonMachine_metadata (614.57s)
=== RUN TestAccTritonVLAN_basic
--- PASS: TestAccTritonVLAN_basic (0.93s)
=== RUN TestAccTritonVLAN_update
--- PASS: TestAccTritonVLAN_update (1.50s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/triton 1463.621s
```
* provider/triton: Update docs for provider config
* deps: Vendor github.com/joyent/triton-go/...
* deps: Remove github.com/joyent/gosdc
Fixes: #13216
Prior to Terraform 0.9.2, we always set placement_strategies to
lowercase. Therefore, people using it in Terraform 0.9.2 are getting
continual diffs:
```
-/+ aws_ecs_service.mongo
cluster: "arn:aws:ecs:us-west-2:187416307283:cluster/terraformecstest1" => "arn:aws:ecs:us-west-2:187416307283:cluster/terraformecstest1"
deployment_maximum_percent: "200" => "200"
deployment_minimum_healthy_percent: "100" => "100"
desired_count: "1" => "1"
name: "mongodb" => "mongodb"
placement_strategy.#: "1" => "1"
placement_strategy.1676812570.field: "instanceid" => "" (forces new resource)
placement_strategy.1676812570.type: "spread" => "" (forces new resource)
placement_strategy.3946258308.field: "" => "instanceId" (forces new resource)
placement_strategy.3946258308.type: "" => "spread" (forces new resource)
task_definition: "arn:aws:ecs:us-west-2:187416307283:task-definition/mongodb:1991" => "arn:aws:ecs:us-west-2:187416307283:task-definition/mongodb:1991"
Plan: 1 to add, 0 to change, 1 to destroy.
```
This adds a DiffSuppression func to make sure this doesn't trigger a
ForceNew resource:
```
% terraform plan ✹ ✭
[WARN] /Users/stacko/Code/go/bin/terraform-provider-aws overrides an internal plugin for aws-provider.
If you did not expect to see this message you will need to remove the old plugin.
See https://www.terraform.io/docs/internals/internal-plugins.html
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
aws_ecs_cluster.default: Refreshing state... (ID: arn:aws:e...ecstest1)
aws_ecs_task_definition.mongo: Refreshing state... (ID: mongodb)
aws_ecs_service.mongo: Refreshing state... (ID: arn:aws:e.../mongodb)
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, Terraform
doesn't need to do anything.
```
```
```
The plug-in crashes if "localNetworkGateways" is not found in the id parameter. The fix is to verify the parameter before proceeding.
Also the "import" example in the documentation is wrong, "localNetworkGateways" should be case sensitive. The current example actually causes the plugin to crash due to the bug.
* provider/aws: Add Support for maintenance_window and back_window to rds_cluster_instance
Fixes: #9489
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSRDSClusterInstance_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/28 23:08:45 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSRDSClusterInstance_basic -timeout 120m
=== RUN TestAccAWSRDSClusterInstance_basic
--- PASS: TestAccAWSRDSClusterInstance_basic (1433.41s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 1433.438s
```
* Update rds_cluster_instance.html.markdown
* Update rds_cluster_instance.html.markdown
It turns out if you're trying to write a config to conditionally restore
an instance from a snapshot, you end up painted into a corner with the
combination of `snapshot_identifier` and `name`.
You need `name` to be specified for DBs you're creating, but when
`snapshot_identifier` is populated you need it to be blank or else the
AWS API throws an error.
The logical next step is to drop a ternary in:
```tf
resource "aws_db_instance" "foo" {
# ...
snapshot_identifier = "${var.snap}"
name = "${var.snap != "" ? "" : var.dbname}"
}
```
**BUT** the above config will _replace_ the DB on subsequent runs as the
config basically has `name = ""` which will trigger a ForceNew diff once
the `name` is populated from the snapshot restore.
**SO** we can get a workable solution by actively avoiding populating
DBName when we're using one of the engines the docs explicitly mention.
It does not look like there are any tests covering `snapshot_identifer`,
so I'll subject this to some manual tests and follow up with some
results.
* provider/aws: Deprecate roles in favour of role in iam_instance_profile
You can only specify a single role to an IAM Instance Profile. So having
a slice of roles in the provider makes no sense. Therefore, we are going
to deprecate this infavour of `role`
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSIAMInstanceProfile_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/28 21:24:20 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSIAMInstanceProfile_ -timeout 120m
=== RUN TestAccAWSIAMInstanceProfile_importBasic
--- PASS: TestAccAWSIAMInstanceProfile_importBasic (25.08s)
=== RUN TestAccAWSIAMInstanceProfile_basic
--- PASS: TestAccAWSIAMInstanceProfile_basic (22.40s)
=== RUN TestAccAWSIAMInstanceProfile_withRoleNotRoles
--- PASS: TestAccAWSIAMInstanceProfile_withRoleNotRoles (22.63s)
=== RUN TestAccAWSIAMInstanceProfile_missingRoleThrowsError
--- PASS: TestAccAWSIAMInstanceProfile_missingRoleThrowsError (4.02s)
=== RUN TestAccAWSIAMInstanceProfile_namePrefix
--- PASS: TestAccAWSIAMInstanceProfile_namePrefix (22.18s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 96.349s
```
* Update iam_instance_profile.html.markdown
Fix an issue when upgrading from Terraform < 0.9 to 0.9+, when we added
support for the regional endpoints in OpsWorks Stacks. OpsWorks Stacks
can only be managed via the endpoint with which they were created, not
where the stack resides.
* provider/openstack: Add all_fixed_ips computed attribute to port resource
This commit adds the `all_fixed_ips` attribute to the
openstack_networking_port_v2 resource. This contains all of the port's
Fixed IPs returned by the Networking v2 API.
* provider/openstack: Revert Port fixed_ip back to a List
This commit reverts the changes made in a8c4e81a6e3f2. This
re-enables the ability to reference IP addresses using the
numerical-element notation.
This commit also makes fixed_ip a non-computed field, meaning
Terraform will no longer set fixed_ip with what was returned
by the API. This resolves the original issue about ordering.
The last use-case is for fixed_ips that received an IP address
via DHCP. Because fixed_ip is no longer computed, the DHCP IP
will not be set. The workaround for this use-case is to use the
new all_fixed_ips attribute.
In effect, users should use fixed_ip only as a way of inputting
data into Terraform and use all_fixed_ips as the output returned
by the API. If use-cases exist where fixed_ip can be used as an
output, that's a bonus feature.
This commit adds the all_metadata attribute which contains all
instance metadata returned from the OpenStack Compute API. This
is useful for gaining access to metadata which was applied
out of band of Terraform.
This commit also stops setting the original metadata attribute,
effectively making metadata an input and all_metadata the output
to reference all data from.
* Adding import to resource_aws_iam_server_certificate.
* provider/aws: Update tests for import of aws_iam_server_certificate
Builds upon the work of @mrcopper in #12940
Resource:
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSIAMServerCertificate_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/25 00:08:48 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSIAMServerCertificate_ -timeout 120m
=== RUN TestAccAWSIAMServerCertificate_importBasic
--- PASS: TestAccAWSIAMServerCertificate_importBasic (22.81s)
=== RUN TestAccAWSIAMServerCertificate_basic
--- PASS: TestAccAWSIAMServerCertificate_basic (19.68s)
=== RUN TestAccAWSIAMServerCertificate_name_prefix
--- PASS: TestAccAWSIAMServerCertificate_name_prefix (19.88s)
=== RUN TestAccAWSIAMServerCertificate_disappears
--- PASS: TestAccAWSIAMServerCertificate_disappears (13.94s)
=== RUN TestAccAWSIAMServerCertificate_file
--- PASS: TestAccAWSIAMServerCertificate_file (32.67s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 109.062s
```
Data Source:
```
% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSDataSourceIAMServerCertificate_'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/25 13:07:10 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSDataSourceIAMServerCertificate_ -timeout 120m
=== RUN TestAccAWSDataSourceIAMServerCertificate_basic
--- PASS: TestAccAWSDataSourceIAMServerCertificate_basic (43.86s)
=== RUN TestAccAWSDataSourceIAMServerCertificate_matchNamePrefix
--- PASS: TestAccAWSDataSourceIAMServerCertificate_matchNamePrefix (2.68s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 46.569s
```
Updates `aws_caller_identity` data source to actually include the correct attributes from the `GetCallerIdentity` API function.
```
$ make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSCallerIdentity_basic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/27 09:26:13 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSCallerIdentity_basic -timeout 120m
=== RUN TestAccAWSCallerIdentity_basic
--- PASS: TestAccAWSCallerIdentity_basic (12.74s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/aws 12.767s
```
In function `resourceAwsSesReceiptRuleRead` the position of the receipt
rules in the rule set was taken directly from the index of the rule's
position in the slice returned by the AWS API call. As the slice is
zero-based and the ruleset is one-based, this results in an incorrect
representation.
This manifests as `aws_ses_receipt_rule` resources always showing a
diff during plan or apply.
Allow to resize a droplet permanently (i.e. apply disk resize)
if previously it was resized temporarily (CPU and RAM only).
Fixes: #13007
```
$ make testacc TEST=./builtin/providers/digitalocean TESTARGS='-run=TestAccDigitalOceanDroplet_ResizeOnlyDisk'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/25 03:54:23 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/digitalocean -v -run=TestAccDigitalOceanDroplet_ResizeOnlyDisk -timeout 120m
=== RUN TestAccDigitalOceanDroplet_ResizeOnlyDisk
--- PASS: TestAccDigitalOceanDroplet_ResizeOnlyDisk (198.62s)
PASS
ok github.com/hashicorp/terraform/builtin/providers/digitalocean 198.638s
```
according to the official scaleway support, requests within the same session can
not be parallelized.
While I do not know for sure that this is a write-only limitation, I've
implemented it as a write-only limitation for now.
Previously requests like this would produce a 500 internal server error:
```
resource "scaleway_ip" "test_ip" {
count = 2
}
```
now this limitation should be lifted, for all scaleway resources
The OS_DEBUG feature has worked out great, but frequent logging of
the service catalog during client initialization can make logging
very chatty. This commit omits the service catalog in the logs.
This commit deprecates the floating_ip attributes from the
openstack_compute_instance_v2 resource. It is recommended to use
either the openstack_compute_floatingip_associate resource or
configure an openstack_networking_port_v2 resource with a floating
IP.
This commit deprecates the volume attribute in the
openstack_compute_instance_v2 resource. It's recommended to either
use the block_device attribute or the openstack_compute_volume_attach_v2
resource.
Ryan Roberts