Commit Graph

661 Commits

Author SHA1 Message Date
Radek Simko ad34f1ec74 Add (failing) test for map w/ non-string values 2016-11-17 11:46:32 -05:00
James Bardin e28e11d44c Fix resource.UniqueId to be properly ordered
UniqueId attempted to provide an ordered unique id by using a nanosecond
timestamp, but doesn't take into account that time is not monotonic
increasing. This provides an implementation that will always be
increasing.
2016-11-15 21:58:23 -05:00
Mitchell Hashimoto 39542898b0
helper/schema: mark diff as forcenew if element is computed
Fixes #10125

If the elements are computed and the field is ForceNew, then we should
mark the computed count as potentially forcing a new operation.

Example, assuming `groups` forces new...

**Step 1:**

    groups = ["1", "2", "3"]

At this point, the resource isn't create, so this should result in a
diff like:

    CREATE resource:
      groups: "" => ["1", "2", "3"]

**Step 2:**

    groups = ["${computedvar}"]

The OLD behavior was:

    UPDATE resource
      groups.#: "3" => "computed"

This would cause a diff mismatch because if `${computedvar}` was
different then it should force new. The NEW behavior is:

    DESTROY/CREATE resource:
      groups.#: "3" => "computed" (forces new)
2016-11-15 11:02:14 -08:00
Mitchell Hashimoto 6557a3de18
helper/shadow: Close for auto-closing all values
Fixes #10122

The simple fix was that we forgot to close `ReadDataApply` for the
provider. But I've always felt that this section of the code was brittle
and I wanted to put in a more robust solution. The `shadow.Close` method
uses reflection to automatically close all values.
2016-11-15 08:54:26 -08:00
Mitchell Hashimoto 1a6056b287
command: split out and tag code so compilation works on Solaris
The readline library doesn't support Solaris. For now, we'll just not
support console there.
2016-11-14 00:32:01 -08:00
Mitchell Hashimoto a867457d75
command/console 2016-11-13 23:17:04 -08:00
Mitchell Hashimoto 73a1564dac
helper/wrappedreadline: helper for dealing with wrapped standard streams 2016-11-13 23:17:03 -08:00
Mitchell Hashimoto 785cc7b78a
terraform: default new graphs on, old graphs behind -Xlegacy-graph
This turns the new graphs on by default and puts the old graphs behind a
flag `-Xlegacy-graph`. This effectively inverts the current 0.7.x
behavior with the new graphs.

We've incubated most of these for a few weeks now. We've found issues
and we've fixed them and we've been using these graphs internally for
awhile without any major issue. Its time to default them on and get them
part of a beta.
2016-11-10 21:53:20 -08:00
Mitchell Hashimoto 15f50b86bf
helper/schema: passing tests for computed complex sets 2016-11-10 08:52:37 -08:00
Mitchell Hashimoto 59bd1a22f4
helper/diff: handle unknownvariablevalue 2016-11-09 14:28:16 -08:00
Mitchell Hashimoto 29485f6167
terraform: ResourceConfig.IsComputed cases 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 5792b2cba2
helper/schema: convert _Diff to subtests 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 2b7177cfe7 Merge pull request #9607 from hashicorp/f-provider-stop-redo
terraform: ResourceProvider.Stop (redo)
2016-11-08 15:58:48 -08:00
Mitchell Hashimoto e45debe0e5
helper/schema: only mark "ForceNew" on resources that cause the ForceNew
Fixes #2748

This changes the diff to only mark "forces new resource" on the fields
that actually caused the new resource, not every field that changed.
This makes diffs much more accurate.

I'd like to request a review but I'm going to defer merging until
Terraform 0.8. Changes like this are very possible to cause "diffs
didn't match" errors and I want some real world testing in a beta before
we hit prod with this.
2016-11-08 15:49:28 -08:00
Mitchell Hashimoto b7bee66df5
helper/schema: sort errors in helper/schema test for deterministic tests 2016-11-04 16:51:26 -07:00
Mitchell Hashimoto f0abe6d1a0 Merge pull request #9812 from hashicorp/b-bool-computed-crash
helper/schema: computed bool fields should not crash
2016-11-04 08:47:49 -07:00
Mitchell Hashimoto 65b17ccd06
helper/schema: allow ConflictsWith and Computed Optional fields 2016-11-02 22:24:34 -07:00
Mitchell Hashimoto 7834cf7190
helper/schema: computed bool fields should not crash
Fixes #7715

If a bool field was computed and the raw value was not convertable to a
boolean, helper/schema would crash. The correct behavior is to try not
to read the raw value when the value is computed and to simply mark that
it is computed. This does that (and matches the behavior of the other
primitives).
2016-11-02 13:25:23 -07:00
Mitchell Hashimoto 2d84582881 Merge pull request #9699 from hashicorp/b-removed-forcenew
helper/schema: removed optional items force new
2016-10-31 13:24:36 -07:00
Mitchell Hashimoto f7a234bc71
helper/schema: validate Read, Delete are set 2016-10-30 15:04:32 -07:00
Mitchell Hashimoto 5489d8c549
helper/schema: removed optional items force new
Fixes #5138

If an item is optional and is removed completely from the configuration,
it should still trigger a destroy/create if the field itself was marked
as "ForceNew".

See the example in #5138.
2016-10-28 18:45:12 -04:00
Mitchell Hashimoto 3f36787207 Merge pull request #9618 from hashicorp/b-computed-prim
helper/schema,terraform: handle computed primitives in diffs
2016-10-28 10:44:13 -04:00
Mitchell Hashimoto af82be19ea
helper/experiment: a helper for setting, making experiments
This creates a standard package and interface for defining, querying,
setting experiments (`-X` flags).

I expect we'll want to continue to introduce various features behind
experimental flags. I want to make doing this as easy as possible and I
want to make _removing_ experiments as easy as possible as well.

The goal with this packge has been to rely on the compiler enforcing our
experiment references as much as possible. This means that every
experiment is a global variable that must be referenced directly, so
when it is removed you'll get compiler errors where the experiment is
referenced.

This also unifies and makes it easy to grab CLI flags to enable/disable
experiments as well as env vars! This way defining an experiment is just
a couple lines of code (documented on the package).
2016-10-26 15:47:58 -04:00
Mitchell Hashimoto 95d37ea79c
helper/schema,terraform: handle computed primtives in diffs
Fixes #3309

There are two primary changes, one to how helper/schema creates diffs
and one to how Terraform compares diffs. Both require careful
understanding.

== 1. helper/schema Changes

helper/schema, given any primitive field (string, int, bool, etc.)
_used to_ create a basic diff when given a computed new value (i.e. from
an unkown interpolation). This would put in the plan that the old value
is whatever the old value was, and the new value was the actual
interpolation. For example, from #3309, the diff showed the following:

```
~ module.test.aws_eip.test-instance.0
    instance: "<INSTANCE ID>" => "${element(aws_instance.test-instance.*.id, count.index)}"
```

Then, when running `apply`, the diff would be realized and you would get
a diff mismatch error because it would realize the final value is the
same and remove it from the diff.

**The change:** `helper/schema` now marks unknown primitive values with
`NewComputed` set to true. Semantically this is correct for the diff to
have this information.

== 2. Terraform Diff.Same Changes

Next, the way Terraform compares diffs needed to be updated

Specifically, the case where the diff from the plan had a NewComputed
primitive and the diff from the apply _no longer has that value_. This
is possible if the computed value ended up being the same as the old
value. This is allowed to pass through.

Together, these fix #3309.
2016-10-25 22:36:59 -04:00
Mitchell Hashimoto ad5a82de75
helper/resource: remove StopCh, use contexts instead 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto 5ee8042dff
helper/schema: expose stop information as a Context 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto be34dfe7c1
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto d1bb2f3487
helper/schema: support Stop() 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto 60140b28f4
Revert "Merge pull request #9536 from hashicorp/f-provider-stop"
This reverts commit c3a4cff133, reversing
changes made to 791a02e6e4.

This change requires plugin recompilation and we should hold off until a
minor release for that.
2016-10-25 12:00:36 -07:00
Mitchell Hashimoto 61bbaf6f85
helper/resource: remove StopCh, use contexts instead 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 86eb30b8a2
helper/schema: expose stop information as a Context 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 89647745b0
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 11:31:56 -07:00
Mitchell Hashimoto 8c11f137f5
helper/schema: support Stop() 2016-10-25 11:31:55 -07:00
James Nugent 43dd13cd36 helper/schema: Introduce Noop and RemoveFromState
This commit implements reusable functions for when resources have no
need to implement a particular operation:

- Noop - does nothing and returns no error.
- RemoveFromState - sets the resource ID to empty string (removing it
  from state) and returns no error.
2016-10-25 11:47:03 -05:00
Mitchell Hashimoto fa9758e162
helper/schema: test with DiffSuppress and Default 2016-10-24 22:23:13 -07:00
Mitchell Hashimoto de827887bf
helper/shadow: keyedValue.WaitForChange must unlock 2016-10-21 17:57:00 -07:00
Mitchell Hashimoto 0fe51b334c Merge pull request #9334 from hashicorp/f-shadow-graph
terraform: Shadow Graph
2016-10-19 13:36:10 -07:00
Sean Chittenden b1c3649eac
Append to debug log files, don't clobber. 2016-10-18 13:08:23 -07:00
Mitchell Hashimoto 10bcdd04d4
helper/shadow: KeyedValue.Init 2016-10-12 18:45:40 +08:00
James Bardin 404a76e2b9 Merge pull request #9144 from hashicorp/b-source
config/module: use the raw source as part of the key
2016-10-11 12:29:42 -04:00
Mitchell Hashimoto 50e0647c53
helper/shadow: ComparedValue 2016-10-11 22:17:30 +08:00
Mitchell Hashimoto c92ee5a8bd
helper/shadow: KeyedValue.WaitForChange returns immediately if closed 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 0408c2dfb2
helper/shadow: KeyedValue.WaitForChange 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 47f4343bf5
helper/shadow: KeyedValue add test case to avoid panic 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 136ac4728d
helper/shadow: KeyedValue.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d2fb630df8
helper/shadow: Value.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d6168edc50
helper/shadow: KeyedValue.ValueOk 2016-10-11 22:17:28 +08:00
Mitchell Hashimoto bd69e41c14
helper/shadow: KeyedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 8426cea6b0
helper/shadow: OrderedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 1df3bbdc37
terraform: working on the resource provider shadow, not working yet 2016-10-11 22:17:27 +08:00
Paul Stack 4ac7d0a6bb Merge pull request #8103 from BedeGaming/validation-helper
helper: create validation package to provide common validation functions
2016-10-07 13:42:29 +01:00
stack72 5a537cdbf9
helper/schema: Adding of MinItems as a validation to Lists and Maps
This is required for the times when the configuration cannot have an
empty configuration. An example would be in AzureRM, when you create a
LoadBalancer with a configuration, you can delete *all* but 1 of these
configurations
2016-10-04 18:57:58 +01:00
Paul Hinze 534a5a3b75 Merge pull request #8891 from gotascii/fix-diff-set-update
Unchanged NestedSets are not returned by DiffFieldReader.
2016-10-03 11:06:40 -05:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Justin Marney 895d10a627 Use DeepEqual instead of multiple if statements to reduce noise. 2016-09-17 17:41:44 +00:00
Justin Marney 53e10dfa89 Name test appropriately. 2016-09-17 16:56:12 +00:00
Justin Marney 9217f5a063 Unchanged NestedSets are not returned by DiffFieldReader. 2016-09-16 23:57:33 +00:00
Radek Simko 86acdccaf2 helper/resource: Add timeout to TimeoutError msg (#8773) 2016-09-15 10:53:25 +01:00
James Nugent e0226c9039 core: Check for attrV being nil before dereference
This can be an issue with unset computed fields.

Fixes #8815.
2016-09-14 09:51:15 +01:00
James Nugent 6c23181686 Merge pull request #8383 from kjmkznr/import-aws-s3-b-notification
provider/aws: Support import `aws_s3_bucket_notification`
2016-09-03 15:50:25 -07:00
Paul Stack bf755bb5c9 Merge pull request #8585 from hashicorp/f-diff-suppression
helper/schema: Add diff suppression callback
2016-09-01 14:46:03 +01:00
James Nugent 85ec09111b helper/schema: Add diff suppression callback
This commit adds a new callback, DiffSuppressFunc, to  the schema.Schema
structure. If set for a given schema, a callback to the user-supplied
function will be made for each attribute for which the default
type-based diff mechanism produces an attribute diff. Returning `true`
from the callback will suppress the diff (i.e. pretend there was no
diff), and returning false will retain it as part of the plan.

There are a number of motivating examples for this - one of which is
included as an example:

1. On SSH public keys, trailing whitespace does not matter in many
   cases - and in some cases it is added by provider APIs. For
   digitalocean_ssh_key resources we previously had a StateFunc that
   trimmed the whitespace - we now have a DiffSuppressFunc which
   verifies whether the trimmed strings are equivalent.

2. IAM policy equivalence for AWS. A good proportion of AWS issues
   relate to IAM policies which have been "normalized" (used loosely)
   by the IAM API endpoints. This can make the JSON strings differ
   from those generated by iam_policy_document resources or template
   files, even though the semantics are the same (for example,
   reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3
   bucket policy. DiffSupressFunc can be used to test for semantic
   equivalence rather than pure text equivalence, but without having to
   deal with the complexity associated with a full "provider-land" diff
   implementation without helper/schema.
2016-08-31 19:13:53 -05:00
James Bardin e0014198e1 Merge pull request #8560 from hashicorp/jbardin/race2
Fix races in WaitForState
2016-08-31 14:02:43 -04:00
Mitchell Hashimoto aaaed823af Merge pull request #8508 from TimeIncOSS/f-schema-all-validation-errors
schema: Return all validation errors together from InternalValidate
2016-08-31 10:44:13 -07:00
James Bardin 481f12cf2c Remove `first` variable an extra if block
clean up the code slighly by moving the Sleep in WaitForState to the end
of the loop.
2016-08-30 17:56:26 -04:00
James Bardin 82be35a797 Fix races in WaitForState
The WaitForState method can't read the result values in a timeout
because they are still owned by the running goroutine. Keep all values
scoped inside the goroutine, and save them into an atomic.Value to be
returned.

Fixes race introduced in #8510
2016-08-30 16:22:21 -04:00
Radek Simko 506c118383
helper/resource: Show last state in timeout err message 2016-08-27 21:25:14 +01:00
Radek Simko aef2513b44
helper/resource: Fix WaitForState tests 2016-08-27 15:11:09 +01:00
Radek Simko b90ab0b705
schema: Return all validation errors together from InternalValidate 2016-08-27 10:34:16 +01:00
James Nugent df06d5623d Merge pull request #8249 from meteor/glasser/name-prefix-timestamp
core: name_prefix names now start with a timestamp
2016-08-23 14:37:56 +02:00
Kazunori Kojima ed05161fd0
provider/aws: Support import `aws_s3_bucket_notification` 2016-08-23 08:19:06 +09:00
James Nugent 6ae7140251 Merge pull request #8370 from hashicorp/dtolnoy-wait
Do not sleep between first attempt or between successful attempts
2016-08-22 14:30:54 +02:00
James Nugent 0a5acaa0d8 Change error message return structure 2016-08-22 10:34:32 +02:00
David Tolnay 9d3f40a513 Fix bad message from TimeoutError
Before:

    Error creating IAM Role my-role: timeout while waiting for state
    to become 'success'. last error: %!s(<nil>)
2016-08-22 10:23:53 +02:00
David Tolnay e27a735da2 Do not sleep between first attempt or between successful attempts 2016-08-22 10:17:23 +02:00
David Glasser 4ad825fe08 core: name_prefix names now start with a timestamp
This means that two resources created by the same rule will get names
 which sort in the order they are created.

The rest of the ID is still random base32 characters; we no longer set
the bit values that denote UUIDv4.

The length of the ID returned by PrefixedUniqueId is not changed by this
commit; that way we don't break any resources where the underlying
resource has a name length limit.

Fixes #8143.
2016-08-17 11:06:28 -07:00
James Nugent b551981cc7 testing: Add ComposeAggregateTestFunc
This commit adds a function which composes a series of TestFuncs, but
will run all tests before returning an error, unlike ComposeTestFunc.
This is useful when verifying contents of state in acceptance tests and
it is desirable to see all the failing cases in one run for slow
resources.
2016-08-16 19:56:18 +01:00
Mitchell Hashimoto f26d1b40e0 Merge pull request #8215 from hashicorp/f-signalwrapper
helper/signalwrapper and azurerm_storage_account listens for signals
2016-08-16 09:57:54 -07:00
Mitchell Hashimoto 8dafcb36fd
providers/azurerm: cancellable storage account creation 2016-08-15 21:12:32 -07:00
Mitchell Hashimoto d8337920f9
helper/signalwrapper: more tests 2016-08-15 20:58:48 -07:00
Mitchell Hashimoto bdcea55117
helper/signalwrapper 2016-08-15 20:55:35 -07:00
James Nugent b681ab23d3 testing: Add TestCheckResourceAttrSet helper
This commit adds a TestCheckFunc which ensures that a value is set for a
given name/key combination. It is primarily useful for ensuring that
computed values are set where it is not possible to know the expected
value ahead of time.
2016-08-15 15:42:25 -05:00
Paul Hinze 3dccfa0cc9
terraform: diffs with only tainted set are non-empty
Fixes issue where a resource marked as tainted with no other attribute
diffs would never show up in the plan or apply as needing to be
replaced.

One unrelated test needed updating due to a quirk in the testDiffFn
logic - it adds a "type" field diff if the diff is non-Empty. NBD
2016-08-12 17:37:49 -05:00
Peter McAtominey 8abec085ec helper: create validation package to provide common validation functions 2016-08-10 13:27:49 +01:00
James Bardin 1af7ee87a2 Silence log output when not verbose
Set the default log package output to iotuil.Discard during tests if the
`-v` flag isn't set. If we are verbose, then apply the filter according
to the TF_LOG env variable.
2016-08-01 17:19:14 -04:00
James Bardin 074be9ae56 Another race in resource.Retry 2016-07-29 18:35:54 -04:00
Paul Hinze 261043fd1a Merge pull request #7523 from hashicorp/f-aws-beanstalk-env-poll-timing
provider/aws: Beanstalk environments, bump the minimum timeout between API calls
2016-07-21 16:43:21 -05:00
Paul Hinze 614806d59f
helper/resource: Fix import test harness, which was modifying state
Maps are reference types, it turns out :D
2016-07-15 13:15:47 -06:00
clint shryock de60481428 provider/aws: Rework Beanstalk optional polling
expose a poll_interval for users to configure polling for updates
2016-07-13 15:38:23 -06:00
David Tolnay 7096e4d3da Return nonnegative hash if int is 32 bits 2016-07-01 14:40:53 -07:00
Paul Hinze 3b732131d2 Merge pull request #7446 from hashicorp/b-jit-resource-validate
core: rerun resource validation before plan and apply
2016-07-01 15:00:40 -05:00
Paul Hinze ad27190852 Merge pull request #7448 from dtolnay/timeout
Timing out is not success
2016-07-01 14:28:11 -05:00
Paul Hinze afccf62e3e Merge pull request #7459 from hashicorp/r-resource-unit-test
helper/resource: Consolidate unit test override
2016-07-01 13:23:13 -05:00
Paul Hinze 4a1b36ac0d
core: rerun resource validation before plan and apply
In #7170 we found two scenarios where the type checking done during the
`context.Validate()` graph walk was circumvented, and the subsequent
assumption of type safety in the provider's `Diff()` implementation
caused panics.

Both scenarios have to do with interpolations that reference Computed
values. The sentinel we use to indicate that a value is Computed does
not carry any type information with it yet.

That means that an incorrect reference to a list or a map in a string
attribute can "sneak through" validation only to crop up...

 1. ...during Plan for Data Source References
 2. ...during Apply for Resource references

In order to address this, we:

 * add high-level tests for each of these two scenarios in `provider/test`
 * add context-level tests for the same two scenarios in `terraform`
   (these tests proved _really_ tricky to write!)
 * place an `EvalValidateResource` just before `EvalDiff` and `EvalApply` to
   catch these errors
 * add some plumbing to `Plan()` and `Apply()` to return validation
   errors, which were previously only generated during `Validate()`
 * wrap unit-tests around `EvalValidateResource`
 * add an `IgnoreWarnings` option to `EvalValidateResource` to prevent
   active warnings from halting execution on the second-pass validation

Eventually, we might be able to attach type information to Computed
values, which would allow for these errors to be caught earlier. For
now, this solution keeps us safe from panics and raises the proper
errors to the user.

Fixes #7170
2016-07-01 13:12:57 -05:00
Paul Hinze 5656d7388c
helper/resource: Consolidate unit test override
I noticed we had two mechanisms for unit test override. One that dropped
a sentinel into the env var, and another with a struct member on
TestCase. This consolidates the two, using the cleaner struct member
internal mechanism and the nicer `resource.UnitTest()` entry point.
2016-07-01 13:08:43 -05:00
David Tolnay 7e1cd34819 Timing out is not success 2016-06-30 18:07:56 -07:00
Clint a84aa5e914 Revert "helper/schema: Make nested Set(s) in List(s) work" (#7436) 2016-06-30 10:48:52 -05:00
Radek Simko 37d57f4a85 Improve naming convention (resultSet -> exists) 2016-06-28 17:40:45 +01:00
Radek Simko 917ad44cf0 helper/schema: Fix readSet implementation (DiffFieldReader) 2016-06-28 17:40:44 +01:00
Radek Simko c738c5a9a3 helper/schema: Implement reader-specific readList method 2016-06-28 17:40:44 +01:00