Commit Graph

667 Commits

Author SHA1 Message Date
Mitchell Hashimoto 136ac4728d
helper/shadow: KeyedValue.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d2fb630df8
helper/shadow: Value.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d6168edc50
helper/shadow: KeyedValue.ValueOk 2016-10-11 22:17:28 +08:00
Mitchell Hashimoto bd69e41c14
helper/shadow: KeyedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 8426cea6b0
helper/shadow: OrderedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 1df3bbdc37
terraform: working on the resource provider shadow, not working yet 2016-10-11 22:17:27 +08:00
Paul Stack 4ac7d0a6bb Merge pull request #8103 from BedeGaming/validation-helper
helper: create validation package to provide common validation functions
2016-10-07 13:42:29 +01:00
stack72 5a537cdbf9
helper/schema: Adding of MinItems as a validation to Lists and Maps
This is required for the times when the configuration cannot have an
empty configuration. An example would be in AzureRM, when you create a
LoadBalancer with a configuration, you can delete *all* but 1 of these
configurations
2016-10-04 18:57:58 +01:00
Paul Hinze 534a5a3b75 Merge pull request #8891 from gotascii/fix-diff-set-update
Unchanged NestedSets are not returned by DiffFieldReader.
2016-10-03 11:06:40 -05:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Justin Marney 895d10a627 Use DeepEqual instead of multiple if statements to reduce noise. 2016-09-17 17:41:44 +00:00
Justin Marney 53e10dfa89 Name test appropriately. 2016-09-17 16:56:12 +00:00
Justin Marney 9217f5a063 Unchanged NestedSets are not returned by DiffFieldReader. 2016-09-16 23:57:33 +00:00
Radek Simko 86acdccaf2 helper/resource: Add timeout to TimeoutError msg (#8773) 2016-09-15 10:53:25 +01:00
James Nugent e0226c9039 core: Check for attrV being nil before dereference
This can be an issue with unset computed fields.

Fixes #8815.
2016-09-14 09:51:15 +01:00
James Nugent 6c23181686 Merge pull request #8383 from kjmkznr/import-aws-s3-b-notification
provider/aws: Support import `aws_s3_bucket_notification`
2016-09-03 15:50:25 -07:00
Paul Stack bf755bb5c9 Merge pull request #8585 from hashicorp/f-diff-suppression
helper/schema: Add diff suppression callback
2016-09-01 14:46:03 +01:00
James Nugent 85ec09111b helper/schema: Add diff suppression callback
This commit adds a new callback, DiffSuppressFunc, to  the schema.Schema
structure. If set for a given schema, a callback to the user-supplied
function will be made for each attribute for which the default
type-based diff mechanism produces an attribute diff. Returning `true`
from the callback will suppress the diff (i.e. pretend there was no
diff), and returning false will retain it as part of the plan.

There are a number of motivating examples for this - one of which is
included as an example:

1. On SSH public keys, trailing whitespace does not matter in many
   cases - and in some cases it is added by provider APIs. For
   digitalocean_ssh_key resources we previously had a StateFunc that
   trimmed the whitespace - we now have a DiffSuppressFunc which
   verifies whether the trimmed strings are equivalent.

2. IAM policy equivalence for AWS. A good proportion of AWS issues
   relate to IAM policies which have been "normalized" (used loosely)
   by the IAM API endpoints. This can make the JSON strings differ
   from those generated by iam_policy_document resources or template
   files, even though the semantics are the same (for example,
   reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3
   bucket policy. DiffSupressFunc can be used to test for semantic
   equivalence rather than pure text equivalence, but without having to
   deal with the complexity associated with a full "provider-land" diff
   implementation without helper/schema.
2016-08-31 19:13:53 -05:00
James Bardin e0014198e1 Merge pull request #8560 from hashicorp/jbardin/race2
Fix races in WaitForState
2016-08-31 14:02:43 -04:00
Mitchell Hashimoto aaaed823af Merge pull request #8508 from TimeIncOSS/f-schema-all-validation-errors
schema: Return all validation errors together from InternalValidate
2016-08-31 10:44:13 -07:00
James Bardin 481f12cf2c Remove `first` variable an extra if block
clean up the code slighly by moving the Sleep in WaitForState to the end
of the loop.
2016-08-30 17:56:26 -04:00
James Bardin 82be35a797 Fix races in WaitForState
The WaitForState method can't read the result values in a timeout
because they are still owned by the running goroutine. Keep all values
scoped inside the goroutine, and save them into an atomic.Value to be
returned.

Fixes race introduced in #8510
2016-08-30 16:22:21 -04:00
Radek Simko 506c118383
helper/resource: Show last state in timeout err message 2016-08-27 21:25:14 +01:00
Radek Simko aef2513b44
helper/resource: Fix WaitForState tests 2016-08-27 15:11:09 +01:00
Radek Simko b90ab0b705
schema: Return all validation errors together from InternalValidate 2016-08-27 10:34:16 +01:00
James Nugent df06d5623d Merge pull request #8249 from meteor/glasser/name-prefix-timestamp
core: name_prefix names now start with a timestamp
2016-08-23 14:37:56 +02:00
Kazunori Kojima ed05161fd0
provider/aws: Support import `aws_s3_bucket_notification` 2016-08-23 08:19:06 +09:00
James Nugent 6ae7140251 Merge pull request #8370 from hashicorp/dtolnoy-wait
Do not sleep between first attempt or between successful attempts
2016-08-22 14:30:54 +02:00
James Nugent 0a5acaa0d8 Change error message return structure 2016-08-22 10:34:32 +02:00
David Tolnay 9d3f40a513 Fix bad message from TimeoutError
Before:

    Error creating IAM Role my-role: timeout while waiting for state
    to become 'success'. last error: %!s(<nil>)
2016-08-22 10:23:53 +02:00
David Tolnay e27a735da2 Do not sleep between first attempt or between successful attempts 2016-08-22 10:17:23 +02:00
David Glasser 4ad825fe08 core: name_prefix names now start with a timestamp
This means that two resources created by the same rule will get names
 which sort in the order they are created.

The rest of the ID is still random base32 characters; we no longer set
the bit values that denote UUIDv4.

The length of the ID returned by PrefixedUniqueId is not changed by this
commit; that way we don't break any resources where the underlying
resource has a name length limit.

Fixes #8143.
2016-08-17 11:06:28 -07:00
James Nugent b551981cc7 testing: Add ComposeAggregateTestFunc
This commit adds a function which composes a series of TestFuncs, but
will run all tests before returning an error, unlike ComposeTestFunc.
This is useful when verifying contents of state in acceptance tests and
it is desirable to see all the failing cases in one run for slow
resources.
2016-08-16 19:56:18 +01:00
Mitchell Hashimoto f26d1b40e0 Merge pull request #8215 from hashicorp/f-signalwrapper
helper/signalwrapper and azurerm_storage_account listens for signals
2016-08-16 09:57:54 -07:00
Mitchell Hashimoto 8dafcb36fd
providers/azurerm: cancellable storage account creation 2016-08-15 21:12:32 -07:00
Mitchell Hashimoto d8337920f9
helper/signalwrapper: more tests 2016-08-15 20:58:48 -07:00
Mitchell Hashimoto bdcea55117
helper/signalwrapper 2016-08-15 20:55:35 -07:00
James Nugent b681ab23d3 testing: Add TestCheckResourceAttrSet helper
This commit adds a TestCheckFunc which ensures that a value is set for a
given name/key combination. It is primarily useful for ensuring that
computed values are set where it is not possible to know the expected
value ahead of time.
2016-08-15 15:42:25 -05:00
Paul Hinze 3dccfa0cc9
terraform: diffs with only tainted set are non-empty
Fixes issue where a resource marked as tainted with no other attribute
diffs would never show up in the plan or apply as needing to be
replaced.

One unrelated test needed updating due to a quirk in the testDiffFn
logic - it adds a "type" field diff if the diff is non-Empty. NBD
2016-08-12 17:37:49 -05:00
Peter McAtominey 8abec085ec helper: create validation package to provide common validation functions 2016-08-10 13:27:49 +01:00
James Bardin 1af7ee87a2 Silence log output when not verbose
Set the default log package output to iotuil.Discard during tests if the
`-v` flag isn't set. If we are verbose, then apply the filter according
to the TF_LOG env variable.
2016-08-01 17:19:14 -04:00
James Bardin 074be9ae56 Another race in resource.Retry 2016-07-29 18:35:54 -04:00
Paul Hinze 261043fd1a Merge pull request #7523 from hashicorp/f-aws-beanstalk-env-poll-timing
provider/aws: Beanstalk environments, bump the minimum timeout between API calls
2016-07-21 16:43:21 -05:00
Paul Hinze 614806d59f
helper/resource: Fix import test harness, which was modifying state
Maps are reference types, it turns out :D
2016-07-15 13:15:47 -06:00
clint shryock de60481428 provider/aws: Rework Beanstalk optional polling
expose a poll_interval for users to configure polling for updates
2016-07-13 15:38:23 -06:00
David Tolnay 7096e4d3da Return nonnegative hash if int is 32 bits 2016-07-01 14:40:53 -07:00
Paul Hinze 3b732131d2 Merge pull request #7446 from hashicorp/b-jit-resource-validate
core: rerun resource validation before plan and apply
2016-07-01 15:00:40 -05:00
Paul Hinze ad27190852 Merge pull request #7448 from dtolnay/timeout
Timing out is not success
2016-07-01 14:28:11 -05:00
Paul Hinze afccf62e3e Merge pull request #7459 from hashicorp/r-resource-unit-test
helper/resource: Consolidate unit test override
2016-07-01 13:23:13 -05:00
Paul Hinze 4a1b36ac0d
core: rerun resource validation before plan and apply
In #7170 we found two scenarios where the type checking done during the
`context.Validate()` graph walk was circumvented, and the subsequent
assumption of type safety in the provider's `Diff()` implementation
caused panics.

Both scenarios have to do with interpolations that reference Computed
values. The sentinel we use to indicate that a value is Computed does
not carry any type information with it yet.

That means that an incorrect reference to a list or a map in a string
attribute can "sneak through" validation only to crop up...

 1. ...during Plan for Data Source References
 2. ...during Apply for Resource references

In order to address this, we:

 * add high-level tests for each of these two scenarios in `provider/test`
 * add context-level tests for the same two scenarios in `terraform`
   (these tests proved _really_ tricky to write!)
 * place an `EvalValidateResource` just before `EvalDiff` and `EvalApply` to
   catch these errors
 * add some plumbing to `Plan()` and `Apply()` to return validation
   errors, which were previously only generated during `Validate()`
 * wrap unit-tests around `EvalValidateResource`
 * add an `IgnoreWarnings` option to `EvalValidateResource` to prevent
   active warnings from halting execution on the second-pass validation

Eventually, we might be able to attach type information to Computed
values, which would allow for these errors to be caught earlier. For
now, this solution keeps us safe from panics and raises the proper
errors to the user.

Fixes #7170
2016-07-01 13:12:57 -05:00
Paul Hinze 5656d7388c
helper/resource: Consolidate unit test override
I noticed we had two mechanisms for unit test override. One that dropped
a sentinel into the env var, and another with a struct member on
TestCase. This consolidates the two, using the cleaner struct member
internal mechanism and the nicer `resource.UnitTest()` entry point.
2016-07-01 13:08:43 -05:00
David Tolnay 7e1cd34819 Timing out is not success 2016-06-30 18:07:56 -07:00
Clint a84aa5e914 Revert "helper/schema: Make nested Set(s) in List(s) work" (#7436) 2016-06-30 10:48:52 -05:00
Radek Simko 37d57f4a85 Improve naming convention (resultSet -> exists) 2016-06-28 17:40:45 +01:00
Radek Simko 917ad44cf0 helper/schema: Fix readSet implementation (DiffFieldReader) 2016-06-28 17:40:44 +01:00
Radek Simko c738c5a9a3 helper/schema: Implement reader-specific readList method 2016-06-28 17:40:44 +01:00
Radek Simko 378b526dc3 helper/schema: Add regression tests for nested Set/List
Although DiffFieldReader was the one mostly responsible for a buggy behaviour
more tests were added throughout the debugging process most of which
would fail without the bugfix.

 - ResourceData
 - MultiLevelFieldReader
 - MapFieldReader
 - DiffFieldReader
2016-06-28 17:40:44 +01:00
Patrick Sodré 80936e3562 provider/triton: add machine domain names (#7149)
* vendor joyent/gosdc/cloudapi

* provider/triton: Add machine domain names

 - Includes acceptance test.
2016-06-26 22:18:17 +01:00
James Nugent dbf725bd68 core: Allow dynamic attributes in helper/schema
The helper/schema framework for building providers previously validated
in all cases that each field being set in state was in the schema.
However, in order to support remote state in a usable fashion, the need
has arisen for the top level attributes of the resource to be created
dynamically. In order to still be able to use helper/schema, this commit
adds the capability to assign additional fields.

Though I do not forsee this being used by providers other than remote
state (and that eventually may move into Terraform Core rather than
being a provider), the usage and semantics are:

To opt into dynamic attributes, add a schema attribute named
"__has_dynamic_attributes", and make it an optional string with no
default value, in order that it does not appear in diffs:

        "__has_dynamic_attributes": {
            Type: schema.TypeString
            Optional: true
        }

In the read callback, use the d.UnsafeSetFieldRaw(key, value) function
to set the dynamic attributes.

Note that other fields in the schema _are_ copied into state, and that
the names of the schema fields cannot currently be used as dynamic
attribute names, as we check to ensure a value is not already set for a
given key.
2016-06-11 13:29:05 +01:00
James Nugent 9c7cf639b3 testing: Allow acceptance test to run as unit test
This commit adds a flag to acceptance tests in order to make
appropriately named tests work during `make test` irrespective of the
TF_ACC environment variable. This should only be used on tests which are
known to be fast.
2016-06-11 12:55:14 +01:00
clint shryock 7d71b8cc3c helper and terraform interpolate test update 2016-06-10 10:07:17 -05:00
James Bardin 2c7b702d1f Merge pull request #7091 from hashicorp/jbardin/serialize
Serialization for hash panics on TypeMap
2016-06-09 16:16:41 -04:00
James Bardin bab031aac5 Add test for TypeMap in a Schema 2016-06-09 16:00:33 -04:00
James Bardin d8fbaa7924 Serialization for hash panics on TypeMap
The serializeCollectionMemberForHash helper can't be called for the
MapType values, because MapType doesn't have a schema.Elem. Instead, we
can write the key/value pairs directly to the buffer. This still doesn't
allow for nested maps or lists, but we need to define that use case
before committing to it here.
2016-06-09 13:37:58 -04:00
James Nugent 074545e536 core: Use .% instead of .# for maps in state
The flatmapped representation of state prior to this commit encoded maps
and lists (and therefore by extension, sets) with a key corresponding to
the number of elements, or the unknown variable indicator under a .# key
and then individual items. For example, the list ["a", "b", "c"] would
have been encoded as:

    listname.# = 3
    listname.0 = "a"
    listname.1 = "b"
    listname.2 = "c"

And the map {"key1": "value1", "key2", "value2"} would have been encoded
as:

    mapname.# = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

Sets use the hash code as the key - for example a set with a (fictional)
hashcode calculation may look like:

    setname.# = 2
    setname.12312512 = "value1"
    setname.56345233 = "value2"

Prior to the work done to extend the type system, this was sufficient
since the internal representation of these was effectively the same.
However, following the separation of maps and lists into distinct
first-class types, this encoding presents a problem: given a state file,
it is impossible to tell the encoding of an empty list and an empty map
apart. This presents problems for the type checker during interpolation,
as many interpolation functions will operate on only one of these two
structures.

This commit therefore changes the representation in state of maps to use
a "%" as the key for the number of elements. Consequently the map above
will now be encoded as:

    mapname.% = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

This has the effect of an empty list (or set) now being encoded as:

    listname.# = 0

And an empty map now being encoded as:

    mapname.% = 0

Therefore we can eliminate some nasty guessing logic from the resource
variable supplier for interpolation, at the cost of having to migrate
state up front (to follow in a subsequent commit).

In order to reduce the number of potential situations in which resources
would be "forced new", we continue to accept "#" as the count key when
reading maps via helper/schema. There is no situation under which we can
allow "#" as an actual map key in any case, as it would not be
distinguishable from a list or set in state.
2016-06-09 10:49:42 +01:00
James Nugent cb9ef298f3 core: Defeat backward compatibilty in mapstructure
The mapstructure library has a regrettable backward compatibility
concern whereby a WeakDecode of []interface{}{} into a target of
map[string]interface{} yields an empty map rather than an error. One
possibility is to switch to using Decode instead of WeakDecode, but this
loses the nice handling of type conversion, requiring a large volume of
code to be added to Terraform or HIL in order to retain that behaviour.

Instead we add a DecodeHook to our usage of the mapstructure library
which checks for decoding []interface{}{} or []string{} into a map and
returns an error instead.

This has the effect of defeating the code added to retain backwards
compatibility in mapstructure, giving us the correct (for our
circumstances) behaviour of Decode for empty structures and the type
conversion of WeakDecode.

The code is identical to that in the HIL library, and packaged into a
helper.
2016-06-08 18:38:41 +01:00
James Nugent 91587a49f3 core: Remove unnecessary debug logging
Some unnecessary debug logging was introduced in 7b6df27e4, this commit
removes it so as not to clutter logs.
2016-06-08 18:38:41 +01:00
Radek Simko 1ea5cff9b3 Merge pull request #6946 from TimeIncOSS/f-aws-s3-object-data-source
provider/aws: Add aws_s3_bucket_object data source
2016-06-02 07:04:46 +01:00
Radek Simko bf8931b1ab provider/aws: Mark Lambda function as gone when it's gone (#6924)
* helper/error: Introduce TimeoutError & UnexpectedStateError

* provider/aws: Mark Lambda function as gone when it's gone
2016-06-02 00:50:43 +01:00
Radek Simko 84ab00d92e helper/resource: Add TestStep.PreventPostDestroyRefresh
- This is to allow easier testing of data sources which read data from resources created in the same scope
2016-06-01 19:13:00 +01:00
Chris Marchesi 9d7fb89114 core: Adding Sensitive attribute to resource schema
This an effort to address hashicorp/terraform#516.

Adding the Sensitive attribute to the resource schema, opening up the
ability for resource maintainers to mark some fields as sensitive.
Sensitive fields are hidden in the output, and, possibly in the future,
could be encrypted.
2016-05-29 22:18:44 -07:00
Sander van Harmelen 8560f50cbc
Change taint behaviour to act as a normal resource
This means it’s shown correctly in a plan and takes into account any
actions that are dependant on the tainted resource and, vice verse, any
actions that the tainted resource depends on.

So this changes the behaviour from saying this resource is tainted so
just forget about it and make sure it gets deleted in the background,
to saying I want that resource to be recreated (taking into account the
existing resource and it’s place in the graph).
2016-05-26 19:55:26 -05:00
Martin Atkins 031b561ef0 helper/resource: ignore data resource diffs during destroy
When testing destroy, the test harness calls Refresh followed by Plan,
with the expectation that the resulting diff will be empty.

Data resources challenge this expectation, because they will always be
instantiated during refresh if their configuration isn't computed, and so
the subsequent diff will want to destroy what was instantiated.

To work around this, we make an exception that data resource destroy
diffs may appear in the plan but nothing else.

This fixes #6713.
2016-05-21 14:16:02 -07:00
Mitchell Hashimoto 55583baa7e
Merge branch 'f-aws-import' 2016-05-18 15:28:12 -06:00
James Nugent 3ea3c657b5 core: Use OutputState in JSON instead of map
This commit forward ports the changes made for 0.6.17, in order to store
the type and sensitive flag against outputs.

It also refactors the logic of the import for V0 to V1 state, and
fixes up the call sites of the new format for outputs in V2 state.

Finally we fix up tests which did not previously set a state version
where one is required.
2016-05-18 13:25:20 -05:00
Mitchell Hashimoto 884980da1a
providers/aws: instance, nat, internet gateway 2016-05-16 10:03:57 -07:00
Mitchell Hashimoto b7d4767dd6
helper/schema: pass through import state func 2016-05-16 10:03:57 -07:00
Martin Atkins 6cd22a4c9a helper/schema: emit warning when using data source resource shim
For backward compatibility we will continue to support using the data
sources that were formerly logical resources as resources for the moment,
but we want to warn the user about it since this support is likely to
be removed in future.

This is done by adding a new "deprecation message" feature to
schema.Resource, but for the moment this is done as an internal feature
(not usable directly by plugins) so that we can collect additional
use-cases and design a more general interface before creating a
compatibility constraint.
2016-05-14 08:26:36 -07:00
Martin Atkins fb262d0dbe helper/schema: shim for making data sources act like resources
Historically we've had some "read-only" and "logical" resources. With the
addition of the data source concept these will gradually become data
sources, but we need to retain backward compatibility with existing
configurations that use the now-deprecated resources.

This shim is intended to allow us to easily create a resource from a
data source implementation. It adjusts the schema as needed and adds
stub Create and Delete implementations.

This would ideally also produce a deprecation warning whenever such a
shimmed resource is used, but the schema system doesn't currently have
a mechanism for resource-specific validation, so that remains just a TODO
for the moment.
2016-05-14 08:26:36 -07:00
Martin Atkins 6a468dcd83 helper/schema: Resource can be writable or not
In the "schema" layer a Resource is just any "thing" that has a schema
and supports some or all of the CRUD operations. Data sources introduce
a new use of Resource to represent read-only resources, which require
some different InternalValidate logic.
2016-05-14 08:26:36 -07:00
Martin Atkins 0e0e3d73af core: New ResourceProvider methods for data resources
This is a breaking change to the ResourceProvider interface that adds the
new operations relating to data sources.

DataSources, ValidateDataSource, ReadDataDiff and ReadDataApply are the
data source equivalents of Resources, Validate, Diff and Apply (respectively)
for managed resources.

The diff/apply model seems at first glance a rather strange workflow for
read-only resources, but implementing data resources in this way allows them
to fit cleanly into the standard plan/apply lifecycle in cases where the
configuration contains computed arguments and thus the read must be deferred
until apply time.

Along with breaking the interface, we also fix up the plugin client/server
and helper/schema implementations of it, which are all of the callers
used when provider plugins use helper/schema. This would be a breaking
change for any provider plugin that directly implements the provider
interface, but no known plugins do this and it is not recommended.

At the helper/schema layer the implementer sees ReadDataApply as a "Read",
as opposed to "Create" or "Update" as in the managed resource Apply
implementation. The planning mechanics are handled entirely within
helper/schema, so that complexity is hidden from the provider implementation
itself.
2016-05-14 08:26:36 -07:00
Mitchell Hashimoto 6a675b4a15
helper/resource: ImportState test can verify states 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto ec02c8c0e2
helper/resource: testing of almost all aspects of ImportState tests 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 2d99c451fb
helper/resource: basic ImportState acceptance testing
Still some TODOs, and more test cases to write, but the basics are all
here.
2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 9bd1c9e7ca
helper/resource: reshuffling to prepare for importstate testing 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto b728e55861
helper/schema: Resource.Data should set latest schema version 2016-05-11 13:02:36 -07:00
Mitchell Hashimoto c02c6c3f9c
helper/schema: default state func for import 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 2bb814e3de
helper/schema: adapt to ID being arg to ImportState 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 03931bfda9
helper/schema: ImportState must set ID on the resource data 2016-05-11 13:02:34 -07:00
Mitchell Hashimoto 445194ebdf
helper/schema: test ImportState 2016-05-11 13:02:33 -07:00
Mitchell Hashimoto 1685054a9a
helper/schema: cleaner way to store Ephemeral 2016-05-11 13:02:31 -07:00
Mitchell Hashimoto 84531a3fd5
helper/schema: sets Importable to true for resources that have importer 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 19609bde0e
helper/schema: can specify Importer on Resource, InternalValidate 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto b8121ea63e
helper/schema: Resource.Data to return a ResourceData for a Resource 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 4e9877179c
helper/schema: start the resource importer 2016-05-11 13:02:29 -07:00
James Nugent 7b6df27e4a helper/schema: Read native maps from configuration
This adds a test and the support necessary to read from native maps
passed as variables via interpolation - for example:

```
resource ...... {
     mapValue = "${var.map}"
}
```

We also add support for interpolating maps from the flat-mapped resource
config, which is necessary to support assignment of computed maps, which
is now valid.

Unfortunately there is no good way to distinguish between a list and a
map in the flatmap. In lieu of changing that representation (which is
risky), we assume that if all the keys are numeric, this is intended to
be a list, and if not it is intended to be a map. This does preclude
maps which have purely numeric keys, which should be noted as a
backwards compatibility concern.
2016-05-10 14:49:14 -04:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
Mitchell Hashimoto 35c87836b4 core: Add terraform_version to state
This adds a field terraform_version to the state that represents the
Terraform version that wrote that state. If Terraform encounters a state
written by a future version, it will error. You must use at least the
version that wrote that state.

Internally we have fields to override this behavior (StateFutureAllowed),
but I chose not to expose them as CLI flags, since the user can just
modify the state directly. This is tricky, but should be tricky to
represent the horrible disaster that can happen by enabling it.

We didn't have to bump the state format version since the absense of the
field means it was written by version "0.0.0" which will always be
older. In effect though this change will always apply to version 2 of
the state since it appears in 0.7 which bumped the version for other
purposes.
2016-05-10 14:40:11 -04:00
Paul Hinze b4df304b47
helper/schema: Normalize bools to "true"/"false" in diffs
For a long time now, the diff logic has relied on the behavior of
`mapstructure.WeakDecode` to determine how various primitives are
converted into strings.  The `schema.DiffString` function is used for
all primitive field types: TypeBool, TypeInt, TypeFloat, and TypeString.

The `mapstructure` library's string representation of booleans is "0"
and "1", which differs from `strconv.FormatBool`'s "false" and "true"
(which is used in writing out boolean fields to the state).

Because of this difference, diffs have long had the potential for
cosmetically odd but semantically neutral output like:

    "true" => "1"
    "false" => "0"

So long as `mapstructure.Decode` or `strconv.ParseBool` are used to
interpret these strings, there's no functional problem.

We had our first clear functional problem with #6005 and friends, where
users noticed diffs like the above showing up unexpectedly and causing
troubles when `ignore_changes` was in play.

This particular bug occurs down in Terraform core's EvalIgnoreChanges.
There, the diff is modified to account for ignored attributes, and
special logic attempts to handle properly the situation where the
ignored attribute was going to trigger a resource replacement. That
logic relies on the string representations of the Old and New fields in
the diff to be the same so that it filters properly.

So therefore, we now get a bug when a diff includes `Old: "0", New:
"false"` since the strings do not match, and `ignore_changes` is not
properly handled.

Here, we introduce `TypeBool`-specific normalizing into `finalizeDiff`.
I spiked out a full `diffBool` function, but figuring out which pieces
of `diffString` to duplicate there got hairy. This seemed like a simpler
and more direct solution.

Fixes #6005 (and potentially others!)
2016-05-05 09:00:58 -05:00
Mitchell Hashimoto d85df63526
providers/aws: aws_instance id-only 2016-04-22 09:37:41 -07:00
Mitchell Hashimoto e0da21d381
helper/resource: make id-only check opt-in
As I've been working through the resources, I'm finding that a lot are
going to need some serious work. Given we have hundreds, I think it
might be prudent to make this opt-in for now and we can revisit
automatic/opt-out at some future point.

Importability will likely be opt-in it appears so this will match up
with that.
2016-04-21 08:37:08 -07:00
Mitchell Hashimoto ff7b58f032
providers/aws: peering connection id-only test settings 2016-04-20 12:19:21 -07:00
Mitchell Hashimoto 0ef1b3b84a
providers/aws: response value for DescribeVpcAttribute needs to be
.Value
2016-04-20 11:35:43 -07:00
Mitchell Hashimoto 1db1bf6639
helper/resource: use the full config for id-only checks
Originally I used an empty config module. This caused problems since
important provider configurations weren't available. Instead, I now set
it to use the full config. This isn't an issue since the attributes
themselves aren't available to Refresh anyways.
2016-04-20 11:18:13 -07:00
Mitchell Hashimoto 0c8b0bff2c
helper/resource: can specify specific name to id refresh test 2016-04-20 11:12:30 -07:00
Mitchell Hashimoto 1a9fae6b2e
helper/resource: can disable ID refresh check 2016-04-20 11:09:54 -07:00
Mitchell Hashimoto 35f4201b9e
providers/aws: instance_tenancy is computed, set 2016-04-20 10:48:22 -07:00
Mitchell Hashimoto f2c4f8e9ba
helper/resource: fix tests 2016-04-20 10:08:34 -07:00
Mitchell Hashimoto 5c4f78796b
helper/resource: don't need to id-only check if no test steps 2016-04-20 09:52:53 -07:00
Mitchell Hashimoto a285c04dc9
helper/resource: only verify id-only run if no error 2016-04-20 09:50:59 -07:00
Mitchell Hashimoto baac14aaeb
helper/resource: guard id-only by acc var 2016-04-20 09:34:54 -07:00
Mitchell Hashimoto cb32cb8947
helper/resource: error if id-only check didn't run 2016-04-20 09:25:23 -07:00
Mitchell Hashimoto 86e0c853db
helper/resource: test for failing id-only refresh check 2016-04-20 09:18:25 -07:00
Mitchell Hashimoto 060b43fbd9
helper/resource: remove debug 2016-04-20 09:17:04 -07:00
Mitchell Hashimoto 4f6edf4fe4
helper/resource: id-only refresh testing 2016-04-20 09:16:48 -07:00
James Nugent cb8a0549e1 Merge pull request #5757 from hashicorp/f-output-testing
Add TestCheckOutput helper to resource testing
2016-03-21 17:19:08 +00:00
James Nugent 35f9d2e081 Add TestCheckOutput helper to resource testing
This allows outputs in test configuration to have test functions written
conveniently. Useful for azurerm_template_deployment.
2016-03-21 16:54:02 +00:00
Radek Simko bf9f5879ca helper/resource: Remove NotFoundError function 2016-03-21 16:47:50 +00:00
James Nugent f946695187 Merge pull request #5444 from TimeIncOSS/f-aws-logs-metric-filter
provider/aws: Add support for CloudWatch Log Metric Filter
2016-03-21 16:43:38 +00:00
Paul Hinze c3e27b3e0a provider/test: a test provider
Here we also introduce a `test` provider meant as an aid to exposing
via automated tests issues involving interactions between
`helper/schema` and Terraform core.

This has been helpful so far in diagnosing `ignore_changes` problems,
and I imagine it will be helpful in other contexts as well.

We'll have to be careful to prevent the `test` provider from becoming a
dumping ground for poorly specified tests that have a clear home
elsewhere. But for bug exposure I think it's useful to have.
2016-03-21 08:59:54 -05:00
Radek Simko fd2fcaf1d5 helper/resource: Implement resource-wide NotFoundError 2016-03-15 13:58:25 +00:00
Paul Hinze 25fce81bfc provider/aws: log HTTP req/resp at DEBUG level
This should be quite helpful in debugging aws-sdk-go operations.

Required some tweaking around the `helper/logging` functions to expose an
`IsDebugOrHigher()` helper for us to use.
2016-03-14 12:26:37 -05:00
Radek Simko 034287fdc2 helper/resource: Error shouldn't be returned in case of success 2016-03-10 14:14:14 +00:00
Radek Simko 8582f4df9f helper/resource: Fix TestRetry 2016-03-10 14:13:23 +00:00
Paul Hinze 108ccf0007 builtin: Refactor resource.Retry to clarify return
Change the `RetryFunc` from a plain `error` return type to a
specialized `RetryError` which must decide whether it is
retryable or not.

Add `RetryableError` / `NonRetryableError` factory functions that
callers are meant to use to build up these errors.

This makes it eminently clear whether or not a given error is
retryable from inside the client code.

Goal here is to _not_ change any behavior, simply reflect the
existing behavior with the new, clearer, API.
2016-03-09 17:37:56 -06:00
Paul Hinze 5160578e18 helper/resource: restore retval of resource.Retry on timeout
In #4700 while fixing a data race I made an incorrect assumption about
the return value of StateChangeConf, and ended up changing the behavior
in the timeout case to always return:

> timeout while waiting for state to become '[success]'

When it used to capture the "most recent error" from the function
itself.

It's much more useful to see that error bubbling up, so here we revert
to pulling it out of the function as we did before, and we protect
against the data race with a good old fashioned mutex.
2016-03-04 11:20:48 -06:00
Martin Atkins c1ce8ff31a Merge pull request #5218 from paybyphone/paybyphone_set_maxitems
Add MaxItems attribute to Schema
2016-03-01 09:27:59 -08:00
Radek Simko 8bdd92187c Merge pull request #4446 from TimeIncOSS/f-schema-new-resource
helper/schema: Allow identification of a new resource in update func
2016-02-29 20:07:00 +00:00
Paul Hinze bba8a79a52 acctests: log a line w/ the non-empty plan
Helpful when iterating on a drift test.

Eventually I think this assertion could be fanned out to something much
more targeted like:

    ExpectAttributeDiff(resource, attr, oldval, newval)

But this is a step in the right direction.
2016-02-29 11:50:42 -06:00
Chris Marchesi 8c5354b7dc Add MaxItems attribute to Schema
* MaxItems defines a maximum amount of items that can exist within a
   TypeSet or TypeList. Specific use cases would be if a TypeSet is being
   used to wrap a complex structure, however more than one instance would
   cause instability.
2016-02-23 16:41:32 -08:00
Radek Simko ab73f2b762 helper: Add StateChangeConf.ContinuousTargetOccurence (int) 2016-02-23 20:18:57 +00:00
Trevor Pounds 0cd0ff0f8e Use built-in schema.HashString. 2016-02-07 16:29:34 -08:00
Mitchell Hashimoto 4576eaa966 helper/schema: replace config/lang 2016-02-03 13:24:04 -05:00
Mitchell Hashimoto 55a9b1ba4c helper/diff: replace ocnfig/lang 2016-02-03 13:24:04 -05:00
Paul Hinze 24048b4dca providers: Mention check number when acctest fails 2016-02-02 10:57:28 -06:00
Paul Hinze da872eee66 Merge pull request #4864 from hashicorp/phinze/aws-min-elb-cap-regression
aws: undeprecate min_elb_capacity; restore min capacity waiting
2016-01-27 14:17:10 -06:00
Paul Hinze c70eab6500 aws: undeprecate min_elb_capacity; restore min capacity waiting
It was a mistake to switched fully to `==` when activating waiting for
capacity on updates in #3947. Users that didn't set `min_elb_capacity ==
desired_capacity` and instead treated it as an actual "minimum" would
see timeouts for every create, since their target numbers would never be
reached exactly.

Here, we fix that regression by restoring the minimum waiting behavior
during creates.

In order to preserve all the stated behavior, I had to split out
different criteria for create and update, criteria which are now
exhaustively unit tested.

The set of fields that affect capacity waiting behavior has become a bit
of a mess. Next major release I'd like to rework all of these into a
more consistently named block of config. For now, just getting the
behavior correct and documented.

(Also removes all the fixed names from the ASG tests as I was hitting
collision issues running them over here.)

Fixes #4792
2016-01-27 13:30:44 -06:00
Paul Hinze 069425a700 consul: Fix several problems w/ consul_keys update
Implementation notes:

 * The hash implementation was not considering key value, causing "diffs
   did not match" errors when a value was updated. Switching to default
   HashResource implementation fixes this
 * Using HashResource as a default exposed a bug in helper/schema that
   needed to be fixed so the Set function is picked up properly during
   Read
 * Stop writing back values into the `key` attribute; it triggers extra
   diffs when `default` is used. Computed values all just go into `var`.
 * Includes a state migration to prevent unnecessary diffs based on
   "key" attribute hashcodes changing.

In the tests:

 * Switch from leaning on the public demo Consul instance to requiring a
   CONSUL_HTTP_ADDR variable be set pointing to a `consul agent -dev`
   instance to be used only for testing.
 * Add a test that exposes the updating issues and covers the fixes

Fixes #774
Fixes #1866
Fixes #3023
2016-01-26 14:46:26 -06:00
Paul Hinze 0b11ace9ac Merge pull request #4788 from hashicorp/phinze/skip-remote-test-option
tests: allow opt-out of remote tests via env var
2016-01-25 10:57:08 -06:00
Paul Hinze 6bafa74011 tests: allow opt-out of remote tests via env var
Adds the `TF_SKIP_REMOTE_TESTS` env var to be used in cases where the
`http.Get()` smoke test passes but the network is not able to service
the needs of the tests.

Fixes #4421
2016-01-21 15:44:18 -06:00
Ian Duffy 47ac10d66b Change resource.StateChangeConf to use an array for target states
Signed-off-by: Ian Duffy <ian@ianduffy.ie>
2016-01-21 01:20:41 +00:00
Paul Hinze d59db52f06 helper/resource: Fix data race in resource.Retry
The implementation was attempting to capture the error using a scoped
variable reference, but the error value is already exposed via the
return value of `WaitForState()`. Using that instead fixes the data
race.

Fixes the following data race:

```
==================
WARNING: DATA RACE
Read by goroutine 74:
  github.com/hashicorp/terraform/helper/resource.Retry()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/wait.go:35 +0x284
  github.com/hashicorp/terraform/helper/resource.TestRetry_timeout()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/wait_test.go:35 +0x60
  testing.tRunner()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:456 +0xdc

Previous write by goroutine 90:
  github.com/hashicorp/terraform/helper/resource.Retry.func1()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/wait.go:20 +0x87
  github.com/hashicorp/terraform/helper/resource.(*StateChangeConf).WaitForState.func1()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/state.go:83 +0x284

Goroutine 74 (running) created at:
  testing.RunTests()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:561 +0xaa3
  testing.(*M).Run()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:494 +0xe4
  main.main()
      github.com/hashicorp/terraform/helper/resource/_test/_testmain.go:84 +0x20f

Goroutine 90 (running) created at:
  github.com/hashicorp/terraform/helper/resource.(*StateChangeConf).WaitForState()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/state.go:127 +0x283
  github.com/hashicorp/terraform/helper/resource.Retry()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/wait.go:34 +0x276
  github.com/hashicorp/terraform/helper/resource.TestRetry_timeout()
      /Users/phinze/go/src/github.com/hashicorp/terraform/helper/resource/wait_test.go:35 +0x60
  testing.tRunner()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:456 +0xdc
==================
```
2016-01-16 13:38:50 -05:00
Paul Hinze a8d2ad3ebe refactor s3 bucket test to expect non-empty plan
pushing to master but paging @catsby for post-hoc review
2016-01-05 17:38:38 -06:00
Paul Hinze e916bd1527 provider/google: enchance storage acctests to avoid collisions
Generate bucket names and object names per test instead of once at the
top level. Should help avoid failures like this one:

https://travis-ci.org/hashicorp/terraform/jobs/100254008

All storage tests checked on this commit:

```
TF_ACC=1 go test -v ./builtin/providers/google -run TestAccGoogleStorage
=== RUN   TestAccGoogleStorageBucketAcl_basic
--- PASS: TestAccGoogleStorageBucketAcl_basic (8.90s)
=== RUN   TestAccGoogleStorageBucketAcl_upgrade
--- PASS: TestAccGoogleStorageBucketAcl_upgrade (14.18s)
=== RUN   TestAccGoogleStorageBucketAcl_downgrade
--- PASS: TestAccGoogleStorageBucketAcl_downgrade (12.83s)
=== RUN   TestAccGoogleStorageBucketAcl_predefined
--- PASS: TestAccGoogleStorageBucketAcl_predefined (4.51s)
=== RUN   TestAccGoogleStorageObject_basic
--- PASS: TestAccGoogleStorageObject_basic (3.77s)
=== RUN   TestAccGoogleStorageObjectAcl_basic
--- PASS: TestAccGoogleStorageObjectAcl_basic (4.85s)
=== RUN   TestAccGoogleStorageObjectAcl_upgrade
--- PASS: TestAccGoogleStorageObjectAcl_upgrade (7.68s)
=== RUN   TestAccGoogleStorageObjectAcl_downgrade
--- PASS: TestAccGoogleStorageObjectAcl_downgrade (7.37s)
=== RUN   TestAccGoogleStorageObjectAcl_predefined
--- PASS: TestAccGoogleStorageObjectAcl_predefined (4.16s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/google 68.275s
```
2016-01-05 09:07:45 -06:00
Paul Hinze 028664a015 provider/digitalocean: acctest improvements
* Add SSH Keys to all droplets in tests, this prevents acctests from
   spamming account owner email with root password details
 * Add a new helper/acctest package to be a home for random string / int
   implementations used in tests.
 * Insert some random details into record tests to prevent collisions
 * Normalize config style in tests to hclfmt conventions
2016-01-04 15:30:35 -06:00
Radek Simko 4c6ceef9b8 helper/schema: Allow identification of a new resource in update func 2015-12-27 14:01:03 +01:00
James Nugent e976d6e787 testing: Use a copy of pre-destroy state in destroy check
In the acceptance testing framework, it is neccessary to provide a copy
of the state _before_ the destroy is applied to the check in order that
it can loop over resources to verify their destruction. This patch makes
a deep copy of the state prior to applying test steps which have the
Destroy option set and then passes that to the destroy check.
2015-12-10 13:14:17 -05:00
Paul Hinze edaf5795a5 Merge pull request #3257 from fatih/fix-nil-setting-schema
schema: delete non existing values
2015-12-08 20:15:00 -06:00
Paul Hinze 4bd4e18def core: use same logging setup for acctests
We weren't doing any log setup for acceptance tests, which made it
difficult to wrangle log output in CI.

This moves the log setup functions we use in `main` over into a helper
package so we can use them for acceptance tests as well.

This means that acceptance tests will by default be a _lot_ quieter,
only printing out actual test output. Setting `TF_LOG=trace` will
restore the full prior noise level.

Only minor behavior change is to make `ioutil.Discard` the default
return value rather than a `nil` that needs to be checked for.
2015-12-08 17:50:36 -06:00
Paul Hinze 99244c5597 helper/schema: skip provider input for deprecated fields
There's no reason that a field that's been deprecated should ever
prompt.

fixes #4033
2015-12-07 11:28:45 -06:00