Commit Graph

515 Commits

Author SHA1 Message Date
Jake Champlin 7894478c8c Merge pull request #14681 from svanharmelen/f-review
Use `helpers.shema.Provisoner` in Chef provisioner V2
2017-05-30 14:26:51 -04:00
James Bardin 6bc52be0a5 check for IsComputed when validating a schema obj
GH-14784 allowed nested structures to be validate, rather than relying
on the raw value. However this still returns the same validation error
if the structures contain a computed value, since Get will return the
raw string in that case.

This simply skips the validation in the IsComputed case, since there's
nothing that can be checked.
2017-05-24 12:34:53 -04:00
Matt Robenolt b9a3433f6b helper/schema: fix validating nested objects
When interpreting a nested object, we were validating against the "raw"
value, and not the interpolated value, causing incorrect errors.

This affects structures such as:

```tf
tags = "${list(map("foo", "bar"))}"
```

Prior to this, a complaint about "expected object, got string" since the
raw value is obviously a string, when the interpolated value is the
correct shape.
2017-05-24 01:57:13 -07:00
Sander van Harmelen 0e422737ba Fix and refactor the Chef provisioner
The tests did pass, but that was because they only tested part of the changes. By using the `schema.TestResourceDataRaw` function the schema and config are better tested and so they pointed out a problem with the schema of the Chef provisioner.

The `Elem` fields did not have a `*schema.Schema` but a `schema.Schema` and in an `Elem` schema only the `Type` field may (and must) be set. Any other fields like `Optional` are not allowed here.

Next to fixing that problem I also did a little refactoring and cleaning up. Mainly making the `ProvisionerS` private (`provisioner`) and removing the deprecated fields.
2017-05-19 21:05:21 +02:00
Vladislav Rassokhin f5449a62e0 Various built-in provisioners improvements:
1. Migrate `chef` provisioner to `schema.Provisioner`:

 * `chef.Provisioner` structure was renamed to `ProvisionerS`and  now it's decoded from `schema.ResourceData` instead of `terraform.ResourceConfig` using simple copy-paste-based solution;
 * Added simple schema without any validation yet.

 2. Support `ValidateFunc` validate function : implemented in `file` and `chef` provisioners.
2017-05-19 20:43:51 +02:00
Radek Simko 44a99e0ae5 core: Avoid crash on empty TypeSet blocks (#14305) 2017-05-09 20:45:53 +02:00
Radek Simko 9867ce4dde
helper/schema: Disallow validation+diff suppression on computed-only fields 2017-04-23 12:25:40 +02:00
James Bardin 9aaf220efb Merge pull request #13863 from hashicorp/jbardin/computed-list-requires-new
make sure a computed list is can be RequiresNew
2017-04-21 18:02:14 -04:00
James Bardin caadb4297f make sure a computed list is can be RequiresNew
If a schema.TypeList had a Schema with ForceNew, and if that list was
NewComputed, the diff would not have RequiresNew set. This causes apply
to fail when the diffs didn't match because of the change to
RequiresNew.

Set the RequiresNew field on the list's ResourceAttrDiff based on the
Schema value.
2017-04-21 17:51:15 -04:00
Martin Atkins b1763e262a Restore stringer-generated files back to new version
stringer has changed the boilerplate it generates in a recent version.
We'd previously updated to the new format but accientally rolled back
to the old while merging a long-running feature branch.

This restores us back to the new format again.
2017-04-21 14:49:18 -07:00
Jasmin Gacic 61499cfcf0 Provider Oneandone (#13633)
* Terraform Provider 1&1

* Addressing pull request remarks

* Fixed imports

* Fixing remarks

* Test optimiziation
2017-04-21 17:19:10 +03:00
Sander van Harmelen 3d0073e05c core: fix a crash by suggesting a different approach to solve #11170 (#13541)
* Revert #11245, #11321, #11498 and #11757

These PR’s are all related to issue #11170 for which I would like to propose a different solution then the one currently implemented.

* A different approach to solve #11170

This approach has (IMHO) a few advantages with regards to the solution currently implemented. I will elaborate on this in the PR.
2017-04-14 22:32:30 +02:00
James Bardin 735dfc416d Merge pull request #13427 from hashicorp/jbardin/context-keys
Fix context key types in schema
2017-04-10 15:38:27 -04:00
Martin Atkins a0269c688c helper/schema: Clarify the expectations for DefaultFunc
Discussion in #9512 revealed that some of the comments here were
inaccurate and that the comments here did not paint a complete enough
picture of the behavior and expectations of Default and DefaultFunc.

This is a comments-only change that aims to clarify the situation and
call attention to the fact that the defaults only affect the handling of
the configuration and that changes to defaults may require migration of
existing resource states.

This closes #9512.
2017-04-06 09:51:43 -07:00
James Bardin 8a1089a161 convert the other context keys to the correct type 2017-04-06 10:51:24 -04:00
James Bardin 812f9fb253 don't use primitive types for context value keys
A context value key should be typed within the package namespace,
otherwise different packages could have colliding values in a context.
2017-04-06 10:36:06 -04:00
Martin Atkins 21cd5595e2 Update stringer-generated files to new boilerplate
golang/tools commit 23ca8a263 changed the format of the leading comment
to comply with some new standards discussed here:
https://golang.org/issue/13560

This is the result of running generate with the latest version of
stringer. Everyone working on Terraform will need to update stringer
after this is merged, to avoid reverting this:
    go get -u golang.org/x/tools/cmd/stringer
2017-03-29 08:07:06 -07:00
James Bardin 2e6a44d5ff reify the list values before validation
If the list was marked as computed, all values will be raw config
values. Fetch the individual keys from the config to get any known
values before validating.
2017-03-24 12:04:18 -04:00
James Bardin 99a12f5df3 interpolation strings were beeing validated
Interpolation strings for non-computed values in a list were being
passed to the schema's ValidateFunc.
2017-03-24 12:04:18 -04:00
James Bardin efd0f5b0db Fix logic when skipping schema input
The Required||Optional logic in schemaMap.Input was incorrect, causing
it to always request input. Fix the logic, and the associated tests
which were passing "just because".
2017-03-17 14:55:24 -04:00
Radek Simko 1df1c21d5b schema: Allow *Resource as Elem of TypeMap in validation (#12722) 2017-03-15 14:54:41 +00:00
Radek Simko afd34f79df schema: Enable map value validation (#12638) 2017-03-13 15:58:58 +00:00
Clint 3fdeacdca7 helper/schema: Rename Timeout resource block to Timeouts (#12533)
helper/schema: Rename Timeout resource block to Timeouts

- Pluralize configuration argument name to better represent that there is
one block for many timeouts
- use a const for the configuration timeouts key
- update docs
2017-03-09 14:40:14 -06:00
James Bardin ecb1944c31 Merge pull request #12498 from hashicorp/jbardin/test-reset
Add schema.Provider.TestReset to reset StopContext between tests
2017-03-09 08:34:54 -05:00
James Bardin 1d9d7be28c Add schema.Provider.TestReset method
Provider.TestReset resets the internal state of the Provider at the
start of a test. This also adds a MetaReset function field to
schema.Provider, which is called by TestReset and can be used to reset
any other tsated stored in the provider metadata.

This is currently used to reset the internal Context returned by
StopContext between tests, and should  be implemented by a provider if
it stores a Context from a previous test.
2017-03-07 11:39:11 -05:00
clint shryock aa3677cd89 helper/schema: Guard against a panic if Timeout is not properly
structured
2017-03-07 10:25:32 -06:00
Clint 2fe5976aec helper/schema: Add configurable Timeouts (#12311)
* helper/schema: Add custom Timeout block for resources

* refactor DefaultTimeout to suuport multiple types. Load meta in Refresh from Instance State

* update vpc but it probably wont last anyway

* refactor test into table test for more cases

* rename constant keys

* refactor configdecode

* remove VPC demo

* remove comments

* remove more comments

* refactor some

* rename timeKeys to timeoutKeys

* remove note

* documentation/resources: Document the Timeout block

* document timeouts

* have a test case that covers 'hours'

* restore a System default timeout of 20 minutes, instead of 0

* restore system default timeout of 20 minutes, refactor tests, add test method to handle system default

* rename timeout key constants

* test applying timeout to state

* refactor test

* Add resource Diff test

* clarify docs

* update to use constants
2017-03-02 11:07:49 -06:00
Mitchell Hashimoto 3342aa580c
terraform: InstanceState.Meta is value type interface{}
This changes the type of values in Meta for InstanceState to
`interface{}`. They were `string` before.

This will allow richer structures to be persisted to this without
flatmapping them (down with flatmap!). The documentation clearly states
that only primitives/collections are allowed here.

The only thing using this was helper/schema for schema versioning.
Appropriate type checking was added to make this change safe.

The timeout work @catsby is doing will use this for a richer structure.
2017-02-23 10:44:05 -08:00
Mitchell Hashimoto c6d0333dc0
flatmap: mark computed list as a computed value in Expand
Fixes #12183

The fix is in flatmap for this but the entire issue is a bit more
complex. Given a schema with a computed set, if you reference it like
this:

    lookup(attr[0], "field")

And "attr" contains a computed set within it, it would panic even though
"field" is available. There were a couple avenues I could've taken to
fix this:

1.) Any complex value containing any unknown value at any point is
entirely unknown.

2.) Only the specific part of the complex value is unknown.

I took route 2 so that the above works without any computed (since
"name" is not computed but something else is). This may actually have an
effect on other parts of Terraform configs, however those similar
configs would've simply crashed previously so it shouldn't break any
pre-existing configs.
2017-02-23 10:03:59 -08:00
James Bardin 7359a18a71 Make sure to diff all nested schema.Set elements
This follows on GH-11498, using the same method to ensure all set
elements are marked as NewRemoved if the set is being removed in the
diff.
2017-02-07 16:55:20 -05:00
Mitchell Hashimoto 61881d2795 Merge pull request #10934 from hashicorp/f-provisioner-stop
core: stoppable provisioners, helper/schema for provisioners
2017-01-30 12:53:15 -08:00
Radek Simko d5ac48de2a helper/schema: Remove missed subfields when parent list is removed (#11498) 2017-01-29 21:15:00 +00:00
Mitchell Hashimoto 487a37b0dd
helper/schema: PromoteSingle for legacy support of "maybe list" types 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto 3c0c81957a
provisioners/remote-exec: switch to helper/schema 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto a2e044829b
provisioners/file: use the old communicator.New just to minimize risk 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto a1da59a73e
helper/schema: provisioner allows for nil state 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto b2891bc9ef
helper/schema: Provisioner support 2017-01-26 15:03:27 -08:00
Mitchell Hashimoto 251e5c6f87
helper/schema: framework for Backends 2017-01-26 14:33:49 -08:00
Conor Mongey 77c8683281 provider/vault: Remove user input for optional vault provider fields (#11082)
* Remove the need to input vault optional settings

* Allow TypeList to skip input

* Remove conflicts on vault ca_cert_* fields
2017-01-17 12:06:55 +00:00
clint shryock 6bafd4c896 fix typo 2017-01-10 11:06:51 -06:00
James Bardin 01be1a5ecd Check for interpolated values when reading a map
Accessing an interpolated value in a map through ConfigFieldReader can
fail, because GetRaw can't access interpolated values, so check if the
value exists at all by looking in the config. If the config has a value,
assume our map's value is interpolated and proceed as such.

We also need to lookup the correct schema to properly read a field from
a nested structure.

- Maps previously always defaulted to TypeString. Now check if Elem is a
  ValueType and use that if applicable
- Lists now return the schema for nested element types, defaulting to a
  TypeString like maps.

This only allows maps and lists to be nested one level deep, and the
inner map or list must only contain string values.
2016-11-28 09:04:24 -05:00
Mitchell Hashimoto 0634aada69
Revert "core: Ensure hasComputedSubKeys iterates over Sets and Lists properly"
This reverts commit 4d8208d840.
2016-11-21 17:31:19 -08:00
Mitchell Hashimoto f2fcf1494a
Merge branch 'fix-field-reader-setsubkeys-computed' of https://github.com/vancluever/terraform into vancluever-fix-field-reader-setsubkeys-computed 2016-11-21 17:24:26 -08:00
Chris Marchesi 8d06d68d0f core: Backport NewComputed change to nested list/set tests
Needed due to work done in 95d37ea, we may need to adjust
hasComputedSubKeys to propagate NewComputed in the same way that we
have added "~", however will wait for comment from @mitchellh.
2016-11-19 09:29:48 -08:00
Chris Marchesi f258056731 core: Tests for hasComputedSubKeys fix
This covers:

 * Complex sets with computed fields in a set
 * Complex lists with computed fields in a set

Adding a test to test basic lists with computed fields seemed to fail,
but possibly for an unrelated reason (the list returned as nil). The fix
to this inparticular case may be out of the scope of this specific
issue.

Reference gist and details in hashicorp/terraform#9171.
2016-11-19 08:56:16 -08:00
Chris Marchesi 4d8208d840 core: Ensure hasComputedSubKeys iterates over Sets and Lists properly
This fixes some edge-ish cases where a set in a config has a set or list
in it that contains computed values, but non-set or list values in the
parent do not.

This can cause "diffs didn't match during apply" errors in a scenario
such as when a set's hash is calculated off of child items (including
any sub-lists or sets, as it should be), and the hash changes between
the plan and apply diffs due to the computed values present in the
sub-list or set items. These will be marked as computed, but due to the
fact that the function was not iterating over the list or set items
properly (ie: not adding the item number to the address, so
set.0.set.foo was being yielded instead of set.0.set.0.foo), these
computed values were not being properly propagated to the parent set to
be marked as computed.

Fixes hashicorp/terraform#6527.
Fixes hashicorp/terraform#8271.

This possibly fixes other non-CloudFront related issues too.
2016-11-19 08:56:16 -08:00
James Bardin c4eefd4b5e Allow primitive type in maps via all FieldReaders
Now that we can read primitive type out of a map, each field reader
needs to be able to set the proper type as expected.
2016-11-17 15:35:08 -05:00
James Bardin 730014b33e Convert the map fields values when reading diff
Convert the value to the correct type when reading a diff and the map
schema has an primitive Elem type.
2016-11-17 14:34:18 -05:00
Radek Simko ad34f1ec74 Add (failing) test for map w/ non-string values 2016-11-17 11:46:32 -05:00
Mitchell Hashimoto 39542898b0
helper/schema: mark diff as forcenew if element is computed
Fixes #10125

If the elements are computed and the field is ForceNew, then we should
mark the computed count as potentially forcing a new operation.

Example, assuming `groups` forces new...

**Step 1:**

    groups = ["1", "2", "3"]

At this point, the resource isn't create, so this should result in a
diff like:

    CREATE resource:
      groups: "" => ["1", "2", "3"]

**Step 2:**

    groups = ["${computedvar}"]

The OLD behavior was:

    UPDATE resource
      groups.#: "3" => "computed"

This would cause a diff mismatch because if `${computedvar}` was
different then it should force new. The NEW behavior is:

    DESTROY/CREATE resource:
      groups.#: "3" => "computed" (forces new)
2016-11-15 11:02:14 -08:00
Mitchell Hashimoto 15f50b86bf
helper/schema: passing tests for computed complex sets 2016-11-10 08:52:37 -08:00
Mitchell Hashimoto 29485f6167
terraform: ResourceConfig.IsComputed cases 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 5792b2cba2
helper/schema: convert _Diff to subtests 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 2b7177cfe7 Merge pull request #9607 from hashicorp/f-provider-stop-redo
terraform: ResourceProvider.Stop (redo)
2016-11-08 15:58:48 -08:00
Mitchell Hashimoto e45debe0e5
helper/schema: only mark "ForceNew" on resources that cause the ForceNew
Fixes #2748

This changes the diff to only mark "forces new resource" on the fields
that actually caused the new resource, not every field that changed.
This makes diffs much more accurate.

I'd like to request a review but I'm going to defer merging until
Terraform 0.8. Changes like this are very possible to cause "diffs
didn't match" errors and I want some real world testing in a beta before
we hit prod with this.
2016-11-08 15:49:28 -08:00
Mitchell Hashimoto b7bee66df5
helper/schema: sort errors in helper/schema test for deterministic tests 2016-11-04 16:51:26 -07:00
Mitchell Hashimoto f0abe6d1a0 Merge pull request #9812 from hashicorp/b-bool-computed-crash
helper/schema: computed bool fields should not crash
2016-11-04 08:47:49 -07:00
Mitchell Hashimoto 65b17ccd06
helper/schema: allow ConflictsWith and Computed Optional fields 2016-11-02 22:24:34 -07:00
Mitchell Hashimoto 7834cf7190
helper/schema: computed bool fields should not crash
Fixes #7715

If a bool field was computed and the raw value was not convertable to a
boolean, helper/schema would crash. The correct behavior is to try not
to read the raw value when the value is computed and to simply mark that
it is computed. This does that (and matches the behavior of the other
primitives).
2016-11-02 13:25:23 -07:00
Mitchell Hashimoto 2d84582881 Merge pull request #9699 from hashicorp/b-removed-forcenew
helper/schema: removed optional items force new
2016-10-31 13:24:36 -07:00
Mitchell Hashimoto f7a234bc71
helper/schema: validate Read, Delete are set 2016-10-30 15:04:32 -07:00
Mitchell Hashimoto 5489d8c549
helper/schema: removed optional items force new
Fixes #5138

If an item is optional and is removed completely from the configuration,
it should still trigger a destroy/create if the field itself was marked
as "ForceNew".

See the example in #5138.
2016-10-28 18:45:12 -04:00
Mitchell Hashimoto 3f36787207 Merge pull request #9618 from hashicorp/b-computed-prim
helper/schema,terraform: handle computed primitives in diffs
2016-10-28 10:44:13 -04:00
Mitchell Hashimoto 95d37ea79c
helper/schema,terraform: handle computed primtives in diffs
Fixes #3309

There are two primary changes, one to how helper/schema creates diffs
and one to how Terraform compares diffs. Both require careful
understanding.

== 1. helper/schema Changes

helper/schema, given any primitive field (string, int, bool, etc.)
_used to_ create a basic diff when given a computed new value (i.e. from
an unkown interpolation). This would put in the plan that the old value
is whatever the old value was, and the new value was the actual
interpolation. For example, from #3309, the diff showed the following:

```
~ module.test.aws_eip.test-instance.0
    instance: "<INSTANCE ID>" => "${element(aws_instance.test-instance.*.id, count.index)}"
```

Then, when running `apply`, the diff would be realized and you would get
a diff mismatch error because it would realize the final value is the
same and remove it from the diff.

**The change:** `helper/schema` now marks unknown primitive values with
`NewComputed` set to true. Semantically this is correct for the diff to
have this information.

== 2. Terraform Diff.Same Changes

Next, the way Terraform compares diffs needed to be updated

Specifically, the case where the diff from the plan had a NewComputed
primitive and the diff from the apply _no longer has that value_. This
is possible if the computed value ended up being the same as the old
value. This is allowed to pass through.

Together, these fix #3309.
2016-10-25 22:36:59 -04:00
Mitchell Hashimoto 5ee8042dff
helper/schema: expose stop information as a Context 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto d1bb2f3487
helper/schema: support Stop() 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto 60140b28f4
Revert "Merge pull request #9536 from hashicorp/f-provider-stop"
This reverts commit c3a4cff133, reversing
changes made to 791a02e6e4.

This change requires plugin recompilation and we should hold off until a
minor release for that.
2016-10-25 12:00:36 -07:00
Mitchell Hashimoto 86eb30b8a2
helper/schema: expose stop information as a Context 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 8c11f137f5
helper/schema: support Stop() 2016-10-25 11:31:55 -07:00
James Nugent 43dd13cd36 helper/schema: Introduce Noop and RemoveFromState
This commit implements reusable functions for when resources have no
need to implement a particular operation:

- Noop - does nothing and returns no error.
- RemoveFromState - sets the resource ID to empty string (removing it
  from state) and returns no error.
2016-10-25 11:47:03 -05:00
Mitchell Hashimoto fa9758e162
helper/schema: test with DiffSuppress and Default 2016-10-24 22:23:13 -07:00
stack72 5a537cdbf9
helper/schema: Adding of MinItems as a validation to Lists and Maps
This is required for the times when the configuration cannot have an
empty configuration. An example would be in AzureRM, when you create a
LoadBalancer with a configuration, you can delete *all* but 1 of these
configurations
2016-10-04 18:57:58 +01:00
Paul Hinze 534a5a3b75 Merge pull request #8891 from gotascii/fix-diff-set-update
Unchanged NestedSets are not returned by DiffFieldReader.
2016-10-03 11:06:40 -05:00
Justin Marney 895d10a627 Use DeepEqual instead of multiple if statements to reduce noise. 2016-09-17 17:41:44 +00:00
Justin Marney 53e10dfa89 Name test appropriately. 2016-09-17 16:56:12 +00:00
Justin Marney 9217f5a063 Unchanged NestedSets are not returned by DiffFieldReader. 2016-09-16 23:57:33 +00:00
James Nugent e0226c9039 core: Check for attrV being nil before dereference
This can be an issue with unset computed fields.

Fixes #8815.
2016-09-14 09:51:15 +01:00
Paul Stack bf755bb5c9 Merge pull request #8585 from hashicorp/f-diff-suppression
helper/schema: Add diff suppression callback
2016-09-01 14:46:03 +01:00
James Nugent 85ec09111b helper/schema: Add diff suppression callback
This commit adds a new callback, DiffSuppressFunc, to  the schema.Schema
structure. If set for a given schema, a callback to the user-supplied
function will be made for each attribute for which the default
type-based diff mechanism produces an attribute diff. Returning `true`
from the callback will suppress the diff (i.e. pretend there was no
diff), and returning false will retain it as part of the plan.

There are a number of motivating examples for this - one of which is
included as an example:

1. On SSH public keys, trailing whitespace does not matter in many
   cases - and in some cases it is added by provider APIs. For
   digitalocean_ssh_key resources we previously had a StateFunc that
   trimmed the whitespace - we now have a DiffSuppressFunc which
   verifies whether the trimmed strings are equivalent.

2. IAM policy equivalence for AWS. A good proportion of AWS issues
   relate to IAM policies which have been "normalized" (used loosely)
   by the IAM API endpoints. This can make the JSON strings differ
   from those generated by iam_policy_document resources or template
   files, even though the semantics are the same (for example,
   reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3
   bucket policy. DiffSupressFunc can be used to test for semantic
   equivalence rather than pure text equivalence, but without having to
   deal with the complexity associated with a full "provider-land" diff
   implementation without helper/schema.
2016-08-31 19:13:53 -05:00
Radek Simko b90ab0b705
schema: Return all validation errors together from InternalValidate 2016-08-27 10:34:16 +01:00
Paul Hinze 3dccfa0cc9
terraform: diffs with only tainted set are non-empty
Fixes issue where a resource marked as tainted with no other attribute
diffs would never show up in the plan or apply as needing to be
replaced.

One unrelated test needed updating due to a quirk in the testDiffFn
logic - it adds a "type" field diff if the diff is non-Empty. NBD
2016-08-12 17:37:49 -05:00
Clint a84aa5e914 Revert "helper/schema: Make nested Set(s) in List(s) work" (#7436) 2016-06-30 10:48:52 -05:00
Radek Simko 37d57f4a85 Improve naming convention (resultSet -> exists) 2016-06-28 17:40:45 +01:00
Radek Simko 917ad44cf0 helper/schema: Fix readSet implementation (DiffFieldReader) 2016-06-28 17:40:44 +01:00
Radek Simko c738c5a9a3 helper/schema: Implement reader-specific readList method 2016-06-28 17:40:44 +01:00
Radek Simko 378b526dc3 helper/schema: Add regression tests for nested Set/List
Although DiffFieldReader was the one mostly responsible for a buggy behaviour
more tests were added throughout the debugging process most of which
would fail without the bugfix.

 - ResourceData
 - MultiLevelFieldReader
 - MapFieldReader
 - DiffFieldReader
2016-06-28 17:40:44 +01:00
James Nugent dbf725bd68 core: Allow dynamic attributes in helper/schema
The helper/schema framework for building providers previously validated
in all cases that each field being set in state was in the schema.
However, in order to support remote state in a usable fashion, the need
has arisen for the top level attributes of the resource to be created
dynamically. In order to still be able to use helper/schema, this commit
adds the capability to assign additional fields.

Though I do not forsee this being used by providers other than remote
state (and that eventually may move into Terraform Core rather than
being a provider), the usage and semantics are:

To opt into dynamic attributes, add a schema attribute named
"__has_dynamic_attributes", and make it an optional string with no
default value, in order that it does not appear in diffs:

        "__has_dynamic_attributes": {
            Type: schema.TypeString
            Optional: true
        }

In the read callback, use the d.UnsafeSetFieldRaw(key, value) function
to set the dynamic attributes.

Note that other fields in the schema _are_ copied into state, and that
the names of the schema fields cannot currently be used as dynamic
attribute names, as we check to ensure a value is not already set for a
given key.
2016-06-11 13:29:05 +01:00
clint shryock 7d71b8cc3c helper and terraform interpolate test update 2016-06-10 10:07:17 -05:00
James Bardin 2c7b702d1f Merge pull request #7091 from hashicorp/jbardin/serialize
Serialization for hash panics on TypeMap
2016-06-09 16:16:41 -04:00
James Bardin bab031aac5 Add test for TypeMap in a Schema 2016-06-09 16:00:33 -04:00
James Bardin d8fbaa7924 Serialization for hash panics on TypeMap
The serializeCollectionMemberForHash helper can't be called for the
MapType values, because MapType doesn't have a schema.Elem. Instead, we
can write the key/value pairs directly to the buffer. This still doesn't
allow for nested maps or lists, but we need to define that use case
before committing to it here.
2016-06-09 13:37:58 -04:00
James Nugent 074545e536 core: Use .% instead of .# for maps in state
The flatmapped representation of state prior to this commit encoded maps
and lists (and therefore by extension, sets) with a key corresponding to
the number of elements, or the unknown variable indicator under a .# key
and then individual items. For example, the list ["a", "b", "c"] would
have been encoded as:

    listname.# = 3
    listname.0 = "a"
    listname.1 = "b"
    listname.2 = "c"

And the map {"key1": "value1", "key2", "value2"} would have been encoded
as:

    mapname.# = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

Sets use the hash code as the key - for example a set with a (fictional)
hashcode calculation may look like:

    setname.# = 2
    setname.12312512 = "value1"
    setname.56345233 = "value2"

Prior to the work done to extend the type system, this was sufficient
since the internal representation of these was effectively the same.
However, following the separation of maps and lists into distinct
first-class types, this encoding presents a problem: given a state file,
it is impossible to tell the encoding of an empty list and an empty map
apart. This presents problems for the type checker during interpolation,
as many interpolation functions will operate on only one of these two
structures.

This commit therefore changes the representation in state of maps to use
a "%" as the key for the number of elements. Consequently the map above
will now be encoded as:

    mapname.% = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

This has the effect of an empty list (or set) now being encoded as:

    listname.# = 0

And an empty map now being encoded as:

    mapname.% = 0

Therefore we can eliminate some nasty guessing logic from the resource
variable supplier for interpolation, at the cost of having to migrate
state up front (to follow in a subsequent commit).

In order to reduce the number of potential situations in which resources
would be "forced new", we continue to accept "#" as the count key when
reading maps via helper/schema. There is no situation under which we can
allow "#" as an actual map key in any case, as it would not be
distinguishable from a list or set in state.
2016-06-09 10:49:42 +01:00
James Nugent 91587a49f3 core: Remove unnecessary debug logging
Some unnecessary debug logging was introduced in 7b6df27e4, this commit
removes it so as not to clutter logs.
2016-06-08 18:38:41 +01:00
Chris Marchesi 9d7fb89114 core: Adding Sensitive attribute to resource schema
This an effort to address hashicorp/terraform#516.

Adding the Sensitive attribute to the resource schema, opening up the
ability for resource maintainers to mark some fields as sensitive.
Sensitive fields are hidden in the output, and, possibly in the future,
could be encrypted.
2016-05-29 22:18:44 -07:00
Sander van Harmelen 8560f50cbc
Change taint behaviour to act as a normal resource
This means it’s shown correctly in a plan and takes into account any
actions that are dependant on the tainted resource and, vice verse, any
actions that the tainted resource depends on.

So this changes the behaviour from saying this resource is tainted so
just forget about it and make sure it gets deleted in the background,
to saying I want that resource to be recreated (taking into account the
existing resource and it’s place in the graph).
2016-05-26 19:55:26 -05:00
Mitchell Hashimoto 55583baa7e
Merge branch 'f-aws-import' 2016-05-18 15:28:12 -06:00
Mitchell Hashimoto b7d4767dd6
helper/schema: pass through import state func 2016-05-16 10:03:57 -07:00
Martin Atkins 6cd22a4c9a helper/schema: emit warning when using data source resource shim
For backward compatibility we will continue to support using the data
sources that were formerly logical resources as resources for the moment,
but we want to warn the user about it since this support is likely to
be removed in future.

This is done by adding a new "deprecation message" feature to
schema.Resource, but for the moment this is done as an internal feature
(not usable directly by plugins) so that we can collect additional
use-cases and design a more general interface before creating a
compatibility constraint.
2016-05-14 08:26:36 -07:00
Martin Atkins fb262d0dbe helper/schema: shim for making data sources act like resources
Historically we've had some "read-only" and "logical" resources. With the
addition of the data source concept these will gradually become data
sources, but we need to retain backward compatibility with existing
configurations that use the now-deprecated resources.

This shim is intended to allow us to easily create a resource from a
data source implementation. It adjusts the schema as needed and adds
stub Create and Delete implementations.

This would ideally also produce a deprecation warning whenever such a
shimmed resource is used, but the schema system doesn't currently have
a mechanism for resource-specific validation, so that remains just a TODO
for the moment.
2016-05-14 08:26:36 -07:00
Martin Atkins 6a468dcd83 helper/schema: Resource can be writable or not
In the "schema" layer a Resource is just any "thing" that has a schema
and supports some or all of the CRUD operations. Data sources introduce
a new use of Resource to represent read-only resources, which require
some different InternalValidate logic.
2016-05-14 08:26:36 -07:00
Martin Atkins 0e0e3d73af core: New ResourceProvider methods for data resources
This is a breaking change to the ResourceProvider interface that adds the
new operations relating to data sources.

DataSources, ValidateDataSource, ReadDataDiff and ReadDataApply are the
data source equivalents of Resources, Validate, Diff and Apply (respectively)
for managed resources.

The diff/apply model seems at first glance a rather strange workflow for
read-only resources, but implementing data resources in this way allows them
to fit cleanly into the standard plan/apply lifecycle in cases where the
configuration contains computed arguments and thus the read must be deferred
until apply time.

Along with breaking the interface, we also fix up the plugin client/server
and helper/schema implementations of it, which are all of the callers
used when provider plugins use helper/schema. This would be a breaking
change for any provider plugin that directly implements the provider
interface, but no known plugins do this and it is not recommended.

At the helper/schema layer the implementer sees ReadDataApply as a "Read",
as opposed to "Create" or "Update" as in the managed resource Apply
implementation. The planning mechanics are handled entirely within
helper/schema, so that complexity is hidden from the provider implementation
itself.
2016-05-14 08:26:36 -07:00
Mitchell Hashimoto b728e55861
helper/schema: Resource.Data should set latest schema version 2016-05-11 13:02:36 -07:00
Mitchell Hashimoto c02c6c3f9c
helper/schema: default state func for import 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 2bb814e3de
helper/schema: adapt to ID being arg to ImportState 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 03931bfda9
helper/schema: ImportState must set ID on the resource data 2016-05-11 13:02:34 -07:00
Mitchell Hashimoto 445194ebdf
helper/schema: test ImportState 2016-05-11 13:02:33 -07:00
Mitchell Hashimoto 1685054a9a
helper/schema: cleaner way to store Ephemeral 2016-05-11 13:02:31 -07:00
Mitchell Hashimoto 84531a3fd5
helper/schema: sets Importable to true for resources that have importer 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 19609bde0e
helper/schema: can specify Importer on Resource, InternalValidate 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto b8121ea63e
helper/schema: Resource.Data to return a ResourceData for a Resource 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 4e9877179c
helper/schema: start the resource importer 2016-05-11 13:02:29 -07:00
James Nugent 7b6df27e4a helper/schema: Read native maps from configuration
This adds a test and the support necessary to read from native maps
passed as variables via interpolation - for example:

```
resource ...... {
     mapValue = "${var.map}"
}
```

We also add support for interpolating maps from the flat-mapped resource
config, which is necessary to support assignment of computed maps, which
is now valid.

Unfortunately there is no good way to distinguish between a list and a
map in the flatmap. In lieu of changing that representation (which is
risky), we assume that if all the keys are numeric, this is intended to
be a list, and if not it is intended to be a map. This does preclude
maps which have purely numeric keys, which should be noted as a
backwards compatibility concern.
2016-05-10 14:49:14 -04:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
Paul Hinze b4df304b47
helper/schema: Normalize bools to "true"/"false" in diffs
For a long time now, the diff logic has relied on the behavior of
`mapstructure.WeakDecode` to determine how various primitives are
converted into strings.  The `schema.DiffString` function is used for
all primitive field types: TypeBool, TypeInt, TypeFloat, and TypeString.

The `mapstructure` library's string representation of booleans is "0"
and "1", which differs from `strconv.FormatBool`'s "false" and "true"
(which is used in writing out boolean fields to the state).

Because of this difference, diffs have long had the potential for
cosmetically odd but semantically neutral output like:

    "true" => "1"
    "false" => "0"

So long as `mapstructure.Decode` or `strconv.ParseBool` are used to
interpret these strings, there's no functional problem.

We had our first clear functional problem with #6005 and friends, where
users noticed diffs like the above showing up unexpectedly and causing
troubles when `ignore_changes` was in play.

This particular bug occurs down in Terraform core's EvalIgnoreChanges.
There, the diff is modified to account for ignored attributes, and
special logic attempts to handle properly the situation where the
ignored attribute was going to trigger a resource replacement. That
logic relies on the string representations of the Old and New fields in
the diff to be the same so that it filters properly.

So therefore, we now get a bug when a diff includes `Old: "0", New:
"false"` since the strings do not match, and `ignore_changes` is not
properly handled.

Here, we introduce `TypeBool`-specific normalizing into `finalizeDiff`.
I spiked out a full `diffBool` function, but figuring out which pieces
of `diffString` to duplicate there got hairy. This seemed like a simpler
and more direct solution.

Fixes #6005 (and potentially others!)
2016-05-05 09:00:58 -05:00
Martin Atkins c1ce8ff31a Merge pull request #5218 from paybyphone/paybyphone_set_maxitems
Add MaxItems attribute to Schema
2016-03-01 09:27:59 -08:00
Radek Simko 8bdd92187c Merge pull request #4446 from TimeIncOSS/f-schema-new-resource
helper/schema: Allow identification of a new resource in update func
2016-02-29 20:07:00 +00:00
Chris Marchesi 8c5354b7dc Add MaxItems attribute to Schema
* MaxItems defines a maximum amount of items that can exist within a
   TypeSet or TypeList. Specific use cases would be if a TypeSet is being
   used to wrap a complex structure, however more than one instance would
   cause instability.
2016-02-23 16:41:32 -08:00
Trevor Pounds 0cd0ff0f8e Use built-in schema.HashString. 2016-02-07 16:29:34 -08:00
Mitchell Hashimoto 4576eaa966 helper/schema: replace config/lang 2016-02-03 13:24:04 -05:00
Paul Hinze da872eee66 Merge pull request #4864 from hashicorp/phinze/aws-min-elb-cap-regression
aws: undeprecate min_elb_capacity; restore min capacity waiting
2016-01-27 14:17:10 -06:00
Paul Hinze c70eab6500 aws: undeprecate min_elb_capacity; restore min capacity waiting
It was a mistake to switched fully to `==` when activating waiting for
capacity on updates in #3947. Users that didn't set `min_elb_capacity ==
desired_capacity` and instead treated it as an actual "minimum" would
see timeouts for every create, since their target numbers would never be
reached exactly.

Here, we fix that regression by restoring the minimum waiting behavior
during creates.

In order to preserve all the stated behavior, I had to split out
different criteria for create and update, criteria which are now
exhaustively unit tested.

The set of fields that affect capacity waiting behavior has become a bit
of a mess. Next major release I'd like to rework all of these into a
more consistently named block of config. For now, just getting the
behavior correct and documented.

(Also removes all the fixed names from the ASG tests as I was hitting
collision issues running them over here.)

Fixes #4792
2016-01-27 13:30:44 -06:00
Paul Hinze 069425a700 consul: Fix several problems w/ consul_keys update
Implementation notes:

 * The hash implementation was not considering key value, causing "diffs
   did not match" errors when a value was updated. Switching to default
   HashResource implementation fixes this
 * Using HashResource as a default exposed a bug in helper/schema that
   needed to be fixed so the Set function is picked up properly during
   Read
 * Stop writing back values into the `key` attribute; it triggers extra
   diffs when `default` is used. Computed values all just go into `var`.
 * Includes a state migration to prevent unnecessary diffs based on
   "key" attribute hashcodes changing.

In the tests:

 * Switch from leaning on the public demo Consul instance to requiring a
   CONSUL_HTTP_ADDR variable be set pointing to a `consul agent -dev`
   instance to be used only for testing.
 * Add a test that exposes the updating issues and covers the fixes

Fixes #774
Fixes #1866
Fixes #3023
2016-01-26 14:46:26 -06:00
Radek Simko 4c6ceef9b8 helper/schema: Allow identification of a new resource in update func 2015-12-27 14:01:03 +01:00
Paul Hinze edaf5795a5 Merge pull request #3257 from fatih/fix-nil-setting-schema
schema: delete non existing values
2015-12-08 20:15:00 -06:00
Paul Hinze 99244c5597 helper/schema: skip provider input for deprecated fields
There's no reason that a field that's been deprecated should ever
prompt.

fixes #4033
2015-12-07 11:28:45 -06:00
Paul Hinze f1e7cec566 Merge pull request #3992 from svanharmelen/f-change-sets
core: change set internals and make (extreme) performance improvements
2015-12-04 09:03:43 -06:00
Sander van Harmelen ef4726bd50 Change Set internals and make (extreme) performance improvements
Changing the Set internals makes a lot of sense as it saves doing
conversions in multiple places and gives a central place to alter
the key when a item is computed.

This will have no side effects other then that the ordering is now
based on strings instead on integers, so the order will be different.
This will however have no effect on existing configs as these will
use the individual codes/keys and not the ordering to determine if
there is a diff or not.

Lastly (but I think also most importantly) there is a fix in this PR
that makes diffing sets extremely more performand. Before a full diff
required reading the complete Set for every single parameter/attribute
you wanted to diff, while now it only gets that specific parameter.

We have a use case where we have a Set that has 18 parameters and the
set consist of about 600 items (don't ask 😉). So when doing a diff
it would take 100% CPU of all cores and stay that way for almost an
hour before being able to complete the diff.

Debugging this we learned that for retrieving every single parameter
it made over 52.000 calls to `func (c *ResourceConfig) get(..)`. In
this function a slice is created and used only for the duration of the
call, so the time needed to create all needed slices and on the other
hand the time the garbage collector needed to clean them up again caused
the system to cripple itself. Next to that there are also some expensive
reflect calls in this function which also claimed a fair amount of CPU
time.

After this fix the number of calls needed to get a single parameter
dropped from 52.000+ to only 2! 😃
2015-11-22 14:21:28 +01:00
Paul Hinze c7dc1c10a3 helper/schema: skip StateFunc when value is nil
This takes the nil checking burden off of StateFunc.

fixes #3586, see that issue for further discussion
2015-11-20 14:07:18 -06:00
Paul Hinze 938281024f helper/schema: name test cases w/ strings
I promised myself that next time I jumped in this file I'd fix this up.
Now we don't have to manually index the file with comments, we can just
add descriptive names to the test cases!
2015-11-20 13:51:34 -06:00
James Nugent f4c03ec2a6 Reflect new comment format in stringer.go
As of November 8th 2015, (4b07c5ce8a), the word "Code" is prepended to
the comments in Go source files generated by the stringer utility.
2015-11-09 11:38:51 -05:00
Martin Atkins a67182543c Nicer error when list/map assigned to string argument.
Previous this would return the following sort of error:
expected type 'string', got unconvertible type '[]interface {}'

This is the raw error returned by the underlying mapstructure library.
This is not a helpful error message for anyone who doesn't know Go's
type system, and it exposes Terraform's internals to the UI.

Instead we'll catch these cases before we try to use mapstructure and
return a more straightforward message.

By checking the type before the IsComputed exception this also avoids
a crash caused when the assigned value is a computed list. Otherwise
the list of interpolations is allowed through here and then crashes later
during Diff when the value is not a primitive as expected.
2015-10-22 21:16:02 -07:00
Paul Hinze 2a179d1065 helper/schema: ValidateFunc support for maps 2015-10-14 15:10:22 -05:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Martin Atkins cc8e8a55de helper/schema: Default hashing function for sets
A common issue with new resource implementations is not considering parts
of a complex structure that's used inside a set, which causes quirky
behavior.

The schema helper has enough information to provide a default reasonable
implementation of a set function that includes all non-computed attributes
in a deterministic way. Here we implement such a function and use it
when no explicit hashing function is provided.

In order to achieve this we encapsulate the construction of the zero
value for a schema in a new method schema.ZeroValue, which allows us to
put the fallback logic to the new default function in a single spot.
It is no longer valid to use &Set{F: schema.Set} and all uses of that
construct should be replaced with schema.ZeroValue().(*Set) .
2015-10-03 18:10:47 -07:00
Radek Simko 641b701830 schema: Make validation more strict 2015-10-03 14:29:19 -07:00
Fatih Arslan f269d4fc8c schema: add test for nil string case 2015-09-16 23:35:10 +03:00
Fatih Arslan 8e7fc240f9 schema: delete non existing values
We need to set the value to an empty value so the state file does
indeed change the value. Otherwise the obsolote value is still
intact and doesn't get changed at all. This means `terraform show`
still shows the obsolote value when the particular value is not
existing anymore. This is due the AWS API which is returning a null
instead of an empty string.
2015-09-16 23:26:27 +03:00
Anthony Scalisi 198e1a5186 remove various typos 2015-09-11 11:56:20 -07:00
Paul Hinze 7eb72e7a12 helper/schema: record schema version when destroy fails
This was just a missed exit from the resource.Apply function -
subsequent refreshes would add the SchemaVersion back into the state,
but having the state recorded once without the meta information can
cause problems with Atlas's remote state checksumming.
2015-08-03 15:53:15 -05:00
Sander van Harmelen 4a8ef78d33 Fixes #2676 by prefixing all Windows commands
By prefixing them with `cmd /c` it will work with both `winner` and
`ssh` connection types.

This PR also reverts some bad stringer changes made in PR #2673
2015-07-10 12:56:27 +02:00
Sander van Harmelen 97fd4f5b7d Tweaking the tests 2015-07-09 21:29:27 +02:00
Paul Hinze 5c38456b05 core: don't prompt for variables with defaults
In `helper/schema` we already makes a distinction between `Default`
which is always applied and `InputDefault` which is displayed to the
user for an empty field.

But for variables we just have `Default` which is treated like
`InputDefault`. This changes it to _not_ prompt the user for a value
when the variable declaration includes a default.

Treating this as a UX bugfix and the "don't prompt for variables w/
defaults set" behavior as the originally expected behavior we were
failing to honor.

Added an already-passing test to verify and cover the `helper/schema`
behavior.

Perhaps down the road we can add a `input_default` attribute to
variables to allow similar behavior to `helper/schema` in variables, but
for now just sticking with the fix.

Fixes #2592
2015-07-02 10:40:30 -05:00
Mitchell Hashimoto 2f08a2bb15 Merge pull request #2507 from hashicorp/b-set-remove
helper/schema: diff should include removed set items [GH-1823]
2015-06-26 08:18:28 -07:00
Mitchell Hashimoto 0100d4139b helper/schema: clean up style 2015-06-25 22:01:54 -07:00
Mitchell Hashimoto 6e509aedcb helper/schema: diff should include removed set items [GH-1823] 2015-06-25 21:52:49 -07:00
Paul Hinze 7238b3b4af core: encapsulate representation of StringList
Now the only code that cares about how StringLists are represented lives
inside string_list.go

...which gives us the ability to change it! :)
2015-06-25 17:55:57 -05:00
Paul Hinze 10b3abf405 config: introduce StringList to abstract over list hack
This is the initial pure "all tests passing without a diff" stage. The
plan is to change the internal representation of StringList to include a
suffix delimiter, which will allow us to recognize empty and
single-element lists.
2015-06-25 17:55:56 -05:00
Radek Simko 6fdbca8e58 Merge pull request #2466 from TimeIncOSS/f-schema-field-name-validate
schema: Add field name to ValidateFunc
2015-06-24 18:52:53 +01:00
Mitchell Hashimoto 6b7c2bcb35 Merge pull request #2450 from hashicorp/b-schema-validate-type
helper/schema: validate objects are objects [GH-2166]
2015-06-24 10:35:26 -07:00
Mitchell Hashimoto 630646335f Merge pull request #2451 from hashicorp/b-provider-validate
helper/schema: internal validate as part of provider validation
2015-06-24 10:28:49 -07:00