Commit Graph

452 Commits

Author SHA1 Message Date
Chris Marchesi f7e42728b6 helper/schema: Remove unused Destroy check for CustomizeDiff
Both Destroy and DestroyDeposed are not propagated down the diff stack,
meaning that there is no way we can tell at this point if an instance is
being destroyed or deposed, so this check would never be used.

In this regard, Destroy never runs a diff down the stack at all, and a
deposition check is not run until *after* the provider's diff function
is called. To answer this question and close it off, we could either
determine if a resource is deposed earlier, and propagate that down, or
treat deposed resources like full destroy nodes, and not diff them at
all (but rather making a diff with the only thing in it being
DestroyDeposed flagged).
2017-11-01 14:25:32 -07:00
Chris Marchesi fa1fc2ca8e helper/schema: CustomizeDiff invocation test
Just one more test to check to make sure that CustomizeDiff is called
on resource level.
2017-11-01 14:25:32 -07:00
Chris Marchesi ee769188d7 helper/schema: More CustomizeDiff test cases
Added a few more test cases for CustomizeDiff, caught another error in
the process. I think this is ready for review now, and possibly some
real-world testing of the waters by way of porting some resources that
would benefit from the feature.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8af9610b87 helper/schema: Hook CustomizeDiffFunc into diff logic
It's alive! CustomizeDiff logic now has been inserted into the diff
process. The test_resource_with_custom_diff resource provides some basic
testing and a reference implementation.

There should now be plenty of test coverage for this feature via the
tests added for ResourceDiff, and the basic test added to the
schemaMap.Diff test, and the test resource, but more can be added to
test any specific case that comes up otherwise.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8126ee8401 helper/schema: Fix supplied config to tests 2017-11-01 14:25:32 -07:00
Chris Marchesi 196d7e63fe helper/schema: Drop ClearAll from ResourceDiff
As the interface has been specced out for ResourceDiff, the only diff
operation that can function on a non-computed key is ForceNew, and that
is only if a diff exists for that key. As such, allowing the user to
clear keys that cannot be operated on afterwards does not really make
much sense.

Will re-add this function if it is determined it's needed after all on
review.
2017-11-01 14:25:32 -07:00
Chris Marchesi 22220fd0f7 helper/schema: Final set of ResourceDiff tests, bug fixes
Final set of coverage for ResourceDiff and bug fixes to correct issues
the test cases brought up.

Will be dropping ClearAll in next commit, as with how this interface is
intended to be used, it does not really make much sense.
2017-11-01 14:25:32 -07:00
Chris Marchesi a5fc664ea6 helper/schema: Add schemaMap.DeepCopy
This provides a deep copy of a schemaMap, which will be needed for the
diff customization process as ResourceDiff will be able to flag fields
as ForceNew - we don't want to affect the source schema.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6a4f7b0dce helper/schema: Don't ignore diffs for certain NewComputed keys
In diffString, removed values are marked if the old value is non-nil and
the new value is nil, regardless of if the new value is marked as a
computed value. With the new diff override behaviour, this can lead to
issues where a nil attribute may get inserted into the diff if the new
value has been marked as computed (as the value will be marked as
missing, but computed).

This propagates into finalizeDiff in some respects because even if
NewComputed is manually set in the diff that gets passed in, it is
ignored, and the schema becomes the only source of truth. Since our new
diff behaviour is mainly designed to be supported on computed keys only,
it stands to reason that we there will be a time where we want to set a
diff as NewComputed on a key, and see that change in the diff.

These two small changes makes that happen. No regressions in tests have
been observed via this change, so it seems safe and non-invasive.
2017-11-01 14:25:32 -07:00
Chris Marchesi 64cc4084d2 helper/schema: ResourceDiff tests, bug fixes
The beginning of tests for SetNew. Noticed that removed values are not
logged for computed keys in a diff - not too sure if that is going to be
a problem (possibly not as GetChange in ResourceData should still
reference state for the old value). I came on this most notably in the
"TestSetNew/complex-ish_set_diff" test, for reference.

More tests pending for other functions for coverage of other functions
and test cases as defined in #8769.
2017-11-01 14:25:32 -07:00
Chris Marchesi f5f4e0329f helper/schema: Track updated keys in ResourceDiff
This ensures that when we hook this into the main diff logic, that we
can just re-diff the keys that are modified, ensuring that the diff is
not re-run on keys that were not touched, or on keys that were
intentionally removed.
2017-11-01 14:25:32 -07:00
Chris Marchesi aeb793f968 helper/schema: Add Clear function to ResourceDiff
This should complete the feature set of the prototype. This function
removes a specific key from the existing diff, preventing conflicts.

Further functionality might be needed to make this behave as expected,
namely ensuring that we don't re-process the whole diff after the
CustomizeDiffFunc runs.
2017-11-01 14:25:32 -07:00
Chris Marchesi 1e8cfc52e9 helper/schema: Updated ResourceDiff prototype
This new prototype removes the dependence on a underlying ResourceData,
moving several items up to ensure an approach that more matches
ResourceData. Namely, we create our own MultiLevelFieldReader that
also layers updated diff values on top. New values use a small
re-implementation of the MapFieldWriter/Reader that allows computed
values to be set.

The assumption here now is that a second diff will happen after the
first one is done, processing the new values set in the 2 new
reader/writer levels and updating the diff as necessary.
2017-11-01 14:25:32 -07:00
Chris Marchesi b99c615ee6 helper/schema: New ResourceDiff object
This adds a new object, ResourceDiff, to the schema package. This
object, in conjunction with a function defined in CustomizeDiff in the
resource schema, allows for the in-flight customization of a Terraform
diff. This helps support use cases such as when there are necessary
changes to a resource that cannot be detected in config, such as via
computed fields (most of the utility in this object works on computed
fields only). It also allows for a wholesale wipe of the diff to allow
for diff logic completely offloaded to an external API, if it is a
better use case for a specific provider.

As part of this work, many internal diff functions have been moved to
use a special resourceDiffer interface, to allow for shared
functionality between ResourceDiff and ResourceData. This may be
extended to the DiffSuppressFunc as well which would restrict use of
ResourceData in DiffSuppressFunc to generally read-only fields.

This work is not yet in its final state - CustomizeDiff is not yet
implemented in the general diff workflow, new functions may be added
(notably Clear() for a single key), and functionality may be altered.
Tests will follow as well.
2017-11-01 14:25:32 -07:00
Dana Hoffman a7e308fc2c helper/schema: Schema.Elem documentation for TypeMap
Updated it to state that Elem is valid for TypeMaps too, and that TypeSets can use a Resource as an Elem.
2017-10-26 10:29:47 -07:00
Radek Simko 7fceccfbf7
helper/schema: Loosen validation for 'id' field 2017-10-26 09:37:38 +01:00
Martin Atkins 183833affc core: terraform.ResourceProvider.GetSchema method
In order to parse provider, resource and data source configuration from
HCL2 config files, we need to know the relevant configuration schema.
This new method allows Terraform Core to request these from a provider.

This is a breaking change to this interface, so all of its implementers
in this package are updated too. This includes concrete implementations
of the new method in helper/schema that use the schema conversion code
added in an earlier commit to produce a configschema.Block automatically.

Plugins compiled against prior versions of helper/schema will not have
support for this method, and so calls to them will fail. Callers of
this new method will therefore need to sniff for support using the
SchemaAvailable field added to both ResourceType and DataSource.

This careful handling will need to persist until next time we increment
the plugin protocol version, at which point we can make the breaking
change of requiring this information to be available.
2017-10-17 07:23:41 -07:00
Martin Atkins b91bd62747 config/configschema: Sensitive flag for attributes
We don't currently have any need for this information, but we're
propagating it out of helper/schema here pre-emptively so that once we
later have a use for it we will not need to rebuild the providers to gain
access to it.

The long-term expected use-case for this is to have Terraform Core use
static analysis techniques to trace the path of sensitive data through
interpolations so that intermediate results can be flagged as sensitive
too, but we have a lot more work to do before such a thing would actually
be possible.
2017-10-04 16:35:11 -07:00
Martin Atkins 69650b0bbc helper/schema: conversion of Schema to configschema.Block
As part of moving to the next-generation HCL implementation,
Terraform Core is getting its own representation of configuration schema
that is tailored for configuration-processing use-cases. The capabilities
of this are a subset of the helper/schema model primarily concerned with
the configuration structure and value types, leaving detailed validation
and defaults for helper/schema to still solve.

These new methods allow mechanical creation of a schema in the new Core
schema model from a schema expressed in the helper/schema model. This is
not yet used as of this commit, but will be used later to implement some
new ResourceProvider methods that will allow core to obtain the schema
for provider, resource and data source configuration while remaining
source-compatible with existing provider implementations.
2017-10-04 16:35:11 -07:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Chris Marchesi 287a5eb66a
helper/schema: More tests for Set.HashEqual
Just to make sure equality on outer and inner values does not affect
anything.
2017-08-15 21:56:01 -07:00
Chris Marchesi ca42980e49
helper/schema: Add Set.HashEqual
Equality of schema.Sets gets tricky when dealing with nested sets -
Set.Equal only superficially compares the underlying maps and hence any
sets nested under the root sets cause issues.

This adds a simple method, HashEqual, that does a top-level hash
comparison, helping to work around this without any complex re-invention
of things like reflect.DeepEqual.

Of course, in order to make effective use of this function, the user
needs to make sure they are properly hashing their nested sets, however
this is trivial with things like HashResource.
2017-08-15 21:50:52 -07:00
Jake Champlin 3600f0b730 Merge pull request #15723 from hashicorp/f-get-boolean-non-default
core: Add `GetOkExists` schema function
2017-08-03 18:06:17 -04:00
Jake Champlin d969f97e73
update tests 2017-08-03 17:53:07 -04:00
Jake Champlin fa272e8c9c
Add more specific exists tests 2017-08-03 14:14:39 -04:00
Jake Champlin 268138dbd4
Rename to GetOkExists 2017-08-03 12:05:19 -04:00
Jake Champlin 270bbdb19b
core: Add `GetOkRaw` schema function
Adds `GetOkRaw` as a schema function. This should only be used to verify
boolean attributes are either set or not set, regardless of their zero
value for their type. There are a few small use cases outside of the boolean
type where this will be helpful as well.

Overall, this shouldn't detract from the zero-value checks that `GetOK()`
currently has, and should only be used when absolutely needed. However,
there are enough use-cases for this addition without checking for the
zero-value of the type, that this is needed.

Primary use case is for a boolean attribute that is `Optional` and `Computed`,
without a default value. There's currently no way to verify that the boolean
attribute was explicitly set to the zero-value literal with the current
`GetOk()` function. This new function allows for that check, keeping the
`Computed` check for the returned `exists` boolean.

```
$ make test TEST=./helper/schema TESTARGS="-run=TestResourceDataGetOkRaw"
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/08/02 11:17:32 Generated command/internal_plugin_list.go
go test -i ./helper/schema || exit 1
echo ./helper/schema | \
        xargs -t -n4 go test -run=TestResourceDataGetOkRaw -timeout=60s -parallel=4
go test -run=TestResourceDataGetOkRaw -timeout=60s -parallel=4 ./helper/schema
ok      github.com/hashicorp/terraform/helper/schema    0.005s
```
2017-08-02 11:18:59 -04:00
James Bardin ca68723c91 memoize DiffFieldReader.ReadField
The field reader code path is extremely inefficient, but refactoring
it all is much to invasive a change at the moment.

Have DiffFieldReader internally cache results for ReadField.
2017-07-28 14:26:36 -04:00
Radek Simko f979b8feef Enforce field names to be alphanum lowercase + underscores (#15562) 2017-07-17 08:37:46 +01:00
Radek Simko 07cbd54fbc Actively disallow reserved field names in schema (#15522) 2017-07-10 21:51:55 -07:00
joel.ringuette ffddf96603 Fix issue with reading timeouts on Delete
Original fix by @jringuette but I couldn't get his patch to apply after
the repo split :/
2017-06-23 09:15:29 -05:00
clint shryock 11dc0c5fdf Regression test for #767 (https://github.com/hashicorp/terraform/issues/14444) 2017-06-23 09:15:29 -05:00
Sander van Harmelen 7e180aec92 Refactor the provisioner validation function (#15273)
It turns out that `d.GetOk` also returns `false` when the user _did_ actually supply a value for it in the config, but the value itself needs to be evaluated before it can be used.

So instead of passing a `ResourceData` we now pass a `ResourceConfig`
which makes much more sense for doing the validation anyway.
2017-06-15 19:57:04 +02:00
Radek Simko 1244309579 Fix stringer comments (#15069) 2017-06-05 10:17:35 +01:00
He Guimin 87562be855 provider/alicloud: Add the function of replacing ecs instance's system disk (#15048)
* add replacing system disk function for ecs

* remove ForceNew of system_disk_size
2017-06-05 11:27:49 +03:00
Jake Champlin ac177492fb
core: Revert stringer changes from earlier commits 2017-06-01 11:37:12 -04:00
Thomas Schaaf 79c91e11c8 provider/aws: Add aws elastic beanstalk solution stack (#14944)
* Add aws elastic beanstalk solution stack

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Fix incorrect naming

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Use unique go variable/function names

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MacBook-Pro.local>

* Add docs to sidebar

* Sort provider by alphabet

* Fix indent

* Add required statement

* Fix acceptance test
2017-06-01 02:23:06 +03:00
Jake Champlin 7894478c8c Merge pull request #14681 from svanharmelen/f-review
Use `helpers.shema.Provisoner` in Chef provisioner V2
2017-05-30 14:26:51 -04:00
James Bardin 6bc52be0a5 check for IsComputed when validating a schema obj
GH-14784 allowed nested structures to be validate, rather than relying
on the raw value. However this still returns the same validation error
if the structures contain a computed value, since Get will return the
raw string in that case.

This simply skips the validation in the IsComputed case, since there's
nothing that can be checked.
2017-05-24 12:34:53 -04:00
Matt Robenolt b9a3433f6b helper/schema: fix validating nested objects
When interpreting a nested object, we were validating against the "raw"
value, and not the interpolated value, causing incorrect errors.

This affects structures such as:

```tf
tags = "${list(map("foo", "bar"))}"
```

Prior to this, a complaint about "expected object, got string" since the
raw value is obviously a string, when the interpolated value is the
correct shape.
2017-05-24 01:57:13 -07:00
Sander van Harmelen 0e422737ba Fix and refactor the Chef provisioner
The tests did pass, but that was because they only tested part of the changes. By using the `schema.TestResourceDataRaw` function the schema and config are better tested and so they pointed out a problem with the schema of the Chef provisioner.

The `Elem` fields did not have a `*schema.Schema` but a `schema.Schema` and in an `Elem` schema only the `Type` field may (and must) be set. Any other fields like `Optional` are not allowed here.

Next to fixing that problem I also did a little refactoring and cleaning up. Mainly making the `ProvisionerS` private (`provisioner`) and removing the deprecated fields.
2017-05-19 21:05:21 +02:00
Vladislav Rassokhin f5449a62e0 Various built-in provisioners improvements:
1. Migrate `chef` provisioner to `schema.Provisioner`:

 * `chef.Provisioner` structure was renamed to `ProvisionerS`and  now it's decoded from `schema.ResourceData` instead of `terraform.ResourceConfig` using simple copy-paste-based solution;
 * Added simple schema without any validation yet.

 2. Support `ValidateFunc` validate function : implemented in `file` and `chef` provisioners.
2017-05-19 20:43:51 +02:00
Radek Simko 44a99e0ae5 core: Avoid crash on empty TypeSet blocks (#14305) 2017-05-09 20:45:53 +02:00
Radek Simko 9867ce4dde
helper/schema: Disallow validation+diff suppression on computed-only fields 2017-04-23 12:25:40 +02:00
James Bardin 9aaf220efb Merge pull request #13863 from hashicorp/jbardin/computed-list-requires-new
make sure a computed list is can be RequiresNew
2017-04-21 18:02:14 -04:00
James Bardin caadb4297f make sure a computed list is can be RequiresNew
If a schema.TypeList had a Schema with ForceNew, and if that list was
NewComputed, the diff would not have RequiresNew set. This causes apply
to fail when the diffs didn't match because of the change to
RequiresNew.

Set the RequiresNew field on the list's ResourceAttrDiff based on the
Schema value.
2017-04-21 17:51:15 -04:00
Martin Atkins b1763e262a Restore stringer-generated files back to new version
stringer has changed the boilerplate it generates in a recent version.
We'd previously updated to the new format but accientally rolled back
to the old while merging a long-running feature branch.

This restores us back to the new format again.
2017-04-21 14:49:18 -07:00
Jasmin Gacic 61499cfcf0 Provider Oneandone (#13633)
* Terraform Provider 1&1

* Addressing pull request remarks

* Fixed imports

* Fixing remarks

* Test optimiziation
2017-04-21 17:19:10 +03:00
Sander van Harmelen 3d0073e05c core: fix a crash by suggesting a different approach to solve #11170 (#13541)
* Revert #11245, #11321, #11498 and #11757

These PR’s are all related to issue #11170 for which I would like to propose a different solution then the one currently implemented.

* A different approach to solve #11170

This approach has (IMHO) a few advantages with regards to the solution currently implemented. I will elaborate on this in the PR.
2017-04-14 22:32:30 +02:00
James Bardin 735dfc416d Merge pull request #13427 from hashicorp/jbardin/context-keys
Fix context key types in schema
2017-04-10 15:38:27 -04:00