Commit Graph

23391 Commits

Author SHA1 Message Date
Martin Atkins e3618f915b backend/local: Fix mock provider in tests
We've changed the contract for PlanResourceChange to now require the
provider to populate any default values (including unknowns) it wants to
set for computed arguments, so our mock provider here now needs to be a
little more complex to deal with that.

This fixes several of the tests in this package. A minor change to
TestLocal_applyEmptyDirDestroy was required to make it properly configure
the mock provider so PlanResourceChange can access the schema.
2019-02-08 12:48:32 -08:00
Martin Atkins 7e186f72d9 command: Update "terraform show -json" tests for changed provider contract
We now require a provider to populate all of its defaults -- including
unknown value placeholders -- during PlanResourceChange. That means the
mock provider for testing "terraform show -json" must now manage the
population of the computed "id" attribute during plan.

To make this logic a little easier, we also change the ApplyResourceChange
implementation to fill in a non-null id, since that makes it easier for
the mock PlanResourceChange to recognize when it needs to populate that
default value during an update.
2019-02-08 11:58:21 -08:00
James Bardin 82588af892 switch blocks based on value type, and check attrs
Check attributes on null objects, and fill in unknowns. If we're
evaluating the object, it either means we are at the top level, or a
NestingSingle block was present, and in either case we need to treat the
attributes as null rather than the entire object.

Switch on the block types rather than Nesting, so we don't need add any
logic to change between List/Tuple or Map/Object when DynamicPseudoType
is involved.
2019-02-08 14:46:29 -05:00
Martin Atkins 312d798a89 core: Restore our EvalReadData behavior
In an earlier commit we changed objchange.ProposedNewObject so that the
task of populating unknown values for attributes not known during apply
is the responsibility of the provider's PlanResourceChange method, rather
than being handled automatically.

However, we were also using objchange.ProposedNewObject to construct the
placeholder new object for a deferred data resource read, and so we
inadvertently broke that deferral behavior. Here we restore the old
behavior by introducing a new function objchange.PlannedDataResourceObject
which is a specialized version of objchange.ProposedNewObject that
includes the forced behavior of populating unknown values, because the
provider gets no opportunity to customize a deferred read.

TestContext2Plan_createBeforeDestroy_depends_datasource required some
updates here because its implementation of PlanResourceChange was not
handling the insertion of the unknown value for attribute "computed".
The other changes here are just in an attempt to make the flow of this
test more obvious, by clarifying that it is simulating a -refresh=false
run, which effectively forces a deferred read since we skip the eager
read that would normally happen in the refresh step.
2019-02-07 18:33:14 -08:00
Martin Atkins 8882dcaf86 core: Fix TestContext2Plan_dataResourceBecomesComputed
Now that ProposedNewState uses null to represent Computed attributes not
set in the configuration, the provider must fill in the unknown value for
"computed" in its plan result.

It seems that this test was incorrectly updated during our bulk-fix after
integrating the HCL2 work, but it didn't really matter because the
ReadDataSource function isn't called in the happy path anyway. But to
make the intent clearer here, we also now make ReadDataSource return an
error if it is called, making it explicit that no call is expected.
2019-02-07 18:33:14 -08:00
Martin Atkins c3e7efec35 core: Reject unknown values after reading a data resource
Data resources do not have a plan/apply distinction, so it is never valid
for a data resource to produce unknown values in its result object.

Unknown values in the data resource _config_ cause us to postpone the read
altogether, so a data source never receives unknown values as input and
therefore may never produce unknown values as output.
2019-02-07 18:33:14 -08:00
James Bardin 32671241e0 set unknowns during initial PlanResourceChange
If ID is not set, make sure it's unknown.

Use SetUnknowns to set the rest of the computed values to Unknown.
2019-02-07 20:29:24 -05:00
James Bardin d17ba647a8 add SetUnknowns
SetUnknown walks through a resource and changes any unset (null) values
that are going computed in the schema to Unknown.
2019-02-07 20:24:36 -05:00
James Bardin be127725cc Additional tests with interpolated values 2019-02-07 20:23:39 -05:00
Martin Atkins f8a6f66be4 lang/funcs: Fix panic in "join" when an element is null
It is now a proper error message.
2019-02-07 14:35:13 -08:00
Martin Atkins c794bf5bcc plans/objchange: Don't presume unknown for values unset in config
Previously we would construct a proposed new state with unknown values in
place of any not-set-in-config computed attributes, trying to save the
provider a little work in specifying that itself.

Unfortunately that turns out to be problematic because it conflates two
concerns: attributes can be explicitly set in configuration to an unknown
value, in which case the final result of that unknown overrides any
default value the provider might normally populate.

In other words, this allows the provider to recognize in the proposed new
state the difference between an Optional+Computed attribute being set to
unknown in the config vs not being set in the config at all.

The provider now has the responsibility to replace these proposed null
values with unknown values during PlanResourceChange if it expects to
select a value during the apply step. It may also populate a known value
if the final result can be predicted at plan time, as is the case for
constant defaults specified in the provider code.

This change comes from a realization that from core's perspective the
helper/schema ideas of zero values, explicit default values, and
customizediff tweaks are all just examples of "defaults", and by allowing
the provider to see during plan whether these attributes are being
explicitly set in configuration and thus decide whether the default will
be provided immediately during plan or deferred until apply.
2019-02-07 14:01:39 -08:00
Sander van Harmelen 651ee113ac
Merge pull request #20239 from hashicorp/svh/b-build-script-v0.12
build.sh: only set git commit info for dev builds
2019-02-07 09:53:15 +01:00
Chris Griggs 8a964a4bc4
Merge pull request #20251 from cgriggs01/cgriggs01-community1
[Website]New community providers
2019-02-06 12:47:32 -08:00
Martin Atkins 1530fe52f7 core: Legacy SDK providers opt out of our new apply result check
The shim layer for the legacy SDK type system is not precise enough to
guarantee it will produce identical results between plan and apply. In
particular, values that are null during plan will often become zero-valued
during apply.

To avoid breaking those existing providers while still allowing us to
introduce this check in the future, we'll introduce a rather-hacky new
flag that allows the legacy SDK to signal that it is the legacy SDK and
thus disable the check.

Once we start phasing out the legacy SDK in favor of one that natively
understands our new type system, we can stop setting this flag and thus
get the additional safety of this check without breaking any
previously-released providers.

No other SDK is permitted to set this flag, and we will remove it if we
ever introduce protocol version 6 in future, assuming that any provider
supporting that protocol will always produce consistent results.
2019-02-06 11:40:30 -08:00
Martin Atkins a81bc23611 core: Verify that objects don't change unexpectedly during apply
Previously we would allow providers to change anything about the planned
object value during apply, possibly returning an entirely-unrelated object
of the same type. In practice this led to some subtle bugs where a single
planned attribute value would change during apply and cause a downstream
failure due to a dependent resource now seeing input other than what
_it_ expected during plan.

Now we'll produce an explicit error message for this case which places the
blame with the correct party: the upstream resource that changed. Without
this, unexpected changes would often lead to the downstream resource
implementation being blamed in error message even though it was just
reacting to the change from upstream.

As with most errors during apply, we'll still save the updated value in
the state but we'll halt the walk to ensure that the unexpected value
cannot propagate further and cause the result to potentially diverge
greatly from the changeset shown in the plan.

Compared to Terraform 0.11, we expect to see this error in many of the
same cases we saw the "diffs didn't match during apply" error in earlier
versions, since it is likely that many errors of that sort were the result
of unexpected upstream changes being incorrectly blamed on the downstream
resource that then used the result.
2019-02-06 11:40:30 -08:00
Martin Atkins 07930aa7fb core: Context apply tests should produce consistent apply results
Because Terraform Core has traditionally not checked that the final apply
result is consistent with what was planned, some of our apply tests were
producing inconsistent results.

Here we fix all of that so that they produce something compatible with
what they planned. This doesn't actually achieve anything in isolation,
but we're about to start enforcing this consistency in a subsequent
commit.
2019-02-06 11:40:30 -08:00
Sander van Harmelen 03e82687d6 build.sh: only set git commit info for dev builds
This commit fixes a problem where `make bin` would strip of any prerelease info.
2019-02-06 20:32:54 +01:00
Martin Atkins a9274beaca build: Run "go generate" in modules mode
It seems that all of the tools we run here are now sufficiently
modules-aware to run without problems in modules mode, and indeed running
_not_ in modules mode was causing problems with locating packages in
mockgen.
2019-02-06 11:19:44 -08:00
cgriggs01 41af6ce54b two new community providers 2019-02-06 11:01:22 -08:00
Radek Simko 5946fd898e
Merge pull request #20247 from surminus/update-outputs-docs
Update docs for 0.12 terraform_remote_state data source
2019-02-06 15:06:38 +01:00
Laura Martin 76dedfbf9d Update docs for 0.12 terraform_remote_state data source
In 0.12, the outputs for a data source of terraform_remote_state are
nested under the 'outputs' attribute [1]. This updates the docs
to make this change clearer.

Worked with @radeksimko at Terraform hackday, who has submitted a
related upgrade guide [2]

[1] 1f4d2f4c50/builtin/providers/terraform/data_source_state.go (L16-L43)
[2] d8e00191b7
2019-02-06 13:53:51 +01:00
James Bardin 1f4d2f4c50
Merge pull request #20235 from hashicorp/jbardin/id
only force top-level id's back to unknown
2019-02-05 16:42:01 -05:00
James Bardin 411df99f33 only force top-level id's back to unknown
Nested structures may have "id" fields, which should be treated
normally.
2019-02-05 16:16:08 -05:00
James Bardin 3b18dd7c01
Merge pull request #20224 from hashicorp/jbardin/sdk
SDK set fixes
2019-02-05 14:11:51 -05:00
Chris Griggs 84e93a7f42
Merge pull request #20233 from cgriggs01/cgriggs01-selectel-namechange
[Website] Update provider name
2019-02-05 11:10:14 -08:00
cgriggs01 4db4f9102d change plath name 2019-02-05 10:10:09 -08:00
James Bardin 8be864c1c7 don't allow computed set elems to be equal
If set elements are computed, we can't be certain that they are actually
equal. Catch identical computed set hashes when they are added to the
set, and alter the set key slightly to keep the set counts correct.

In previous versions the interpolation string would be included in the
set, and different string values would cause the set to hash
differently, so this is change is only activated for the new protocol.
2019-02-05 12:08:17 -05:00
James Bardin 79d1e0d7cf add failing test for multiple computed set elems 2019-02-05 12:08:17 -05:00
James Bardin 58c9c2311a Turn on helper/schema proto5 flag in GetSchema
This turns it on at the last moment, and in one place for all uses of
helper/schema. There's no way to use the new protocol without calling
GetSchema, so we can be sure that any subsequent api calls have this set
when required.
2019-02-05 12:08:17 -05:00
James Bardin 55b4307767 add proto5 feature flag
Add feature flag to allow special proto 5 behavior in helper/schema.
This is Meant to be used as a last resort for shim-related bugs.
2019-02-05 12:08:16 -05:00
James Bardin 81a4e705b1 DiffSuppressFunc should noop diffs in sets
Sets rely on diffs being complete for all elements, even when they are
unchanged. When encountering a DiffSuppressFunc inside a set the diffs
were being dropped entirely, possible causing set elements to be lost.
2019-02-05 12:08:16 -05:00
James Bardin 2e2374cfcb add failing test for set elements with custom diff
Adding a DiffSuppressFunc for set elements can cause them to be missed
in the set diff entirely.
2019-02-05 12:08:16 -05:00
Martin Atkins 7216049fdb plans/objchange: Improve precision of AssertObjectCompatible with sets
Previously we were just asserting that the number of elements didn't grow
between planned and actual. We still can't precisely correlate elements in
sets with unknown values, but here we adapt some logic we added earlier
to config/hcl2shim to ensure that we can find a plausible correlation for
each element in each set to at least one element in the other set, and
thus catch more cases where set elements might vanish or appear between
plan and apply, for improved safety.

This will still generate false negatives in some cases where unknown
values are present due to having to assume correlation is intended
wherever it is possible, but we'll catch situations where the actual value
is obviously contrary to what was planned.
2019-02-04 18:19:46 -08:00
Nick Fagerlund d1daa7e518 website: Make 0.11 and 0.12 subsections of 'Configuration Language' 2019-02-01 17:20:54 -08:00
Nick Fagerlund 58289e2d90 website: Change absolute 0.11 -> 0.11 links to relative 2019-02-01 17:20:54 -08:00
Nick Fagerlund 2c50808bbb website: Version notes: Add new -> old links to 0.12 language docs (function pages) 2019-02-01 17:20:54 -08:00
Nick Fagerlund 4a190127b9 website: Version notes: Add new -> old links to 0.12 language docs (except function pages) 2019-02-01 17:20:54 -08:00
Nick Fagerlund 9ce8e0ee18 website: Version notes: Add old -> new links to 0.11 docs 2019-02-01 17:20:54 -08:00
Nick Fagerlund b726cc4ba9 website: Update H1 header of 0.12 index.html 2019-02-01 17:20:54 -08:00
Nick Fagerlund 2dcd4a886f website: Standardize page titles in 0.11 language docs 2019-02-01 17:20:54 -08:00
Nick Fagerlund a9bcc0fe06 website: Fix sidebar nav links and 'sidebar_current' IDs for 0.11 docs
Because of the different possibilities for arranging the nav sidebars, we want
to make sure:

- IDs for the 0.11 and 0.12 language docs have a common prefix.
- That prefix is not the exact string `docs-config`.

Have I mentioned before that I really dislike this prefix matching behavior.
2019-02-01 17:20:54 -08:00
Nick Fagerlund 11cfdf3b67 website: Add 0.11 configuration language docs (WIP)
This is a non-working commit, because a bunch of links (including the sidebar
nav) are broken. Using a transition commit like this makes it easier to see the
changes necessary to get this content woven into the site.
2019-02-01 17:20:54 -08:00
Martin Atkins 6330abae33 build: Periodic refresh of our go.googlesource.com cookie
Just as with 336b352d6f, this refreshes our rate limit bypass cookie for
go.googlesource.com since the old one has apparently expired.

I did this by visiting https://go.googlesource.com/ and selecting
"Generate Password" from the top navigation. This cookie belongs to a
test account used by the Terraform team and should not be used by
non-Terraform codebases; please generate your own!
2019-02-01 16:01:59 -08:00
Martin Atkins bdcac8792d plugin: Use correct schema when marshaling imported resource objects
Previously we were using the type name requested in the import to select
the schema, but a provider is free to return additional objects of other
types as part of an import result, and so it's important that we perform
schema selection separately for each returned object.

If we don't do this, we get confusing downstream errors where the
resulting object decodes to the wrong type and breaks various invariants
expected by Terraform Core.

The testResourceImportOther test in the test provider didn't catch this
previously because it happened to have an identical schema to the other
resource type being imported. Now the schema is changed and also there's
a computed attribute we can set as part of the refresh phase to make sure
we're completing the Read call properly during import. Refresh was working
correctly, but we didn't have any tests for it as part of the import flow.
2019-02-01 15:22:54 -08:00
Martin Atkins d1c660eecf vendor: go get github.com/zclconf/go-cty@master
This fixes a bug in the TestConformance function that was generating false
positives when given two object types with the same number of attributes
but not identical attribute names.
2019-02-01 15:22:54 -08:00
James Bardin 4a603011c5 don't normalizeNullValues in ReadResource
The required normalization now happens in PlanResourceChange, and this
function is no longer appropriate for ReadResource.
2019-02-01 17:21:37 -05:00
James Bardin 89c1ba099f add computed set test with CustomizeDiff 2019-02-01 17:21:37 -05:00
Kristin Laemmert c810e4582c
command/show: continued work on `terraform show -json` output (#20171)
* command/jsonstate: do not hide SchemaVersion of '0'
* command/jsonconfig: module_calls should be a map
* command/jsonplan: include current terraform version in output
* command/jsonconfig: properly marshal expressions from a module call

Previously this was looking at the root module's variables, instead of
the child module variables, to build the module schema. This fixes that
bug.
2019-02-01 13:47:18 -08:00
Radek Simko 18030a487b
Merge pull request #20189 from hashicorp/vendor-bump-azure
vendor: github.com/hashicorp/go-azure-helpers@0.3.2
2019-02-01 18:27:18 +00:00
Radek Simko e88d4978db
Merge pull request #20192 from hashicorp/build-test-compile-modularized
make: Make test-compile module-aware
2019-02-01 18:01:12 +00:00