Our model for plans/planfile has unfortunately grown inconsistent with
changes to our modeling of plans.Plan.
Originally we considered the plan "header" and the planned changes as an
entirely separate artifact from the prior state, but we later realized
that carrying the prior state around with the plan is important to
ensuring we always have enough context to faithfully render a plan to the
user, and so we added the prior state as a field of plans.Plan.
More recently we've also added the "previous run state" to plans.Plan for
similar reasons.
Unfortunately as a result of that modeling drift our ReadPlan method was
silently producing an incomplete plans.Plan object, causing use-cases like
"terraform show" to produce slightly different results due to the
plan object not round-tripping completely.
As a short-term tactical fix, here we add state snapshot reading into the
ReadPlan function. This is not an ideal solution because it means that
in the case of applying a plan, where we really do need access to the
state _file_, we'll end up reading the prior state file twice. However,
the goal here is only to heal the modelling quirk with as little change
as possible, because we're not currently at a point where we'd be willing
to risk regressions from a larger refactoring.
Until now we've not really cared much about the state snapshot produced
by the previous Terraform operation, except to use it as a jumping-off
point for our refresh step.
However, we'd like to be able to report to an end-user whenever Terraform
detects a change that occurred outside of Terraform, because that's often
helpful context for understanding why a plan contains changes that don't
seem to have corresponding changes in the configuration.
As part of reporting that we'll need to keep track of the state as it
was before we did any refreshing work, so we can then compare that against
the state after refreshing. To retain enough data to achieve that, the
existing Plan field State is now two fields: PrevRunState and PriorState.
This also includes a very shallow change in the core package to make it
populate something somewhat-reasonable into this field so that integration
tests can function reasonably. However, this shallow implementation isn't
really sufficient for real-world use of PrevRunState because we'll
actually need to update PrevRunState as part of planning in order to
incorporate the results of any provider-specific state upgrades to make
the PrevRunState objects compatible with the current provider schema, or
else our diffs won't be valid. This deeper awareness of PrevRunState in
Terraform Core will follow in a subsequent commit, prior to anything else
making use of Plan.PrevRunState.
This is to make it more obvious at all uses of this field that it's not
something to be used for anything other than UI decisions, hopefully
prompting a reader of code elsewhere to refer to the comments to
understand why it has this unusual prefix and thus see what its intended
purpose is.
This only includes the internal mechanisms to make it work, and not any
of the necessary UI changes to "terraform plan" and "terraform apply" to
activate it yet.
The force-replace options are ultimately handled inside the
NodeAbstractResourceInstance.plan method, at the same place we handle the
similar situation of the provider indicating that replacement is needed,
and so the rest of the changes here are just to propagate the settings
through all of the layers in order to reach that point.
This only includes the core mechanisms to make it work. There's not yet
any way to turn this mode on as an end-user, because we have to do some
more work at the UI layer to present this well before we could include it
as an end-user-visible feature in a release.
At the lowest level of abstraction inside the graph nodes themselves, this
effectively mirrors the existing option to disable refreshing with a new
option to disable change-planning, so that either "half" of the process
can be disabled. As far as the nodes are concerned it would be possible
in principle to disable _both_, but the higher-level representation of
these modes prevents that combination from reaching Terraform Core in
practice, because we block using -refresh-only and -refresh=false at the
same time.
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.
For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.
This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.
So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
Previously there were only two planning modes: normal mode and destroy
mode. In that context it made sense for these to be distinguished only by
a boolean flag.
We're now getting ready to add our third mode, "refresh only". This
establishes the idea that planning can be done in one of a number of
mutually-exclusive "modes", which are related to but separate from the
various other options that serve as modifiers for the plan operation.
This commit only introduces the new plans.Mode type and replaces the
existing "destroy" flag with a variable of that type. This doesn't cause
any change in effective behavior because Terraform Core still supports
only NormalMode and DestroyMode, with NewContext rejecting an attempt to
create a RefreshMode context for now.
It is in retrospect a little odd that the "destroy" flag was part of
ContextOpts rather than just an argument to the Plan method, but
refactoring that would be too invasive a change for right now so we'll
leave this as a field of the context for now and save revisiting that for
another day.
Dynamic blocks with unknown for_each expressions are now decoded into an
unknown value rather than using a sentinel object with unknown
and null attributes. This will allow providers to precisely plan the
block values, rather than trying to heuristically paper over the
incorrect plans when dynamic is in use.
The stored planfile now serializes the required-replace path set and the
collection of before/after sensitivity marks. This ensures that storing
a plan and displaying it with `terraform show` renders the same output
for plans with required-replace resources, and those with sensitive
values in the diff.
Generating strings and comparing them to implement Equal is a quick and
easy solution. Unfortunately when this code is in the hot path, it
becomes very expensive, so this commit changes some of those instances
to compare the values directly.
Combined with using addr.Equal instead of checking for string equality,
this makes Terraform dramatically faster for some operations, such as
generating large JSON plans.
Catch attributes which are planed but not computed separately to provide
a clearer error to provider developers.
The error conditions were previously caught, however it was unclear from
the error text as to _why_ the change was an error. The statements about
value inequality would be incorrect when planning no changes for a value
which should not have been set in the first place.
The call to TestConformance needs to be reversed, since we want to
verify that the actual value returned conforms to the planned type.
While the inverse (checking that the planned value conforms to the
applied type) works for everything terraform has been exposed to up
until now, this fails when the planned type has dynamic attributes which
are allowed to become concrete types.
with NestedType objects.
There are a handful of mostly cosmetic changes in this PR which likely
make the diff awkward to read; I renamed several functions to
(hopefully) clarifiy which funcs worked with Blocks vs other types. I
also extracted some small code snippets into their own functions for
reusability.
The code that descends into attributes with NestedTypes is similar to
the block-handling code, and differs in all the ways blocks and
attributes differ: null is valid for attributes, unlike blocks which can
only be present or empty.
- rename ProposedNewObject to ProposedNew:
Now that there is an actual configschema.Object it will be clearer if
the function names match the type the act upon.
- extract attribute-handling logic from assertPlanValid and extend
A new function, assertPlannedAttrsValid, takes the existing
functionality and extends it to validate attributes with NestedTypes.
The NestedType-specific handling is in assertPlannedObjectValid, which
is very similar to the block-handling logic, except that nulls are a
valid plan (an attribute can be null, but not a block).
This is needed for cases where a variable may be fetched and become
a member of a set, and thus the whole set is marked, which means
ElementIterator will panic on unmarked values
The Legacy SDK cannot handle missing strings from objects in sets, and
will insert an empty string when planning the missing value. This
subverts the `couldHaveUnknownBlockPlaceholder` check, and causes
errors when `dynamic` is used with NestingSet blocks.
We don't have a separate codepath to handle the internals of
AssertObjectCompatible differently for the legacy SDK, but we can treat
empty strings as null strings within set objects to avoid the failed
assertions.
If a NestingList or NestingMap contains a dynamic type, they must be
handled as a cty.Tuple and cty.Object respectively, because the elements
may not have precisely matching types.
Remove marks for object compatibility tests to allow apply
to continue. Adds a block to the test provider to use
in testing, and extends the sensitivity apply test to include a block
In order to handle various edge cases during a full destroy, add
FullDestroy to the synchronized changes so we can attempt to deduce if
the plan was created from `terraform destroy`.
It's possible that the plan was created by removing all resourced from
the configuration, but in that case the end result is the same. Any of
the edge cases with provider or destroy provisioner configurations would
not apply, since there would not be any configuration references to
resolve.
Since the refreshed state is now an artifact of the plan process, it
makes sense to add it to the Plan type, rather than adding an additional
return value to the Context.Plan method.
Mark sensitivity on a value. However, when the value is encoded to send to the
provider to produce a changeset we must remove the marks, so unmark the value
and remark it with the saved path afterwards
The couldHaveUnknownBlockPlaceholder helper was added to detect when a
set block has a placeholder for an unknown number of values. This worked
fine when the number increased from 1, but we were still attempting to
validate the unknown placeholder against the empty set when the final
count turned out to be 0.
Since we can't differentiate the unknown dynamic placeholder value from
an actual set value, we have to skip that object's validation
altogether.
* Update error message for apply validation
Add a hint that the validation failure has occurred at the root of the resource
schema to the error message. This is because the root resource has an empty
path when being validated and the path is being relied upon to provide context
into the error message.
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.
This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".
This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.
The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.
We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.
This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
Martin Atkins