Commit Graph

22440 Commits

Author SHA1 Message Date
Martin Atkins 30a46ed8ed command/format: Further refinement on the heuristic JSON-based diff
We'll now show an "update" symbol prior to the argument to this synthetic
jsonencode(...) call, for consistency with how we show nested values in
other cases and to attach a verb to any "# forces replacement".

We'll also show a special form in the case where the value seems to differ
only in whitespace, so users can understand what's going on in that
hopefully-rare situation, particularly if those whitespace-only changes
end up forcing us to replace a remote object.
2018-10-16 19:14:11 -07:00
Martin Atkins 783205948c command/format: JSON value formatting heuristic not for primitive values
Since our own syntax for primitive values is similar to that of JSON, and
since we permit automatic conversions from number and bool to string, we
must do this special JSON value diff formatting only if the value is a
JSON array or object to avoid confusing results.
2018-10-16 19:14:11 -07:00
Martin Atkins 4406b06ef8 command/format: Don't include commas after elements in map diffs
Although commas are allowed in this context, it's not idiomatic to use
them.
2018-10-16 19:14:11 -07:00
Martin Atkins ce157c7f23 command/format: show deep diffs for old/new JSON strings
Because so far we've not supported dynamically-typed complex data
structures, several providers have used strings containing JSON to stand
in for these.

In order to get a readable diff in those cases, we'll recognize situations
where old and new are both JSON and present a diff of the effective value
of the JSON, using a faux call to the jsonencode(...) function to indicate
when we've done so.

This is a bit of a "cute" heuristic, but is important at least for now
until we can migrate away from that practice of passing large JSON strings
to providers and use dynamically-typed attributes instead.
2018-10-16 19:14:11 -07:00
Martin Atkins 9706a00b3a command/format: correct alignment for multi-line values in map diff 2018-10-16 19:14:11 -07:00
Martin Atkins 20318ca193 backend/local: Sort planned resource changes before rendering them
As in the old plan renderer, they are ordered by the natural ordering of
the resource addresses.
2018-10-16 19:14:11 -07:00
Martin Atkins 04d8c17be8 command/format: Nice rendering for changes to map values 2018-10-16 19:14:11 -07:00
Martin Atkins a2f4b8b058 command/format: Show resource diff with header "comment"
This extra comment line gives us a place to show the full resource address
(since the block header line only includes type and name) and also allows
us to explain in long form the meaning of the change icon on the following
line.
2018-10-16 19:14:11 -07:00
Martin Atkins 239a54ad6f command: initial structural diff rendering
This is a light adaptation of our earlier prototype of structural diff
rendering, as a starting point for what we'll actually ship. This is not
consistent with the latest mocks, so will need some additional work before
it is ready, but integrating this allows us to at least see the plan
contents while fixing up remaining issues elsewhere.
2018-10-16 19:14:11 -07:00
Martin Atkins 1aa9ac14cc plans/objchange: LongestCommonSubsequence
This algorithm is the usual first step when generating diffs. This package
is a bit of a strange home for it, but since it works with changes to
cty.Value this feels more natural than any other place it could be.
2018-10-16 19:14:11 -07:00
Martin Atkins 1ced176fc6 plans: Track RequiredReplace as a cty.PathSet
We were previously tracking this as a []cty.Path, but having it turned
into a pathset on creation makes downstream use of it more convenient and
ensures that it'll obey expected invariants like not containing the same
path twice.
2018-10-16 19:14:11 -07:00
Martin Atkins d54b52fa01 core: Fix error reporting for malformed provider response values 2018-10-16 19:14:11 -07:00
Martin Atkins 630b0d147d providers/terraform: Always produce correctly-typed result
We need to produce values for all of the attributes implied by our schema,
even if some of them are null.
2018-10-16 19:14:11 -07:00
Martin Atkins 235f582ae5 core: Re-implement EvalReadDataApply against new provider interface 2018-10-16 19:14:11 -07:00
Martin Atkins d96284f2e2 tfdiags: FormatErrorPrefixed
When presenting an error that may be a PathError, the error's path is
usually relative to some other value. If the caller is able to express
that value (or, more often, a reference to it) in HCL syntax then this
method will produce a complete expression in the error message,
concatenating any path information from the error to the end of the given
prefix string.
2018-10-16 19:14:11 -07:00
Martin Atkins 182e783885 core: EvalReadDataDiff to output direct config value alongside planned
We're now writing the "planned new value" to OutputValue, but the data
resource nodes during refresh need to see the verbatim config value in
order to decide whether read must be deferred to the apply phase, so we'll
optionally export that here too.
2018-10-16 19:14:11 -07:00
Martin Atkins 20adb9d9b7 core: Evaluate resource references from plan where possible
Our state representation is not able to preserve unknown values, so it's
not suitable for retaining the transient incomplete values we produce
during planning.

Instead, we'll discard the unknown values when writing to state and have
the expression evaluator prefer an object from the plan where possible.
We still use the shape of the transient state to inform things like the
resource's "each mode", so the plan only masks the object values
themselves.
2018-10-16 19:14:11 -07:00
Martin Atkins 1afdb055ca core: Re-implement ReadDataDiff around our new approach
This is no longer a call into the provider, since all of the data diff
logic is standard for all data sources anyway. Instead, we just compute
the planned new value and construct a planned change from that as-is.

Previously the provider could, in principle, customize the read diff. In
practice there is no real reason to do that and the existing SDK didn't
pass that possibility through to provider code, so we can safely change
this without impacting provider compatibility.
2018-10-16 19:14:11 -07:00
Martin Atkins 3f8a973846 plans/objchange: when prior is null, computed attributes are unknown 2018-10-16 19:14:11 -07:00
Martin Atkins 7249a73f79 core: Don't panic if "required replace" path doesn't resolve to a value 2018-10-16 19:14:11 -07:00
Martin Atkins 6acf335e60 core: Fix evaluation of collections of resource instances 2018-10-16 19:14:11 -07:00
Martin Atkins 97da905c6e helper/plugin: ReadResource to deal with missing remote object 2018-10-16 19:14:11 -07:00
Martin Atkins 1d672623eb core: Remove changes from the plan after they are applied 2018-10-16 19:14:11 -07:00
Martin Atkins a7342de274 core: Properly handle no-op changes in plan
Previously we just left these out of the plan altogether, but in the new
plan types we intentionally include change information for every resource
instance, even if no changes are actually planned, to allow alternative
plan file viewers to show what isn't changing as well as what is.
2018-10-16 19:14:11 -07:00
Martin Atkins 2f0e5d93c8 plans: ChangesSync.GetResourceInstanceChange must copy the change
This is promised in its doc comment, but wasn't actually done in practice.
2018-10-16 19:14:11 -07:00
Martin Atkins c27f900d92 helper/plugin: Don't panic while preparing response in ApplyResourceChange 2018-10-16 19:14:11 -07:00
Martin Atkins 3d86dc51e0 core: Re-implement EvalReadDiff for the new plan types
This also includes passing in the provider schema to a few more EvalNodes
that were expecting it but not getting it, in order to be able to
successfully test the implementation of EvalReadDiff here.
2018-10-16 19:14:11 -07:00
Martin Atkins 03e6771536 states/statemgr: don't panic if no state file is present on first write 2018-10-16 19:14:11 -07:00
Martin Atkins 5731703de5 command/format: show planned changes in rendered plan
This codepath is going to be significantly changed before release to make
it support structural diff of the new data types, but this lets us lean on
the old renderer to produce partial output in the mean time while we
continue to work on getting things working end-to-end after the
considerable refactoring that's been going on.
2018-10-16 19:14:11 -07:00
Martin Atkins bd299b9a22 core: Re-implement EvalWriteDiff to work with new plan types 2018-10-16 19:14:11 -07:00
Martin Atkins 686018ae12 helper/plugin: don't panic in PlanResourceChange PlannedState 2018-10-16 19:14:11 -07:00
Martin Atkins ec11efc50a core: Tolerate the prior state being nil in EvalDiff
This happens legitimately in situations where we'll be creating an object
for this resource instance for the first time.
2018-10-16 19:14:11 -07:00
Martin Atkins 7d760c09fb core: Update EvalCountFixZeroOneBoundaryGlobal for new state types 2018-10-16 19:14:11 -07:00
Martin Atkins ed7e331041 terraform: Return validation diagnostics with source info where possible 2018-10-16 19:14:11 -07:00
Martin Atkins 9ec00468f8 plugin/convert: Convert cty.PathError to diagnostics with attribute path 2018-10-16 19:14:11 -07:00
Martin Atkins 5d744e6824 plugin: TRACE log for each GRPC-remoted plugin call 2018-10-16 19:14:11 -07:00
Martin Atkins 4bf19797f3 plugin: Fix deadlock in getSchema where mutex is never unlocked
On the second call here this would fail to release the lock.
2018-10-16 19:14:11 -07:00
Martin Atkins 5559d21855 plugin: Register v5 plugins only if present
Previously we would construct both provisioner and the provider objects if
either callback was set, but this is incorrect because a plugin should
actually set only one of these at a time, depending on what kind of plugin
it is.
2018-10-16 19:14:11 -07:00
Martin Atkins a6b5980a4f backend/local: Bail immediately if terraform.Context can't be created 2018-10-16 19:14:11 -07:00
Martin Atkins 7139887565 vendor: upgrade go-cty to latest
This includes the new PathSet type, which we'll use to represent the
"requires replacement" set of attribute paths coming back from providers
during planning.
2018-10-16 19:14:11 -07:00
Martin Atkins 1bb79696c6 vendor: update to latest github.com/zclconf/go-cty
This includes a bugfix to the cty/msgpack package to ensure correct
decoding of unknown and null values.

This also includes updates to cty's dependencies.
2018-10-16 19:14:11 -07:00
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins b5560a01c0 vendor: switch to Go modules
govendor's lack of understanding of transitive versions was making it hell
to figure out some dependency hell between etcd, grpc, and protobuf. After
fighting with it for a few hours, I decided to give Go 1.11rc2 a try since
previous experiments had been promising on the 1.11 tree in master.

The dependencies all worked out first time when managed using the Go
Modules code, and so we'll run with this now to continue to make progress
though we may wish to back out of this nearer to release and return to
govendor for a while until other projects have caught up.

However, since this commit includes a vendor directory produced using Go
Modules it doesn't actually _require_ Go 1.11 to build, and instead
requires it only to make further changes to the selected versions in the
vendor dir. Go 1.10's vendoring support will still find the modules in
their expected locations within the vendor dir.
2018-10-16 19:11:09 -07:00
Martin Atkins a4c98d3365 govendor fetch github.com/google.golang.org/grpc/...@v1.14.0 2018-10-16 19:11:09 -07:00
James Bardin 0b76c42ad2 replace provider and provisioner types in tests
The field assignments and methods are incorrect still, but the type
definitions and returns are now updated.
2018-10-16 19:11:09 -07:00
Martin Atkins 549544f201 configschema: Handle nested blocks containing dynamic-typed attributes
We need to make the collection itself be a tuple or object rather than
list or map in this case, since otherwise all of the elements of the
collection are constrained to be of the same type and that isn't the
intent of a provider indicating that it accepts any type.
2018-10-16 19:11:09 -07:00
Martin Atkins 4c78539c2b plans/objchange: AssertObjectSuperset function
This function's goal is to ensure that the "final" plan value produced
by a provider during the apply step is always consistent with the known
parts of the planned value produced during the plan step.

Any error produced here indicates a bug in the provider.
2018-10-16 19:11:09 -07:00
Martin Atkins 0317da9911 plans/objchange: logic for merging prior state with config
This produces a "proposed new state", which already has prior computed
values propagated into it (since that behavior is standard for all
resource types) but could be customized further by the provider to make
the "_planned_ new state".

In the process of implementing this it became clear that our configschema
DecoderSpec behavior is incorrect, since it's producing list values for
NestingList and map values for NestingMap. While that seems like it should
be right, we should actually be using tuple and object types respectively
to allow each block to have a different runtime type in situations where
an attribute is given the type cty.DynamicPseudoType. That's not fixed
here, and so without a further fix list and map blocks will panic here.
The DecoderSpec implementation will be fixed in a subsequent commit.
2018-10-16 19:11:09 -07:00
Martin Atkins dfe0988f10 plans: Record private data bytes as part of resource change
This allows a provider to retain arbitrary extra data in the plan and
make use of it during apply. The contents are not used by Terraform and
never shown to the user.
2018-10-16 19:11:09 -07:00
Martin Atkins fb57801dfe states: object Private data is []byte, not cty.Value
We're going to allow the provider to encode whatever it wants in here, so
a provider can use whatever is most convenient for its implementation
language and to avoid some of the bugs we saw with the prior model where
the forced round-trip through JSON and back into interface{} would cause
some loss of fidelity, leading to bugs.
2018-10-16 19:11:09 -07:00