Commit Graph

28789 Commits

Author SHA1 Message Date
Chris Arcand 9e71da61eb Remove several ignore rules
The main purpose of this change is to avoid a problem where new golden
files added to certain directories for test purposes (like .log)
shouldn't be ignored.

Cleaning up a bit more and broadening the definition, this removes
ignore rules for artifacts of Terraform itself (state files, plans).
It's generally not recommended to be using this codebase as your
Terraform working directory anyway; build here, test elsewhere.
2021-09-01 14:37:26 -05:00
James Bardin 7c02ef6220
Merge pull request #29482 from hashicorp/jbardin/computed-obj-attrs
Computed values within nested object types in `AssertPlanValid`
2021-08-31 11:26:52 -04:00
James Bardin f195ce7fd4 remove temp test 2021-08-31 11:17:32 -04:00
Martin Atkins 48859417fa
Update CHANGELOG.md 2021-08-30 14:12:17 -07:00
Martin Atkins 7803f69d42 core: Enable TestContext2Plan_movedResourceBasic
This is the first test exercising the basic functionality of config-driven
move. We previously had it skipped because Terraform's previous design
of treating all three of the state artifacts as mutable attributes of
terraform.Context meant that it was too late during planning to deal with
the move operations, and thus this test was failing.

Thanks to the previous commit, which changes the terraform.Context API
such that we can defer creating the three state artifacts until we're
already doing planning, this test now works and shows Terraform correctly
handling a resource that was formerly called "a" and is now called "b",
with a "moved" block recording that renaming.
2021-08-30 13:59:14 -07:00
Martin Atkins 89b05050ec core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.

This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.

Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.

However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.

My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
 - The "terraform graph" command is no longer hooked directly into the
   core graph builders, because that's no longer part of the public API.
   However, I did include a couple new Context functions whose contract
   is to produce a UI-oriented graph, and _for now_ those continue to
   return the physical graph we use for those operations. There's no
   exported API for generating the "validate" and "eval" graphs, because
   neither is particularly interesting in its own right, and so
   "terraform graph" no longer supports those graph types.
 - terraform.NewContext no longer has the responsibility for collecting
   all of the provider schemas up front. Instead, we wait until we need
   them. However, that means that some of our error messages now have a
   slightly different shape due to unwinding through a differently-shaped
   call stack. As of this commit we also end up reloading the schemas
   multiple times in some cases, which is functionally acceptable but
   likely represents a performance regression. I intend to rework this to
   use caching, but I'm saving that for a later commit because this one is
   big enough already.

The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.

However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-30 13:59:14 -07:00
Martin Atkins 4faac6ee43 core: Record move result information in the plan
Here we wire through the "move results" into the graph walk data
structures so that all of the the nodes which produce
plans.ResourceInstanceChange values can capture the "PrevRunAddr" for
each resource instance.

This doesn't actually quite work yet, because the logic in Context.Plan
isn't actually correct and so the updated state from
refactoring.ApplyMoves isn't actually visible as the "previous run state".
For that reason, the context test in this commit is currently skipped,
with the intent of re-enabling it once the updated state is properly
propagating into the plan graph walk and thus we can actually react to
the result of the move while choosing actions for those addresses.
2021-08-30 13:59:14 -07:00
Martin Atkins 22b36d1f4c Field for the previous address of each resource instance in the plan
In order to expose the effect of any relevant "moved" statements we dealt
with prior to creating the plan, we'll record with each
ResourceInstanceChange both is current address and the address it was
tracked at for the previous run.

To save consumers of these objects from having to special-case the
situation where there _was_ no previous run (e.g. because this is a Create
change), we'll just pretend the previous run address was the same as the
current address in that case, the same as for an update without any
renaming in effect.

This includes a breaking change to the plan file format, but one that
doesn't require a version number increment because there is no ambiguity
between the two formats and so mismatched parsers will already fail with
an error message.

As of this commit we've just added the new field but not yet populated it
with any useful information: it always just matches Addr. A future commit
will wire this up to the result of applying the moves so that we can
populate it correctly. We also don't yet expose this new information
anywhere in the UI layer.
2021-08-30 13:59:14 -07:00
James Bardin ea68d79ea2 nested object values can be computed
While blocks were not allowed to be computed by the provider, nested
objects can be. Remove the errors regarding blocks and verify unknown
values are valid.
2021-08-30 14:30:07 -04:00
James Bardin 903797084a objects vs maps in nested object types
When using NestedMap objects, unify the codepath for both maps and
objects as they may be interchangeable.
2021-08-30 14:30:02 -04:00
James Bardin 7ca6be8285 correctly verify planned nested object values
The validation for nested object types with computed attributes was
using the incorrect function call.
2021-08-30 14:29:15 -04:00
Laura Pacilio c29e10b0f1
Merge pull request #29480 from Lasthalf/patch-1
generated -> generate
2021-08-30 12:04:30 -04:00
Topher Ayrhart b1d56076a4
generated -> generate
In the last paragraph, the word "generated" is in the wrong tense for the sentence. The correct word is "generate" (unless I misunderstand the sentence 🙂).
2021-08-30 08:59:50 -05:00
James Bardin fee0bffed3
Merge pull request #29445 from likexian/main
fixed tencentcloud tag
2021-08-30 08:58:24 -04:00
Martin Atkins 6562466c32
Be explicit that community PR review is currently paused 2021-08-27 10:43:51 -07:00
Laura Pacilio daad109067
Merge pull request #29063 from moskyb/s3-backend-kms-permissioning-note
Add a note to the docs on the S3 backend around permissions needed for encrypted state storage
2021-08-25 10:14:12 -04:00
Laura Pacilio cb6218ac65
Merge pull request #28310 from gastrodon/main
docs: fix link to index
2021-08-23 18:05:28 -04:00
Laura Pacilio 0a78072a2b
Merge pull request #28089 from jnerin/patch-1
Update expressions.html link to operators.html
2021-08-23 18:00:42 -04:00
Laura Pacilio 03849d850a
Merge pull request #28486 from joshschmitter/patch-1
setsubtract doc: use "Relative Complement" correctly
2021-08-23 15:49:51 -04:00
joshschmitter cb41b158f1
concise use of "relative complement"
Co-authored-by: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com>
2021-08-23 14:40:35 -05:00
Laura Pacilio 3018289bbf
Merge pull request #29449 from hashicorp/laura-fix-jsonencode-note
Update jsonencode minified note
2021-08-23 15:33:55 -04:00
Laura Pacilio d427c81a89 Update jsonencode minified note 2021-08-23 15:01:38 -04:00
Laura Pacilio dffb8b4a16
Merge pull request #29088 from brenthc/minify_jsonencode
note that output of jsonencode is minified
2021-08-23 14:24:37 -04:00
Brent Woodruff ed84cb886e
Apply suggested wording update. 2021-08-23 14:10:50 -04:00
Laura Pacilio 5863af448f
Merge pull request #28784 from TAYTS/patch-2
Update range.html.md
2021-08-23 14:07:15 -04:00
Laura Pacilio c1d16c7421
Merge pull request #29311 from NargiT/patch-1
fix syntax
2021-08-23 14:00:43 -04:00
Laura Pacilio 6235453d21
Merge pull request #29358 from moajo/add-s3-delete-object-to-document
Update website/docs/language/settings/backends/s3.html.md
2021-08-23 13:59:48 -04:00
Laura Pacilio d0bc382797
Merge pull request #29359 from 45deg/45deg-patch-1
Fix a result in examples for urlencode.html.md
2021-08-23 13:57:27 -04:00
Laura Pacilio c8dfaa12cc
Merge pull request #29433 from fedorg/patch-1
remove large sections of empty space
2021-08-23 13:35:31 -04:00
Laura Pacilio d2f80f35b1
Merge pull request #29316 from alkis-hexa/patch-1
Update for.html.md
2021-08-23 12:31:43 -04:00
Li Kexian 3f876d14d8 fixed tencentcloud tag 2021-08-23 09:46:35 +08:00
Martin Atkins 960f736bc2 build: CI check to verify that the protobuf stubs are up-to-date 2021-08-20 16:18:48 -07:00
Martin Atkins ce96d82de0 build: Centralize our protobuf compilation steps
We have a few different .proto files in this repository that all need to
get recompiled into .pb.go files each time we change them, but we were
previously handling that with some scripts that just assumed that protoc
and the relevant plugins were already installed on the system somewhere,
at the right versions.

In practice we've been constantly flopping between different versions of
these tools due to folks having different versions installed in their
development environments. In particular, the state of the .pb.go files
in the prior commit wasn't reproducible by any single version of the tools
because they've all slightly diverged from one another.

In the interests of being more consistent here and avoiding accidental
inconsistencies, we'll now centralize the protocol buffer compile steps
all into a single tool that knows how to fetch and install the expected
versions of the various tools we need and then run those tools with the
right options to get a stable result.

If we want to upgrade to either a newer protoc or a newer protoc-gen-go
in future then we'll do that in a central location and update all of the
.pb.go files at the same time, so that we're always consistently tracking
the same version of protocol buffers everywhere.

While doing this I attempted to keep as close as possible to the toolchain
we'd most recently used, but since they were not consistent with each
other they've now all changed which version numbers they record at minimum,
and the planproto stub in particular now also has a slightly different
descriptor serialization but is otherwise offering the same API.
2021-08-20 16:18:48 -07:00
James Bardin 249e05f827
Merge pull request #29437 from hashicorp/jbardin/move
refactoring: complete CanChainFrom and NestedWithin
2021-08-20 17:54:27 -04:00
Alisdair McDiarmid 40dd4292b8
Merge pull request #29438 from hashicorp/alisdair/init-force-copy-multiple-workspaces
command: Suppress prompt for init -force-copy
2021-08-20 15:21:43 -04:00
James Bardin 30afb492ab fix ApplyMove test with nested modules working
The addrs package can now correctly handle the combinations of nested
module endpoints, which fixes these 2 tests.
2021-08-20 15:17:06 -04:00
James Bardin bc60f7aae4 Extend CanChainFrom to handle relative modules
CanChainFrom needs to be able to handle move statements from different
relative modules, re-implementing with addrs.anyKey

Add the anyKey InstanceKey value to the addrs package to simplify module
path comparison. This allows all combinations of module path
representation to be normalized into a ModuleInstance which can be
compared directly, rather than dealing with multiple levels of different
prefix types.
2021-08-20 15:17:06 -04:00
Alisdair McDiarmid dbee7e1492
Merge pull request #29435 from hashicorp/alisdair/fix-add-state-lock
command: Fix stale lock after running add
2021-08-20 15:16:26 -04:00
Alisdair McDiarmid 2762a940c0 command: Suppress prompt for init -force-copy
The -force-copy flag to init should automatically migrate state.
Previously this was not applied to one case: when migrating from
a backend with multiple workspaces to another backend supporting
multiple workspaces. I believe this was an oversight so this commit
fixes that.
2021-08-20 14:46:09 -04:00
Alisdair McDiarmid 70a4f7a6b6 command: Fix stale lock after running add 2021-08-20 13:22:11 -04:00
Fedor Golishevskiy 4fd3bd4911
remove large sections of empty space 2021-08-20 17:31:26 +03:00
James Bardin 498dc36635
Merge pull request #29421 from hashicorp/jbardin/tencentcloud-cleanup
cleanup tencentcloud missing tag
2021-08-20 09:38:26 -04:00
James Bardin b8daf58622
Merge pull request #29420 from hashicorp/upgrade-aws-sdk-go-base
backend/s3: Updates `github.com/hashicorp/aws-sdk-go-base` from 0.6.0 to 0.7.1
2021-08-20 09:02:40 -04:00
James Bardin b543705840 cleanup tencentcloud missing tag
TencentCloud deleted all their v3 tags, so any system that does not use
the global Go proxy will fail to find the
github.com/tencentcloud/tencentcloud-sdk-go module source.

Update go.mod to point to the specific commit rather than the
incompatible and missing version tag.
2021-08-19 14:33:09 -04:00
Graham Davison e113974fb2 Updates `github.com/hashicorp/aws-sdk-go-base` from 0.6.0 to 0.7.1 2021-08-19 10:51:43 -07:00
James Bardin 322971f5ad update go.sum 2021-08-19 12:44:35 -04:00
James Bardin 11561b22cd
Merge pull request #29330 from hashicorp/jbardin/move
refactoring: CanChainFrom and NestedWithin
2021-08-19 12:34:38 -04:00
Martin Atkins d0fc5e5262
Update CHANGELOG.md 2021-08-19 09:28:26 -07:00
Martin Atkins 43d753e727 command: "terraform add" is experimental
We're aware of several quirks of this command's current design, which
result from some existing architectural limitations that we can't address
immediately.

However, we do still want to make this command available in its current
capacity as an incremental improvement, so as a compromise we'll document
it as experimental. Our intent here is to exclude it from the
Terraform 1.0 Compatibility Promises so that we can have the space to
continue to improve the design as other parts of the overall Terraform
system gain new capabilities.

We don't currently have any concrete plan for this command to be
stabilized and subject to compatibility promises. That decision will
follow from ongoing discussions with other teams whose systems may need to
change in order to support the final design of "terraform add".
2021-08-19 09:27:30 -07:00
James Bardin 553d6525d2 more move tests 2021-08-19 12:05:53 -04:00