Commit Graph

1882 Commits

Author SHA1 Message Date
Martin Atkins b9a93a0fe7 Move addrs/ to internal/addrs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins e81162c4e1 Move provisioners/ to internal/provisioners/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 1409f30f9c Move providers/ to internal/providers/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 05caff2ca3 Move tfdiags/ to internal/tfdiags/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 4c254cc2be Move httpclient/ to internal/httpclient/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 415ab31db8 Move e2e/ to internal/e2e/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 4c5b866dea Move registry/ to internal/registry/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 2bd2568ad8 Move repl/ to internal/repl/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 70eebe3521 Move dag/ to internal/dag/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins cf93aa8046 Move builtin/ to internal/builtin/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 73dda868cc Move backend/ to internal/backend/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
James Bardin f5e0d13079
Merge pull request #28718 from hashicorp/jbardin/backend-migrate
Prevent automatic backend migration during `terraform init`
2021-05-17 14:42:35 -04:00
James Bardin b11578da45 prevent -migrate-state with -reconfigure 2021-05-17 14:28:34 -04:00
James Bardin 1b48636b42 update init docs for -migrate-state 2021-05-17 12:41:54 -04:00
James Bardin edc2695d18 no longer automatically attempt state migration
Add `init -migrate-state` flag to indicate automatic state migration is
desired. This flag will be implied by the `-force-copy` flag, since that
would indicate state migration is expected.

If `init` encounters a change to the stored backend configuration, it
will now always return an error when neither `-reconfigure` or
`-migrate-state` is supplied.

Turn the most common legacy output strings into diagnostics, removing
the "see above text" error output.
2021-05-17 12:41:54 -04:00
James Bardin feb7622846 monitor plugin std outputs for unexpected data
Once a plugin process is started, go-plugin will redirect the stdout and
stderr stream through a grpc service and provide those streams to the
client. This is rarely used, as it is prone to failing with races
because those same file descriptors are needed for the initial handshake
and logging setup, but data may be accidentally sent to these
nonetheless.

The usual culprits are stray `fmt.Print` usage where logging was
intended, or the configuration of a logger after the os.Stderr file
descriptor was replaced by go-plugin. These situations are very hard for
provider developers to debug since the data is discarded entirely.

While there may be improvements to be made in the go-plugin package to
configure this behavior, in the meantime we can add a simple monitoring
io.Writer to the streams which will surface th data as warnings in the
logs instead of writing it to `io.Discard`
2021-05-13 16:57:36 -04:00
James Bardin ef88c54604
Merge pull request #28687 from hashicorp/jbardin/sensitive-changes
Decode change values with marks
2021-05-13 12:44:03 -04:00
Martin Atkins 917309fb5a command/diff: Small additional context about deposed objects
In the very unusual situation where we end up planning to destroy a
deposed object alone, it's likely that we're exposing users to this idea
of "deposed" for the very first time.

This additional sentence will hopefully give some extra context for what
that means. We don't really have room here for a lengthy explanation about
how deposed objects come to exist but this will hopefully be enough of
a hook to help users connect this with an error they saw on a previous
run, or at least to have some additional keywords to search for if they
want to research further.
2021-05-13 09:05:06 -07:00
Martin Atkins f2adfb6e2a core: Treat deposed objects the same as orphaned current objects
In many ways a deposed object is equivalent to an orphaned current object
in that the only action we can take with it is to destroy it. However, we
do still need to take some preparation steps in both cases: first, we must
ensure we track the upgraded version of the existing object so that we'll
be able to successfully render our plan, and secondly we must refresh the
existing object to make sure it still exists in the remote system.

We were previously doing these extra steps for orphan objects but not for
deposed ones, which meant that the behavior for deposed objects would be
subtly different and violate the invariants our callers expect in order
to display a plan. This also created the risk that a deposed object
already deleted in the remote system would become "stuck" because
Terraform would still plan to destroy it, which might cause the provider
to return an error when it tries to delete an already-absent object.

This also makes the deposed object planning take into account the
"skipPlanChanges" flag, which is important to get a correct result in
the "refresh only" planning mode.

It's a shame that we have almost identical code handling both the orphan
and deposed situations, but they differ in that the latter must call
different functions to interact with the deposed rather than the current
objects in the state. Perhaps a later change can improve on this with some
more refactoring, but this commit is already a little more disruptive than
I'd like and so I'm intentionally deferring that for another day.
2021-05-13 09:05:06 -07:00
Martin Atkins 3c8a4e6e05 command+backend/local: -refresh-only and drift detection
This is a light revamp of our plan output to make use of Terraform core's
new ability to report both the previous run state and the refreshed state,
allowing us to explicitly report changes made outside of Terraform.

Because whether a plan has "changes" or not is no longer such a
straightforward matter, this now merges views.Operation.Plan with
views.Operation.PlanNoChanges to produce a single function that knows how
to report all of the various permutations. This was also an opportunity
to fill some holes in our previous logic which caused it to produce some
confusing messages, including a new tailored message for when
"terraform destroy" detects that nothing needs to be destroyed.

This also allows users to request the refresh-only planning mode using a
new -refresh-only command line option. In that case, Terraform _only_
performs drift detection, and so applying a refresh-only plan only
involves writing a new state snapshot, without changing any real
infrastructure objects.
2021-05-13 09:05:06 -07:00
Martin Atkins ce69c3903f command/views: Show refresh-detected changes as part of a plan
We've always had a mechanism to synchronize the Terraform state with
remote objects before creating a plan, but we previously kept the result
of that to ourselves, and so it would sometimes lead to Terraform
generating a planned action to undo some upstream drift, but Terraform
would give no context as to why that action was planned even though the
relevant part of the configuration hadn't changed.

Now we'll detect any differences between the previous run state and the
refreshed state and, if any managed resources now look different, show
an additional note about it as extra context for the planned changes that
follow.

This appears as an optional extra block of information before the normal
plan output. It'll appear the same way in all of the contexts where we
render plans, including "terraform show" for saved plans.
2021-05-13 09:05:06 -07:00
Martin Atkins 42e0985839 command: use -lock=false consistently in -help output
Previously the docs for this were rather confusing because they showed an
option to turn _on_ state locking, even though it's on by default.

Instead, we'll now show -lock=false in all cases and document it as
_disabling_ the default locking.

While working on this I also noticed that the equivalent docs on the
website were differently inconsistent. I've not made them fully consistent
here but at least moreso than they were before.
2021-05-12 09:27:37 -07:00
Martin Atkins 874f1abb2b cli+website: -ignore-remote-version docs and other cleanup
We previously had only very short descriptions of what
-ignore-remote-version does due to having the documentation for it inline
on many different command pages and -help output.

Instead, we'll now centralize the documentation about this argument on
the remote backend page, and link to it or refer to it from all other
locations. This then allows us to spend more words on discussing what
Terraform normally does _without_ this option and warning about the
consequences of using it.

This continues earlier precedent for some local-backend-specific options
which we also don't recommend for typical use. While this does make these
options a little more "buried" than before, that feels justified given
that they are all "exceptional use only" sort of options where users ought
to learn about various caveats before using them.

While there I also took this opportunity to fix some earlier omissions
with the local-backend-specific options and a few other minor consistency
tweaks.
2021-05-12 09:27:37 -07:00
James Bardin 25cebfbd8b unmark decoded changes for jsonplan
Marks are not needed when transcoding changes from the internal msgpack
format to json.
2021-05-12 11:10:03 -04:00
James Bardin adf6ff7068 changes are now decoded with marks 2021-05-12 09:30:12 -04:00
Martin Atkins b402fd9d3a command/views: Remove command-specific runningInAutomation
We now have RunningInAutomation has a general concern in views.View, so
we no longer need to specify it for each command-specific constructor
separately.

For this initial change I focused only on changing the exported interface
of the views package and let the command-specific views go on having their
own unexported fields containing a copy of the flag because it made this
change less invasive and I wasn't feeling sure yet about whether we
ought to have code within command-specific views directly access the
internals of views.View. However, maybe we'll simplify this further in
a later commit if we conclude that these copies of the flag are
burdensome.

The general version of this gets set directly inside the main package,
which might at some future point allow us to make the command package
itself unaware of this "running in automation" idea and thus reinforce
that it's intended as a presentation-only thing rather than as a
behavioral thing, but we'll save more invasive refactoring for another
day.
2021-05-10 10:50:05 -07:00
Martin Atkins bf396b5f1b command/views: main View type aware if it's running in automation
This "running in automation" idea is a best effort thing where we try to
avoid printing out specific suggestions of commands to run in the main
workflow when the user is running Terraform inside a wrapper script or
other automation, because they probably don't want to bypass that
automation.

This just makes that information available to the main views.View type,
so we can then make use of it in the implementation of more specialized
view types that embed views.View.

However, nothing is using it as of this commit. We'll use it in later
commits.
2021-05-10 10:50:05 -07:00
Martin Atkins b38f3301d1 command/views: Remove baseState argument from plan-rendering views
In practice the current implementation isn't actually using this, and if
we need access to states in future we can access them in either the
plan.PriorState or plan.PrevRunState fields, depending on which stage we
want a state snapshot of.
2021-05-10 09:25:30 -07:00
James Bardin ced676d50d fixed configloader doesn't add unnecessary diags 2021-05-07 12:28:09 -04:00
Alisdair McDiarmid 91cdde1d67
Merge pull request #28608 from hashicorp/alisdair/json-plan-replace-paths
jsonplan: Add replace_paths
2021-05-06 08:23:09 -04:00
Martin Atkins 7c6e78bcb0 plans: Track both the previous run and prior states in the plan
Until now we've not really cared much about the state snapshot produced
by the previous Terraform operation, except to use it as a jumping-off
point for our refresh step.

However, we'd like to be able to report to an end-user whenever Terraform
detects a change that occurred outside of Terraform, because that's often
helpful context for understanding why a plan contains changes that don't
seem to have corresponding changes in the configuration.

As part of reporting that we'll need to keep track of the state as it
was before we did any refreshing work, so we can then compare that against
the state after refreshing. To retain enough data to achieve that, the
existing Plan field State is now two fields: PrevRunState and PriorState.

This also includes a very shallow change in the core package to make it
populate something somewhat-reasonable into this field so that integration
tests can function reasonably. However, this shallow implementation isn't
really sufficient for real-world use of PrevRunState because we'll
actually need to update PrevRunState as part of planning in order to
incorporate the results of any provider-specific state upgrades to make
the PrevRunState objects compatible with the current provider schema, or
else our diffs won't be valid. This deeper awareness of PrevRunState in
Terraform Core will follow in a subsequent commit, prior to anything else
making use of Plan.PrevRunState.
2021-05-05 15:11:05 -07:00
Kristin Laemmert 5f30efe857
command tests: plan and init (#28616)
* command/init: add test for reconfigure

* command/plan: adding tests

* command/apply: tests

* command: show and refresh tests
2021-05-05 14:13:20 -04:00
Alisdair McDiarmid a5b7394f9a command/jsonplan: Add replace_paths
The set of paths which caused a resource update to require replacement
has been stored in the plan since 0.15.0 (#28201). This commit adds a
simple JSON representation of these paths, allowing consumers of this
format to determine exactly which paths caused the resource to be
replaced.

This representation is intentionally more loosely encoded than the JSON
state serialization of paths used for sensitive attributes. Instead of a
path step being represented by an object with type and value, we use a
more-JavaScripty heterogenous array of numbers and strings. Any
practical consumer of this format will likely traverse an object tree
using the index operator, which should work more easily with this
format. It also allows easy prefix comparison for consumers which are
tracking paths.

While updating the documentation to include this new field, I noticed
that some others were missing, so added them too.
2021-05-04 16:51:51 -04:00
Alisdair McDiarmid 53fab10b26
Merge pull request #28598 from hashicorp/alisdair/fix-diagnostic-snippet-crash
cli: Fix diagnostic snippet crash
2021-05-04 13:01:58 -04:00
Kristin Laemmert 3679de0630
command/format: fix repetitive "unchanged attribute hidden" message (#28589)
writeNestedAttrDiff and writeAttrDiff were both printing the "unchanged attribute" message.  This removes one of the redundant prints.

Fixing this led me (in a very roundabout way) to realize that NestedType attributes were printing a sum total of unchanged attributes, including those in entirely unchanged elements, while *not* printing the total of unchanged elements. I added the necessary logic to count and print the number of unchanged elements for maps and lists.
2021-05-04 10:23:50 -04:00
Alisdair McDiarmid 25f99857cf cli: Fix crash with invalid JSON diagnostics
If a JSON diagnostic value has a highlight end offset which is before
the highlight start offset, this would previously panic. This commit
adds a normalization step to prevent the crash.
2021-05-04 08:35:42 -04:00
Alisdair McDiarmid 3a9b369b43 views/json: Fix diag crash with invalid highlight
Some diagnostic sources (I'm looking at you, HCL) fail to set the end of
the subject range. This is a bug in those code paths, but we can ensure
that we generate valid JSON diagnostics by checking for it here.

By doing so before the range normalization, we ensure that we generate a
unit width highlight whenever possible, so that at least something
useful is displayed.
2021-05-04 08:35:42 -04:00
Alisdair McDiarmid e9c9092c40 Fix missing trailing newline 2021-05-04 08:31:39 -04:00
Martin Atkins 1d3e34e35e command: New -replace=... planning option
This allows a similar effect to pre-tainting an object but does the action
within the context of a normal plan and apply, avoiding the need for an
intermediate state where the old object still exists but is marked as
tainted.

The core functionality for this was already present, so this commit is
just the UI-level changes to make that option available for use and to
explain how it contributed to the resulting plan in Terraform's output.
2021-05-03 15:43:23 -07:00
Alisdair McDiarmid 7f39f19ec7
Merge pull request #28584 from hashicorp/alisdair/add-reason-to-json-planned-change
command/views: Add reason to JSON planned change
2021-05-03 09:37:27 -04:00
Alisdair McDiarmid 7b23fa7877 command/views: Add reason to JSON planned change
Now that we have extra information about the reason for a given resource
action, include that in the JSON log output for planned changes.
2021-05-03 06:49:42 -04:00
Alisdair McDiarmid c95e9ada6b cli: Show forces replacement for sensitive attrs
When rendering a plan diff, sensitive resource attributes would
previously omit the "forces replacement" comment, which can lead to
confusion when the only reason for a resource being replaced is a
sensitive attribute.
2021-05-03 06:48:10 -04:00
Alisdair McDiarmid f72730a02b cli: Add JSON logs for operations commands 2021-04-30 11:37:36 -04:00
Martin Atkins b802237e03 plans: Track an optional extra "reason" for some planned actions
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.

For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.

This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.

So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
2021-04-29 17:50:46 -07:00
Alisdair McDiarmid 4b1ff4eda7
Merge pull request #28523 from hashicorp/alisdair/json-plan-sensitive-provider-attrs
core: Add sensitive provider attrs to JSON plan
2021-04-28 15:49:47 -04:00
Martin Atkins 89f986ded6 command+backend: generalized "plan mode"
So far we've only had "normal mode" and "destroy mode", where the latter
is activated either by "terraform plan -destroy" or "terraform destroy".

In preparation for introducing a third mode "refresh only" this
generalizes how we handle modes so we can potentially deal with an
arbitrary number of modes, although for now we only intend to have three.

Mostly this is just a different implementation of the same old behavior,
but there is one small user-visible difference here: the "terraform apply"
command now accepts a -destroy option, mirroring the option of the same
name on "terraform plan", which in turn makes "terraform destroy"
effectively a shorthand for "terraform apply -destroy".

This is intended to make us consistent that "terraform apply" without a
plan file argument accepts all of the same plan-customization options that
"terraform plan" does, which will in turn avoid us having to add a new
alias of "terraform plan" for each new plan mode we might add. The -help
output is changed in that vein here, although we'll wait for subsequent
commit to make a similar change to the website documentation just so we
can deal with the "refresh only mode" docs at the same time.
2021-04-27 08:23:54 -07:00
Martin Atkins c6a7d080d9 core: Generalize the idea of a "plan mode", vs just destroy flag
Previously there were only two planning modes: normal mode and destroy
mode. In that context it made sense for these to be distinguished only by
a boolean flag.

We're now getting ready to add our third mode, "refresh only". This
establishes the idea that planning can be done in one of a number of
mutually-exclusive "modes", which are related to but separate from the
various other options that serve as modifiers for the plan operation.

This commit only introduces the new plans.Mode type and replaces the
existing "destroy" flag with a variable of that type. This doesn't cause
any change in effective behavior because Terraform Core still supports
only NormalMode and DestroyMode, with NewContext rejecting an attempt to
create a RefreshMode context for now.

It is in retrospect a little odd that the "destroy" flag was part of
ContextOpts rather than just an argument to the Plan method, but
refactoring that would be too invasive a change for right now so we'll
leave this as a field of the context for now and save revisiting that for
another day.
2021-04-27 08:23:54 -07:00
Alisdair McDiarmid c89004d223 core: Add sensitive provider attrs to JSON plan
When rendering a stored plan file as JSON, we include a data structure
representing the sensitivity of the changed resource values. Prior to
this commit, this was a direct representation of the sensitivity marks
applied to values via mechanisms such as sensitive variables, sensitive
outputs, and the `sensitive` function.

This commit extends this to include sensitivity based on the provider
schema. This is in line with the UI rendering of the plan, which
considers these two different types of sensitivity to be equivalent.

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2021-04-27 10:29:34 -04:00
Kristin Laemmert f6af7b4f7a
lang/funcs: add (console-only) TypeFunction (#28501)
* lang/funcs: add (console-only) TypeFunction

The type() function, which is only available for terraform console,
prints out the type of a given value. This is mainly intended for
debugging - it's nice to be able to print out terraform's understanding
of a complex variable.

This introduces a new field for Scope: ConsoleMode. When ConsoleMode is true, any additional functions intended for use in the console (only) may be added.
2021-04-23 10:29:50 -04:00
Chris Arcand 15b6a1614c
Merge pull request #28479 from hashicorp/motd-for-tfc
New login success output for TFC/E
2021-04-22 09:13:56 -05:00