Commit Graph

28973 Commits

Author SHA1 Message Date
James Bardin 9708e900eb
Merge pull request #28736 from hashicorp/jbardin/pg
pg backend requires PostgreSQL 10
2021-05-18 09:58:49 -04:00
James Bardin 51a171c7f4 pg requires PostgreSQL 10 2021-05-18 09:39:05 -04:00
Chris Arcand 2171f8a1a6 backend/remote: Support replacements in plans 2021-05-17 16:31:34 -07:00
Chris Arcand a436c7fa2f backend/remote: Support refresh-only plans 2021-05-17 16:31:33 -07:00
Chris Arcand cb49a4c8d6 backend/remote: Remove the -refresh=false restriction 2021-05-17 16:31:32 -07:00
James Bardin e39ba2816b
Merge pull request #28724 from hashicorp/jbardin/apply-diags
save apply diagnostics on backend failure
2021-05-17 17:27:40 -04:00
James Bardin 4d55e2604f save apply diagnostics on backend failure
Make sure to collect apply diagnostics before persisting the backend
state so runtime errors are not lost.
2021-05-17 17:22:04 -04:00
Martin Atkins d35bc05312 Update various non-code references to renamed Go packages
Previous commits have moved all of our Go packages under "internal", so
this is a retroactive update of non-Go references to those same packages.
2021-05-17 14:09:07 -07:00
Martin Atkins 36d0a50427 Move terraform/ to internal/terraform/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins f40800b3a4 Move states/ to internal/states/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins b40a4fb741 Move plugin/ and plugin6/ to internal/plugin{,6}/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 034e944070 Move plans/ to internal/plans/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins cdd9464f9a Move lang/ to internal/lang/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 31349a9c3a Move configs/ to internal/configs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins ffe056bacb Move command/ to internal/command/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins b9a93a0fe7 Move addrs/ to internal/addrs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 26e5320a78 Move experiments/ to internal/experiments/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins a6b8c8ba22 Move moduledeps/ to internal/moduledeps/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins e81162c4e1 Move provisioners/ to internal/provisioners/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 1409f30f9c Move providers/ to internal/providers/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins bc31dbc179 Move instances/ to internal/instances/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 05caff2ca3 Move tfdiags/ to internal/tfdiags/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins ec85fb1960 Move communicator/ to internal/communicator/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 4c254cc2be Move httpclient/ to internal/httpclient/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 415ab31db8 Move e2e/ to internal/e2e/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 4c5b866dea Move registry/ to internal/registry/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 2bd2568ad8 Move repl/ to internal/repl/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 70eebe3521 Move dag/ to internal/dag/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins cf93aa8046 Move builtin/ to internal/builtin/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 73dda868cc Move backend/ to internal/backend/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
James Bardin f5e0d13079
Merge pull request #28718 from hashicorp/jbardin/backend-migrate
Prevent automatic backend migration during `terraform init`
2021-05-17 14:42:35 -04:00
James Bardin b11578da45 prevent -migrate-state with -reconfigure 2021-05-17 14:28:34 -04:00
Martin Atkins 8744f0e8b8 addrs: Reserve "template" and similar prefixes for future expansion
At the time of this commit we have a proposal #28700 which would, if
accepted, need to reserve a new reference prefix to represent template
arguments.

It seems unlikely that the proposal would be accepted and implemented
before Terraform v1.0 creates additional compatibility constraints, and so
this pre-emptively reserves a few candidate symbol names to allow
something like that proposal to potentially move forward later without
requiring a new opt-in language edition.

If we do move forward with the proposal then we'll select one of these
three reserved names depending on which form of the proposal we decide
to move forward with, and then un-reserve the other two. If we decide to
not pursue this proposal at all then we'll un-reserve all three once
that decision is finalized.

It's unlikely that there is any existing provider which has a resource
type named either "template", "lazy", or "arg", but in that unlikely event
users of that provider can keep using it by adding the "resource."
escaping prefix, such as changing "lazy.foo.bar" into
"resource.lazy.foo.bar".
2021-05-17 11:17:25 -07:00
Martin Atkins f09b090d97 lang: Allow "resource." prefix as another way to refer to resources
The current way to refer to a managed resource is to use its resource type
name as a top-level symbol in the reference. This is convenient and makes
sense given that managed resources are the primary kind of object in
Terraform.

However, it does mean that an externally-extensible namespace (the set
of all possible resource type names) overlaps with a reserved word
namespace (the special prefixes like "path", "var", etc), and thus
introducing any new reserved prefix in future risks masking an existing
resource type so it can't be used anymore.

We only intend to introduce new reserved symbols as part of future
language editions that each module can opt into separately, and when doing
so we will always research to try to choose a name that doesn't overlap
with commonly-used providers, but not all providers are visible to us and
so there is always a small chance that the name we choose will already be
in use by a third-party provider.

In preparation for that event, this introduces an alternative way to refer
to managed resources that mimics the reference style used for data
resources: resource.type.name . When using this form, the second portion
is _always_ a resource type name and never a reserved word.

There is currently no need to use this because all of the already-reserved
symbol names are effectively blocked from use by existing Terraform
versions that lack this escape hatch. Therefore there's no explicit
documentation about it yet.

The intended use for this is that a module upgrade tool for a future
language edition would detect references to resource types that have now
become reserved words and add the "resource." prefix to keep that
functionality working. Existing modules that aren't opted in to the new
language edition would keep working without that prefix, thus keeping to
compatibility promises.
2021-05-17 11:17:25 -07:00
Martin Atkins 27ad9861ce configs: Meta-argument escaping blocks
Several top-level block types in the Terraform language have a body where
two different schemas are overlayed on top of one another: Terraform first
looks for "meta-arguments" that are built into the language, and then
evaluates all of the remaining arguments against some externally-defined
schema whose content is not fully controlled by Terraform.

So far we've been cautiously adding new meta-arguments in these namespaces
after research shows us that there are relatively few existing providers
or modules that would have functionality masked by those additions, but
that isn't really a viable path forward as we prepare to make stronger
compatibility promises.

In an earlier commit we've introduced the foundational parts of a new
language versioning mechanism called "editions" which should allow us to
make per-module-opt-in breaking changes in the future, but these shared
namespaces remain a liability because it would be annoying if adopting a
new edition made it impossible to use a feature of a third-party provider
or module that was already using a name that has now become reserved in
the new edition.

This commit introduces a new syntax intended to be a rarely-used escape
hatch for that situation. When we're designing new editions we will do our
best to choose names that don't conflict with commonly-used providers and
modules, but there are many providers and modules that we cannot see and
so there is a risk that any name we might choose could collide with at
least one existing provider or module. The automatic migration tool to
upgrade an existing module to a new edition should therefore detect that
situation and make use of this escaping block syntax in order to retain
the existing functionality until all the called providers or modules are
updated to no longer use conflicting names.

Although we can't put in technical constraints on using this feature for
other purposes (because we don't know yet what future editions will add),
this mechanism is intentionally not documented for now because it serves
no immediate purpose. In effect, this change is just squatting on the
syntax of a special block type named "_" so that later editions can make
use of it without it _also_ conflicting, creating a confusing nested
escaping situation. However, the first time a new edition actually makes
use of this syntax we should then document alongside the meta-arguments
so folks can understand the meaning of escaping blocks produced by
edition upgrade tools.
2021-05-17 11:17:25 -07:00
James Bardin 91a8a8137c
Merge pull request #28699 from hashicorp/jbardin/plugin-output-monitor
monitor plugin std outputs for unexpected data
2021-05-17 12:52:53 -04:00
James Bardin 1b48636b42 update init docs for -migrate-state 2021-05-17 12:41:54 -04:00
James Bardin edc2695d18 no longer automatically attempt state migration
Add `init -migrate-state` flag to indicate automatic state migration is
desired. This flag will be implied by the `-force-copy` flag, since that
would indicate state migration is expected.

If `init` encounters a change to the stored backend configuration, it
will now always return an error when neither `-reconfigure` or
`-migrate-state` is supplied.

Turn the most common legacy output strings into diagnostics, removing
the "see above text" error output.
2021-05-17 12:41:54 -04:00
Alisdair McDiarmid 73d07e28c0
Merge pull request #28296 from upodroid/gcs-imp-v2
backend/gcs Rework Service Account Impersonation
2021-05-17 11:42:11 -04:00
Brian Flad 4b25a07bc7 Remove GitHub HashiBot configuration
For the `closed_issue_locker` behavior, this is a migration to an equivalent action.

For the `label_issue_migrater` behavior, this is not replaced and instead it is suggested to use native GitHub functionality for issue transfer. If mostly equivalent behavior is desired via label automation, it may be possible to submit an issue transfer feature request to dessant/label-actions as it is a popular community action or create a new GitHub Action. Please reach out if this is a major issue for your team.

For the `remove_labels_on_reply` behavior, it is equivalent except this initial configuration does not make the collaborators distinction. There is a workflow configuration workaround for setting up per-user ignores for any job/step, so if you desire that here please reach out.
2021-05-14 13:28:15 -07:00
Nick Fagerlund 65f3ddec52 website: Make apply's usage of plan options harder to miss
- I'm using distinct subheaders and smaller paragraphs to try and make the info
  about apply's two modes more skimmable.

- I'm also adding a separate "Plan Options" subheader (and keeping the section
  tiny so it stays snugged up right next to the "Apply Options" one) to make it
  extra-clear that Hey, There's More Options, They're Over There.
2021-05-14 13:26:33 -07:00
Martin Atkins 358fb54f75 core: "Did you mean" hint for missing data. prefix in references
It's a relatively common mistake to try to refer to a data resource
without including the data. prefix, making Terraform understand it as a
reference to a managed resource.

To help with that case, we'll include an additonal suggestion if we can
see that there's a data resource declared with the same type and name as
in the given address.
2021-05-14 11:38:13 -07:00
Alisdair McDiarmid 3e40a9a4eb
Merge pull request #28507 from stevematney/patch-1
Updating sensitive/nonsensitive docs with v0.14 specifics.
2021-05-14 13:46:12 -04:00
Steve Matney e27a927ba4 Updating sensitive and nonsensitive docs with correct v0.15 info. 2021-05-14 10:32:39 -06:00
James Bardin feb7622846 monitor plugin std outputs for unexpected data
Once a plugin process is started, go-plugin will redirect the stdout and
stderr stream through a grpc service and provide those streams to the
client. This is rarely used, as it is prone to failing with races
because those same file descriptors are needed for the initial handshake
and logging setup, but data may be accidentally sent to these
nonetheless.

The usual culprits are stray `fmt.Print` usage where logging was
intended, or the configuration of a logger after the os.Stderr file
descriptor was replaced by go-plugin. These situations are very hard for
provider developers to debug since the data is discarded entirely.

While there may be improvements to be made in the go-plugin package to
configure this behavior, in the meantime we can add a simple monitoring
io.Writer to the streams which will surface th data as warnings in the
logs instead of writing it to `io.Discard`
2021-05-13 16:57:36 -04:00
James Bardin ef88c54604
Merge pull request #28687 from hashicorp/jbardin/sensitive-changes
Decode change values with marks
2021-05-13 12:44:03 -04:00
Martin Atkins 917309fb5a command/diff: Small additional context about deposed objects
In the very unusual situation where we end up planning to destroy a
deposed object alone, it's likely that we're exposing users to this idea
of "deposed" for the very first time.

This additional sentence will hopefully give some extra context for what
that means. We don't really have room here for a lengthy explanation about
how deposed objects come to exist but this will hopefully be enough of
a hook to help users connect this with an error they saw on a previous
run, or at least to have some additional keywords to search for if they
want to research further.
2021-05-13 09:05:06 -07:00
Martin Atkins f2adfb6e2a core: Treat deposed objects the same as orphaned current objects
In many ways a deposed object is equivalent to an orphaned current object
in that the only action we can take with it is to destroy it. However, we
do still need to take some preparation steps in both cases: first, we must
ensure we track the upgraded version of the existing object so that we'll
be able to successfully render our plan, and secondly we must refresh the
existing object to make sure it still exists in the remote system.

We were previously doing these extra steps for orphan objects but not for
deposed ones, which meant that the behavior for deposed objects would be
subtly different and violate the invariants our callers expect in order
to display a plan. This also created the risk that a deposed object
already deleted in the remote system would become "stuck" because
Terraform would still plan to destroy it, which might cause the provider
to return an error when it tries to delete an already-absent object.

This also makes the deposed object planning take into account the
"skipPlanChanges" flag, which is important to get a correct result in
the "refresh only" planning mode.

It's a shame that we have almost identical code handling both the orphan
and deposed situations, but they differ in that the latter must call
different functions to interact with the deposed rather than the current
objects in the state. Perhaps a later change can improve on this with some
more refactoring, but this commit is already a little more disruptive than
I'd like and so I'm intentionally deferring that for another day.
2021-05-13 09:05:06 -07:00
Martin Atkins 3c8a4e6e05 command+backend/local: -refresh-only and drift detection
This is a light revamp of our plan output to make use of Terraform core's
new ability to report both the previous run state and the refreshed state,
allowing us to explicitly report changes made outside of Terraform.

Because whether a plan has "changes" or not is no longer such a
straightforward matter, this now merges views.Operation.Plan with
views.Operation.PlanNoChanges to produce a single function that knows how
to report all of the various permutations. This was also an opportunity
to fill some holes in our previous logic which caused it to produce some
confusing messages, including a new tailored message for when
"terraform destroy" detects that nothing needs to be destroyed.

This also allows users to request the refresh-only planning mode using a
new -refresh-only command line option. In that case, Terraform _only_
performs drift detection, and so applying a refresh-only plan only
involves writing a new state snapshot, without changing any real
infrastructure objects.
2021-05-13 09:05:06 -07:00
Martin Atkins ce69c3903f command/views: Show refresh-detected changes as part of a plan
We've always had a mechanism to synchronize the Terraform state with
remote objects before creating a plan, but we previously kept the result
of that to ourselves, and so it would sometimes lead to Terraform
generating a planned action to undo some upstream drift, but Terraform
would give no context as to why that action was planned even though the
relevant part of the configuration hadn't changed.

Now we'll detect any differences between the previous run state and the
refreshed state and, if any managed resources now look different, show
an additional note about it as extra context for the planned changes that
follow.

This appears as an optional extra block of information before the normal
plan output. It'll appear the same way in all of the contexts where we
render plans, including "terraform show" for saved plans.
2021-05-13 09:05:06 -07:00