Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.
* states: move state.Locker to statemgr
The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.
* command: remove unnecessary use of state package for writing local terraform state files
* move state.LocalState into terraform package
state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.
* slight change of heart: move state.LocalState into clistate instead of
terraform
* unlock the state if Context() has an error, exactly as backend/remote does today
* terraform console and terraform import will exit before unlocking state in case of error in Context()
* responsibility for unlocking state in the local backend is pushed down the stack, out of backend.go and into each individual state operation
* add tests confirming that state is not locked after apply and plan
* backend/local: add checks that the state is unlocked after operations
This adds tests to plan, apply and refresh which validate that the state
is unlocked after all operations, regardless of exit status. I've also
added specific tests that force Context() to fail during each operation
to verify that locking behavior specifically.
A lingering FIXME caused missing configuration from provider config
blocks in the json output of terraform plan. This fixes the regression
and adds a test. For the sake of testing, I added an optional attribute
to the show test provider, which resulted in the providers schema test
getting an update - not a bad addition, but we can always add a
test-specific provider schema as needed.
If the user specifies a host that isn't a provider registry in a provider
source address then we'll print out some specialized error messages for
different variants of that situation.
In particular, this includes a special case for when the error is on the
hostname "github.com", in anticipation of folks incorrectly attempting to
use GitHub repository URLs (or Go-style module paths that happen to be
on GitHub) to specify providers, so we can give a more specific hint about
that.
This is just a different presentation of an existing error case that we
are already covering in the installer tests, so there are no new tests
here. We could in principle have a test covering the exact text of these
error messages, but we don't have much precedent for command package tests
covering that level of cosmetic detail.
For Terraform v0.12 we introduced a special loading mode where we would
use the 0.11-syntax-compatible "earlyconfig" package as a heuristic to
identify situations where it was likely that the user was trying to use
0.11-only syntax that the upgrade tool might help with.
However, as the language has moved on that is no longer a suitable
heuristic in Terraform 0.13 and later: other new additions to the
language can cause the main loader to disagree with earlyconfig, which
would lead us to give poor advice about how to respond.
Instead, we'll now return the same generic "there are errors" message in
all syntax error cases. We have an extra message for errors in this
case (as compared to other commands) because "terraform init" is usually
the first command a new user interacts with and so this message gives some
extra explanation about what "terraform init" will do with the
configuration once it's valid.
This also includes a reset control character in the output of the message
as part of our ongoing mission to stop Terraform printing out whole
paragraphs of colored text, which can often be hard to read for various
reasons.
After installing providers, we validate the presence of an executable
file, and generate a selected versions lockfile. If this process fails,
notify the user. One possible cause for this is an invalid provider
package with a missing or misnamed executable file.
Instead of searching the installed provider package directory for a
binary as we install it, we can lazily detect the executable as it is
required. Doing so allows us to separately report an invalid unpacked
package, giving the user more actionable error messages.
* command/console: return in case of errors before trying to unlock remote
state
The remote backend `Context` would exit without an active lock if there
was an error, while the local backend `Context` exited *with* a lock. This
caused a problem in `terraform console`, which would call unlock
regardless of error status.
This commit makes the local and remote backend consistently unlock the
state incase of error, and updates terraform console to check for errors
before trying to unlock the state.
* adding tests for remote and local backends
* command/init: return an error with invalid -backend-config files
The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.
Fixes#24845
* emphasize backend configuration file in docs
This is for consistency with other commands which use prompts, all of
which require "yes" rather than "y" to confirm.
We also migrate the login command to use UIInput, which now supports
securely asking for passwords or secrets via the speakeasy library.
* internal/getproviders: decode and return any registry warnings
The public registry may include a list of warnings in the "versions"
response for any given provider. This PR adds support for warnings from
the registry and an installer event to return those warnings to the
user.
Previously, any comments inside the required provider configuration for
a given provider would be wiped out upon rerunning the 0.13upgrade
command. This commit attempts to preserve those comments if the existing
entry is semantically equivalent to the entry we are about to write.
* command: adjust exit code of state rm and state mv
Commands `state rm` and `state mv` will now exit with code 1 when the
target resource is not found in the current state.
This is consistent with `terraform state show non_existent_resource`.
Fixes#17800
We previously intentionally removed support for the allow-missing-config
option to terraform import, requiring that all imported resources have
matching config. See #24412.
However, the option was not removed from the import command, and it is
widely used. This commit reintroduces support for importing with a
missing configuration by falling back to implying the provider FQN based
on the resource type.
When using `-flag=value` with Powershell, unquoted values are broken
into separate arguments. This means that the following command:
terraform init -backend-config=./backend.conf
is interpreted by Terraform as:
terraform init -backend-config= ./backend.conf
This results in an empty backend-config setting (which is semantically
valid!) followed by a custom configuration path (pointing at a file).
Due to a bug where we could exit without printing diagnostics, this
would result in a silent failure that was very difficult to diagnose.
Some of the tests in the command package were running directly on the
fixture directories, and modifying or locking files within them. This
could cause state to leak between tests.
This commit cleans up all such cases that I could find.
We globally support a -v/-version/--version flag, which triggers the
version subcommand. The recent introduction of JSON output support meant
we started parsing the flags for the first time, but we didn't add flags
for these global version arguments.
This commit adds those flags (but doesn't check them, since they have no
effect on the version command itself). Also adds usage information for
terraform version.
The positional argument passed to apply was once used to specify a
source for a Terraform module to fetch and initialize (#337). This
functionality was removed from the init command later (#15032) but not
completely removed from apply.
This code was non-functional but largely not harmful, except for a very
specific case: when passing an absolute path to a plan file as the
positional argument on Windows, the getter.Detect code would incorrectly
interpret the path as a URL. This caused init to fail and the apply
command would exit with code 1 but without diagnostics.
This commit removes this codepath, which fixes this bug, and should
otherwise have no effect on the supported behaviour of apply.
This new version permits omitting the space between the operator and the
boundary in a ruby-style version constraint, like ">1.0.0" instead of
"> 1.0.0".
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.
This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.
For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.
This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
I feel the current confirmation prompt for 0.13upgrade command is
ambiguous what is expected. Actually, when I used it for the first time,
I cancelled it by typing `y` instead of `yes`.
I believe it would be great if the 0.13upgrade command tell us the
expected value for confirmation like 0.12upgrade.
Diagnostics where the highlight range has an empty overlap with a line
would skip lines of the output. This is because if two ranges abut each
other, they can be considered to overlap, but that overlap is empty.
This results in an edge case in the diagnostic printer which causes the
line not to be printed.
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.
This initial version is not customizable aside from being able to select
multiple platforms to install packages for.
Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.
In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.
This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".
This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.
The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.
We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.
This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
Fetching a default namespace provider from the public registry can
result in 404 Not Found error. This might be caused by a previously-
default provider moving to a new namespace, which means that the
configuration needs to be upgraded to use an explicit provider source.
This commit adds a more detailed diagnostic for this situation,
suggesting that the intended provider might be in a new namespace. The
recommended course of action is to run the 0.13upgrade command to
generate the correct required_providers configuration.
Kristin Laemmert