When migrating state to a new workspace, the version check would error
due to a 404 error on fetching the workspace record. This would result
in failed state migration.
Instead we should look specifically for a 404 error, and allow migration
to continue. If we're just about to create the workspace, there can't be
a version incompatibility problem.
When using the -lock-timeout option with the remote backend configured
in local operations mode, Terraform would fail to retry acquiring the
lock. This was caused by the lock error message having a missing Info
field, which the state manager requires to be present in order to
attempt retries.
This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.
The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.
Changes in support of this:
- Some instances of direct error output have been replaced with
diagnostics, most notably in the emergency state backup handler. This
requires reformatting the error messages to allow the diagnostic
renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
set it on the `backend.Operation` struct, as these are the only code
paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
views package, so there is a stub implementation of a `views.Show`
interface there.
Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
* Remove deprecation on undeclared variable
Remove deprecation and add docs specific to the behavior around
undeclared variable values
* Limit full warnings to 2 instances, then summary
This way, the third warning is a summary, rather than the fourth
warning being the summary
* providers.Interface: huge renamification
This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.
One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.
Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
The warning diag added when refreshing an empty state file was never
rendered, and instead a custom (and incorrect) warning was output to the
UI. This commit fixes the dropped diag and removes the custom warning.
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.
This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:
- Move the responsibility of creating the appropriate Locker to the
source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
continue to misuse the -lock-timeout flag to cancel pending runs.
When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.
This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.
To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.
In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.
Some secondary consequences of this change:
- The `ReportResult` method on the backend is now moved to the
`Operation` type, as it needs to access the `ShowDiagnostics` callback
(and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
`cli.Ui` instance are migrated to using a new record/playback diags
helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
struct's `showDiagnostics` method.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
This dramatically simplifies the logic around auto-approve, which is
nice.
Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
CountHook is an implementation of terraform.Hook which is used to
calculate how many resources were added, changed, or destroyed during an
apply. This hook was previously injected in the local backend code,
which means that the apply command code has no access to these counts.
This commit moves the CountHook code into the command package, and
removes an unused instance of the hook in the plan code path. The goal
here is moving UI code into the command package.
We now require the output to accept UTF-8 and we can determine how wide
the terminal (if any) is, so here we begin to make use of that for the
"terraform plan" command.
The horizontal rule is now made of box drawing characters instead of
hyphens and fills the whole terminal width.
The paragraphs of text in the output are now also wrapped to fill the
terminal width, instead of the hard-wrapping we did before.
This is just a start down the road of making better use of the terminal
capabilities. Lots of other commands could benefit from updates like these
too.
Here we propagate in the initialized terminal.Streams from package main,
and then onwards to backends running in CLI mode.
This also replaces our use of helper/wrappedstreams to determine whether
stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect
file descriptors on Windows, causing StdinPiped to always return false on
that platform and thus causing one of the odd behaviors discussed in
Finally, this includes some wrappers around the ability to look up the
number of columns in the terminal in preparation for use elsewhere. These
wrappers deal with the fact that our unit tests typically won't populate
meta.Streams.
If the remote backend is connected to a Terraform Cloud workspace in
local operations mode, we disable the version check, as the remote
Terraform version is meaningless.
Terraform remote version conflicts are not a concern for operations. We
are in one of three states:
- Running remotely, in which case the local version is irrelevant;
- Workspace configured for local operations, in which case the remote
version is meaningless;
- Forcing local operations with a remote backend, which should only
happen in the Terraform Cloud worker, in which case the Terraform
versions by definition match.
This commit therefore disables the version check for operations (plan
and apply), which has the consequence of disabling it in Terraform Cloud
and Enterprise runs. In turn this enables Terraform Enterprise runs with
bundles which have a version that doesn't exactly match the bundled
Terraform version.
Terraform Cloud/Enterprise support a pseudo-version of "latest" for the
configured workspace Terraform version. If this is chosen, we abandon
the attempt to verify the versions are compatible, as the meaning of
"latest" cannot be predicted.
This affects both the StateMgr check (used for commands which execute
remotely) and the full version check (for local commands).
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.
This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.
To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.
Terraform version compatibility is defined as:
- For all releases before 0.14.0, local must exactly equal remote, as
two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
we will not change the state version number in a patch release.
If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.
Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.
In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
The remote backend tests spent most of their execution time sleeping in
various polling and backoff waits. This is unnecessary when testing
against a mock server, so reduce all of these delays when under test to
much lower values.
Only one remaining test has an artificial delay: verifying the discovery
of services against an unknown hostname. This times out at DNS
resolution, which is more difficult to fix than seems worth it at this
time.
Until now the default workspace for every project would have the ID 1,
which would make it impossible to lock them at the same time since we
use the ID to identify the lock. With a global sequence to generate the
IDs, the default workspace will now have a different ID for each project
and it will be possible to lock multiple unrelated projects at the same
time.
If an old version of Terraform tries to get the lock on a project created
with this new version it will work as we continue to use the ID of the
workspace, we just change the way we generate them.
If this version tries to get a lock on a project created by an old
version of Terraform it will work as usual, but we may encounter a
conflict with another unrelated project. This is already the current
behavior so it's not an issue to persist this behavior. As users migrate
to an up-to-date version of Terraform this will stop.
Projects already present in the database will keep their conflicting IDs,
I did not wanted to change them as users may be reading the states
directly in the database for some reason. They can if they want change
them manually to remove conflicts, newly created projects will work
without manual intervention.
Closes https://github.com/hashicorp/terraform/issues/22833
James Bardin