So far we've only had "normal mode" and "destroy mode", where the latter
is activated either by "terraform plan -destroy" or "terraform destroy".
In preparation for introducing a third mode "refresh only" this
generalizes how we handle modes so we can potentially deal with an
arbitrary number of modes, although for now we only intend to have three.
Mostly this is just a different implementation of the same old behavior,
but there is one small user-visible difference here: the "terraform apply"
command now accepts a -destroy option, mirroring the option of the same
name on "terraform plan", which in turn makes "terraform destroy"
effectively a shorthand for "terraform apply -destroy".
This is intended to make us consistent that "terraform apply" without a
plan file argument accepts all of the same plan-customization options that
"terraform plan" does, which will in turn avoid us having to add a new
alias of "terraform plan" for each new plan mode we might add. The -help
output is changed in that vein here, although we'll wait for subsequent
commit to make a similar change to the website documentation just so we
can deal with the "refresh only mode" docs at the same time.
This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.
The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.
Changes in support of this:
- Some instances of direct error output have been replaced with
diagnostics, most notably in the emergency state backup handler. This
requires reformatting the error messages to allow the diagnostic
renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
set it on the `backend.Operation` struct, as these are the only code
paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
views package, so there is a stub implementation of a `views.Show`
interface there.
Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.
This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:
- Move the responsibility of creating the appropriate Locker to the
source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
continue to misuse the -lock-timeout flag to cancel pending runs.
When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.
This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.
To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.
In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.
Some secondary consequences of this change:
- The `ReportResult` method on the backend is now moved to the
`Operation` type, as it needs to access the `ShowDiagnostics` callback
(and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
`cli.Ui` instance are migrated to using a new record/playback diags
helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
struct's `showDiagnostics` method.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
This dramatically simplifies the logic around auto-approve, which is
nice.
Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
pathorcontents was solely used by the gcs backend. I moved the function
into the backend package so it could still be used by other backends for
good measure.
* remove unused code
I've removed the provider-specific code under registry, and unused nil
backend, and replaced a call to helper from backend/oss (the other
callers of that func are provisioners scheduled to be deprecated).
I also removed the Dockerfile, as our build process uses a different
file.
Finally I removed the examples directory, which had outdated examples
and links. There are better, actively maintained examples available.
* command: remove various unused bits
* test wasn't running
* backend: remove unused err
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.
* states: move state.Locker to statemgr
The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.
* command: remove unnecessary use of state package for writing local terraform state files
* move state.LocalState into terraform package
state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.
* slight change of heart: move state.LocalState into clistate instead of
terraform
Terraform Core expects all variables to be set, but for some ancillary
commands it's fine for them to just be set to placeholders because the
variable values themselves are not key to the command's functionality
as long as the terraform.Context is still self-consistent.
For such commands, rather than prompting for interactive input for
required variables we'll just stub them out as unknowns to reflect that
they are placeholders for values that a user would normally need to
provide.
This achieves a similar effect to how these commands behaved before, but
without the tendency to produce a slightly invalid terraform.Context that
would fail in strange ways when asked to run certain operations.
This mirrors the change made for providers, so that default values can
be inserted into the config by the backend implementation. This is only
the interface and method name changes, it does not yet add any default
values.
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.
The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.
The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.
Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.
A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.
Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
In some cases this is needed to keep the UX clean and to make sure any remote exit codes are passed through to the local process.
The most obvious example for this is when using the "remote" backend. This backend runs Terraform remotely and stream the output back to the local terminal.
When an error occurs during the remote execution, all the needed error information will already be in the streamed output. So if we then return an error ourselves, users will get the same errors twice.
By allowing the backend to specify the correct exit code, the UX remains the same while preserving the correct exit codes.
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.
To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.
This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.
Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.
Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.
What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.
Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.
In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
A common reason to want to use `terraform plan` is to have a chance to
review and confirm a plan before running it. If in fact that is the
only reason you are running plan, this new `terraform apply -auto-approve=false`
flag provides an easier alternative to
P=$(mktemp -t plan)
terraform refresh
terraform plan -refresh=false -out=$P
terraform apply $P
rm $P
The flag defaults to true for now, but in a future version of Terraform it will
default to false.
Add fields required to create an appropriate context for all calls to
clistate.Lock.
Add missing checks for Meta.stateLock, where we would attempt to lock,
even if locking should be skipped.
What will hopfully be the final version of the Backend interface. This
combines the MultiState interface into Backend since it will be required
to implement, and simplifies the interface because the Backend is no
longer responsible for tracking the current state.
Split the interface to change environments out from the minimal Backend
interface, to make it optional for backend implementations. If
backend.MultiState isn't implemented, return a "not implemented" from
environment related methods.
Have the Local backend delegate the MultiState methods to the proper
backend.
Backends are a mechanism that allow abstracting the behavior of
Terraform CLI from the actual core. This allows us to slip in special
behavior such as state loading, remote operations, etc.
Martin Atkins