Commit Graph

220 Commits

Author SHA1 Message Date
Alisdair McDiarmid 68558ccd54 backend/local: Replace CLI with view instance
This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.

The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.

Changes in support of this:

- Some instances of direct error output have been replaced with
  diagnostics, most notably in the emergency state backup handler. This
  requires reformatting the error messages to allow the diagnostic
  renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
  view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
  set it on the `backend.Operation` struct, as these are the only code
  paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
  views package, so there is a stub implementation of a `views.Show`
  interface there.

Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
2021-02-18 12:08:08 -05:00
Kristin Laemmert f6505870cc
Mildwonkey/providers interface renaming (#27805)
* providers.Interface: huge renamification

This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.

One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.

Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
2021-02-18 10:13:43 -05:00
Alisdair McDiarmid 6375c6ce6b
Merge pull request #27787 from hashicorp/alisdair/command-views-state-locker
clistate: Update clistate.Locker for command views
2021-02-16 17:37:18 -05:00
Alisdair McDiarmid 1ae3d30383
Merge pull request #27760 from hashicorp/alisdair/command-views-ui-hook
cli: Migrate Terraform UI hook to command views
2021-02-16 09:36:47 -05:00
Alisdair McDiarmid 3257f31aa7 backend/local: Return diag for refresh empty state
The warning diag added when refreshing an empty state file was never
rendered, and instead a custom (and incorrect) warning was output to the
UI. This commit fixes the dropped diag and removes the custom warning.
2021-02-16 07:23:31 -05:00
Alisdair McDiarmid 2d1976bbda clistate: Update clistate.Locker for command views
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.

This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:

- Move the responsibility of creating the appropriate Locker to the
  source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
  method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
  view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
  errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
  continue to misuse the -lock-timeout flag to cancel pending runs.

When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.

This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
2021-02-16 07:19:22 -05:00
Alisdair McDiarmid a7b7cd29fc cli: Migrate Terraform UI hook to command views
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.

To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-16 07:18:22 -05:00
Alisdair McDiarmid 536c80da23 backend: Add per-operation diagnostic rendering
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.

In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.

Some secondary consequences of this change:

- The `ReportResult` method on the backend is now moved to the
  `Operation` type, as it needs to access the `ShowDiagnostics` callback
  (and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
  `cli.Ui` instance are migrated to using a new record/playback diags
  helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
  struct's `showDiagnostics` method.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-12 14:30:35 -05:00
Alisdair McDiarmid b2ba650c21 cli: Remove deprecated destroy -force flag
This dramatically simplifies the logic around auto-approve, which is
nice.

Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
2021-02-03 15:05:05 -05:00
Alisdair McDiarmid 5ca118b4e6 cli: Move resource count code to command package
CountHook is an implementation of terraform.Hook which is used to
calculate how many resources were added, changed, or destroyed during an
apply. This hook was previously injected in the local backend code,
which means that the apply command code has no access to these counts.

This commit moves the CountHook code into the command package, and
removes an unused instance of the hook in the plan code path. The goal
here is moving UI code into the command package.
2021-01-29 15:29:35 -05:00
Jonathan Hall 49ee3d3ef8 Grammar nit: "setup" as a verb should be spelled "set up" 2021-01-26 20:39:11 +01:00
Martin Atkins e6a516d87e backend/local: Use terminal properties to tweak the plan output
We now require the output to accept UTF-8 and we can determine how wide
the terminal (if any) is, so here we begin to make use of that for the
"terraform plan" command.

The horizontal rule is now made of box drawing characters instead of
hyphens and fills the whole terminal width.

The paragraphs of text in the output are now also wrapped to fill the
terminal width, instead of the hard-wrapping we did before.

This is just a start down the road of making better use of the terminal
capabilities. Lots of other commands could benefit from updates like these
too.
2021-01-13 15:37:04 -08:00
Martin Atkins d2c3403ab6 command: Use the new terminal.Streams object
Here we propagate in the initialized terminal.Streams from package main,
and then onwards to backends running in CLI mode.

This also replaces our use of helper/wrappedstreams to determine whether
stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect
file descriptors on Windows, causing StdinPiped to always return false on
that platform and thus causing one of the odd behaviors discussed in

Finally, this includes some wrappers around the ability to look up the
number of columns in the terminal in preparation for use elsewhere. These
wrappers deal with the fact that our unit tests typically won't populate
meta.Streams.
2021-01-13 15:37:04 -08:00
James Bardin 3bc7d77230 update MockProvider usage 2021-01-12 17:47:55 -05:00
James Bardin e614fb9aed refresh is expected for destroy
These tests were not previously running a refresh, and hence did not
expect the resources to be read.
2021-01-08 13:29:54 -05:00
Ben Drucker a39273cfa3 Merge remote-tracking branch 'origin/master' into validate-ignore-empty-provider 2020-12-14 14:39:48 -08:00
James Bardin 7eac9e1d89 set SkipRefresh for plan and apply
The option is set in the same place for plan and apply.
2020-12-10 09:47:13 -05:00
Ben Drucker 2549e53aed add backend refresh test with provider config 2020-12-06 10:02:26 -08:00
Ben Drucker 7e11b97923 Merge remote-tracking branch 'origin/master' into validate-ignore-empty-provider 2020-12-06 09:47:24 -08:00
James Bardin d2c2d58f09 unused 2020-12-02 13:59:19 -05:00
James Bardin de5b022a3b legacy types in backend/local 2020-12-02 12:33:18 -05:00
James Bardin 8e7a9b6312 output test for plan with no root output changes
Module outputs do not show up in the plan, and are not rendered in the
UI.
2020-11-17 16:11:57 -05:00
Ben Drucker afe8b67b95 Merge remote-tracking branch 'origin/master' into validate-ignore-empty-provider
# Conflicts:
#	terraform/eval_validate.go
2020-11-09 16:24:49 -08:00
Alisdair McDiarmid b335918c3c backend: Only show root module output changes
When rendering planned output changes, we need to filter the plan's
output changes to ensure that only root module outputs which have
changed are rendered. Otherwise we will render changes for submodule
outputs, and (with concise diff disabled) render unchanged outputs also.
2020-11-02 10:24:22 -05:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
James Bardin 5eca0788c6 rely solely on the plan changes for outputs
Now that outputs changes are tracked in full, we can remove the
comparisons with the prior state and use the planned changes directly.
2020-10-12 18:59:14 -04:00
James Bardin 103a6cf2db update mock provider call 2020-10-08 13:52:04 -04:00
James Bardin 826ccdd123 re-enable test 2020-10-07 10:44:41 -04:00
James Bardin 37569f5cc3 insert PlanRefresh into the context 2020-09-24 09:34:49 -04:00
James Bardin bc82347a04 fix tests
Update tests to match the new behavior. Some were incorrect, some no
longer make sense, and some just weren't setup to handle th plan api
calls.
2020-09-21 16:17:46 -04:00
James Bardin 312317abd0 wrong instance key in test state
This was never picked up by the tests until now
2020-09-17 09:55:00 -04:00
James Bardin 8658424059 skip plan with no refresh test
We still need to determine if `-refresh=false` is even useful with the
new planning strategy.
2020-09-17 09:55:00 -04:00
James Bardin f52d836e0a fix local backend tests to match new behavior
Leaving plan with -refresh=false tests failing for now.
2020-09-17 09:55:00 -04:00
James Bardin be757bd416 Refresh instances during plan
This change refreshes the instance state during plan, so a complete
Refresh no longer needs to happen before Plan.
2020-09-17 09:54:59 -04:00
Alisdair McDiarmid 09d8355f43 command: Add experimental concise diff renderer
When rendering a diff between current state and projected state, we only
show resources and outputs which have changes. However, we show a full
structural diff for these values, which includes all attributes and
blocks for a changed resource or output. The result can be a very long
diff, which makes it difficult to verify what the changed fields are.

This commit adds an experimental concise diff renderer, which suppresses
most unchanged fields, only displaying the most relevant changes and
some identifying context. This means:

- Always show all identifying attributes, initially defined as `id`,
  `name`, and `tags`, even if unchanged;
- Only show changed, added, or removed primitive values: `string`,
  `number`, or `bool`;
- Only show added or removed elements in unordered collections and
  structural types: `map`, `set`, and `object`;
- Show added or removed elements with any surrounding unchanged elements
  for sequence types: `list` and `tuple`;
- Only show added or removed nested blocks, or blocks with changed
  attributes.

If any attributes, collection elements, or blocks are hidden, a count
is kept and displayed at the end of the parent scope. This ensures that
it is clear that the diff is only displaying a subset of the resource.

The experiment is currently enabled by default, but can be disabled by
setting the TF_X_CONCISE_DIFF environment variable to 0.
2020-09-10 10:35:55 -04:00
Kirill Zaborsky fbd3f191bd
Minor typo 2020-09-03 11:25:55 +03:00
Kristin Laemmert 196c183dda
terraform: remove state from `validate` graph walk (#26063)
This pull reverts a recent change to backend/local which created two context, one with and one without state. Instead I have removed the state entirely from the validate graph (by explicitly passing a states.NewState() to the validate graph builder).

This changed caused a test failure, which (ty so much for the help) @jbardin discovered was inaccurate all along: the test's call to `Validate()` was actually what was removing the output from state. The new expected test output matches terraform's actual behavior on the command line: if you use -target to destroy a resource, an output that references only that resource is *not* removed from state even though that test would lead you to believe it did.

This includes two tests to cover the expected behavior:

TestPlan_varsUnset has been updated so it will panic if it gets more than one request to input a variable
TestPlan_providerArgumentUnset covers #26035

Fixes #26035, #26027
2020-08-31 15:45:39 -04:00
James Bardin 1c09df1a66
Merge pull request #25779 from hashicorp/jbardin/remove-state-attrs
Remove resource state attributes that are no longer in the schema
2020-08-12 10:49:44 -04:00
Kristin Laemmert 6621501ae3
state: remove deprecated state package (#25490)
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.

* states: move state.Locker to statemgr

The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.

* command: remove unnecessary use of state package for writing local terraform state files

* move state.LocalState into terraform package

state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.

* slight change of heart: move state.LocalState into clistate instead of
terraform
2020-08-11 11:43:01 -04:00
Kristin Laemmert 86e9ba3d65
* backend/local: push responsibility for unlocking state into individual operations
* unlock the state if Context() has an error, exactly as backend/remote does today
* terraform console and terraform import will exit before unlocking state in case of error in Context()
* responsibility for unlocking state in the local backend is pushed down the stack, out of backend.go and into each individual state operation
* add tests confirming that state is not locked after apply and plan

* backend/local: add checks that the state is unlocked after operations

This adds tests to plan, apply and refresh which validate that the state
is unlocked after all operations, regardless of exit status. I've also
added specific tests that force Context() to fail during each operation
to verify that locking behavior specifically.
2020-08-11 11:23:42 -04:00
James Bardin 3cf84bb3f9 don't add state to the validate context
The validate command should work with the configuration, but when
validate was run at the start of a plan or apply command the state was
inserted in preparation for the next walk. This could lead to errors
when the resource schemas had changes and the state could not be
upgraded or decoded.
2020-08-07 14:13:57 -04:00
Kristin Laemmert 604e65bb62 Revert "backend/local: release lock if there is an error in Context() (#25427)"
This reverts commit 1ba0d615e7.
2020-06-30 14:12:32 -04:00
Kristin Laemmert 1ba0d615e7
backend/local: release lock if there is an error in Context() (#25427)
* command/console: return in case of errors before trying to unlock remote
state

The remote backend `Context` would exit without an active lock if there
was an error, while the local backend `Context` exited *with* a lock. This
caused a problem in `terraform console`, which would call unlock
regardless of error status.

This commit makes the local and remote backend consistently unlock the
state incase of error, and updates terraform console to check for errors
before trying to unlock the state.

* adding tests for remote and local backends
2020-06-29 14:57:42 -04:00
Martin Atkins 31a4b44d2e backend/local: treat output changes as side-effects to be applied
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.

This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".

This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.

The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.

We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.

This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
2020-05-29 07:36:40 -07:00
Ben Drucker 77f082bda2 remove assertion that PrepareProviderConfig was called 2020-05-07 22:36:49 -07:00
Kristin Laemmert 32062b00a2 backend/local: refactor tests with modern state and default providers (#24524) 2020-04-06 09:24:23 -07:00
Martin Atkins 549aede792 Remove terraform.ResourceProvider, use providercache.Installer instead
Back when we first introduced provider versioning in Terraform 0.10, we
did the provider version resolution in terraform.NewContext because we
weren't sure yet how exactly our versioning model was going to play out
(whether different versions could be selected per provider configuration,
for example) and because we were building around the limitations of our
existing filesystem-based plugin discovery model.

However, the new installer codepath is new able to do all of the
selections up front during installation, so we don't need such a heavy
inversion of control abstraction to get this done: the command package can
select the exact provider versions and pass their factories directly
to terraform.NewContext as a simple static map.

The result of this commit is that CLI commands other than "init" are now
able to consume the local cache directory and selections produced by the
installation process in "terraform init", passing all of the selected
providers down to the terraform.NewContext function for use in
implementing the main operations.

This commit is just enough to get the providers passing into the
terraform.Context. There's still plenty more to do here, including to
repair all of the tests this change has additionally broken.
2020-04-06 09:24:23 -07:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
Yuri Astrakhan 6eb968531d
backend/plan: Show warnings even if no changes are needed 2020-02-19 15:59:15 -08:00