Commit Graph

668 Commits

Author SHA1 Message Date
Alisdair McDiarmid c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Alisdair McDiarmid eee57280f6 backend: Faster remote backend tests
The remote backend tests spent most of their execution time sleeping in
various polling and backoff waits. This is unnecessary when testing
against a mock server, so reduce all of these delays when under test to
much lower values.

Only one remaining test has an artificial delay: verifying the discovery
of services against an unknown hostname. This times out at DNS
resolution, which is more difficult to fix than seems worth it at this
time.
2020-11-18 16:00:05 -05:00
James Bardin 8e7a9b6312 output test for plan with no root output changes
Module outputs do not show up in the plan, and are not rendered in the
UI.
2020-11-17 16:11:57 -05:00
Upo 3b9c5e5bbb
backend/gcs: remove deprecated "path" config argument (#26841) 2020-11-06 16:35:40 -05:00
Upo 4ccc63d79d
backend: Add service account impersonation to GCS Backend and update the docs (#26700) 2020-11-06 09:06:07 -05:00
Alisdair McDiarmid b335918c3c backend: Only show root module output changes
When rendering planned output changes, we need to filter the plan's
output changes to ensure that only root module outputs which have
changed are rendered. Otherwise we will render changes for submodule
outputs, and (with concise diff disabled) render unchanged outputs also.
2020-11-02 10:24:22 -05:00
Paul Thrasher fc8a76a2a3
Stop printing tfce error message log 2020-10-28 16:03:55 -07:00
Kristin Laemmert b8e3b8036a backend: remove deprecated atlas backend 2020-10-26 14:05:18 -04:00
Paul Thrasher 960042ad42
Merge pull request #26618 from hashicorp/pault/tfce-continue-on-error
Allow runs to continue on tfce error
2020-10-20 13:23:09 -07:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Paul Thrasher d380f5873c
Allow runs to continue on tfce error
A cost estimation error does not actually stop a run, so the run was continuing in the background after the cli exits, causing confusion. This change matches the UI behavior.
2020-10-16 17:27:04 -07:00
James Bardin 5e9425b562 unreachable 2020-10-14 14:06:00 -04:00
James Bardin b8df47c9ac add struct field names 2020-10-14 14:05:41 -04:00
James Bardin 657dd33008
Merge pull request #26557 from remilapeyre/skip-ddl-commands
Add skip_table_creation and skip_index_creation options to the pg backend
2020-10-14 13:36:26 -04:00
James Bardin 5eca0788c6 rely solely on the plan changes for outputs
Now that outputs changes are tracked in full, we can remove the
comparisons with the prior state and use the planned changes directly.
2020-10-12 18:59:14 -04:00
Rémi Lapeyre 12a0a21c0b Add skip_table_creation and skip_index_creation options to the pg backend
Closes https://github.com/hashicorp/terraform/issues/25708
2020-10-12 22:47:19 +02:00
James Bardin 57a47d0b82
Merge pull request #26519 from hashicorp/jbardin/FIXME
Cleanup and remove legacy types from test mocks
2020-10-08 14:54:40 -04:00
James Bardin 103a6cf2db update mock provider call 2020-10-08 13:52:04 -04:00
Kristin Laemmert 6f4f6fbc72 helper: deprecate pathorcontents
pathorcontents was solely used by the gcs backend. I moved the function
into the backend package so it could still be used by other backends for
good measure.
2020-10-08 08:42:16 -04:00
Kristin Laemmert d2e999ba1f
remove unused code (#26503)
* remove unused code

I've removed the provider-specific code under registry, and unused nil
backend, and replaced a call to helper from backend/oss (the other
callers of that func are provisioners scheduled to be deprecated).

I also removed the Dockerfile, as our build process uses a different
file.

Finally I removed the examples directory, which had outdated examples
and links. There are better, actively maintained examples available.

* command: remove various unused bits

* test wasn't running

* backend: remove unused err
2020-10-07 11:00:06 -04:00
James Bardin 826ccdd123 re-enable test 2020-10-07 10:44:41 -04:00
Pam Selle f84a7c1d57
Merge pull request #26476 from remilapeyre/postgres-backend-escape-schema_name
Properly quote schema_name in the pg backend configuration
2020-10-05 14:57:53 -04:00
James Bardin 70652814e3 add missing Context argument 2020-10-05 14:43:46 -04:00
James Bardin 5e13d57f4e
Merge pull request #26463 from hashicorp/kt/azurerm-secret-backend-fix
AzureRM backend: correctly lookup environment from metadata host
2020-10-05 14:26:32 -04:00
Alisdair McDiarmid 35d4926bf7
Merge pull request #25837 from remilapeyre/consul-force-unlock
Fix `terraform force-unlock <lock_id>` for Consul backend
2020-10-05 13:11:08 -04:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Rémi Lapeyre 42be5854a2 Properly quote schema_name in the pg backend configuration 2020-10-03 18:02:13 +02:00
kt b66364f7aa
AzureRM backend: correctly lookup environment from metadata host 2020-10-01 16:34:51 -07:00
Kristin Laemmert d76cfc8c0c
Merge pull request #26440 from hashicorp/mildwonkey/remove-older-state-vs
refactor tests to use modern states.State in favor of terraform.State where possible
2020-10-01 08:11:27 -04:00
Pam Selle 3a99405b8b
Merge pull request #26420 from remilapeyre/postgres-default-backend
Always have the default workspace in the pg backend
2020-09-30 17:54:24 -04:00
Kristin Laemmert 479655ad47 refactor tests to use modern states.State in favor of terraform.State where possible 2020-09-30 16:07:54 -04:00
James Bardin ab6d6f99ae fix races in remote backend mock 2020-09-30 13:36:04 -04:00
Rémi Lapeyre 9acfe30ca6 Always have the default workspace in the pg backend
Closes https://github.com/hashicorp/terraform/issues/23121
2020-09-29 23:18:34 +02:00
Rémi Lapeyre 5c7008bd89 Merge remote-tracking branch 'origin/master' into consul-force-unlock 2020-09-29 10:59:36 +02:00
James Bardin 37569f5cc3 insert PlanRefresh into the context 2020-09-24 09:34:49 -04:00
James Bardin bc82347a04 fix tests
Update tests to match the new behavior. Some were incorrect, some no
longer make sense, and some just weren't setup to handle th plan api
calls.
2020-09-21 16:17:46 -04:00
James Bardin 312317abd0 wrong instance key in test state
This was never picked up by the tests until now
2020-09-17 09:55:00 -04:00
James Bardin 8658424059 skip plan with no refresh test
We still need to determine if `-refresh=false` is even useful with the
new planning strategy.
2020-09-17 09:55:00 -04:00
James Bardin f52d836e0a fix local backend tests to match new behavior
Leaving plan with -refresh=false tests failing for now.
2020-09-17 09:55:00 -04:00
James Bardin be757bd416 Refresh instances during plan
This change refreshes the instance state during plan, so a complete
Refresh no longer needs to happen before Plan.
2020-09-17 09:54:59 -04:00
Alisdair McDiarmid e183163739
Merge pull request #25856 from remilapeyre/consul-split-state
Split the state in chunks when they outgrow the limit of Consul KV store
2020-09-15 10:00:25 -04:00
Alisdair McDiarmid 92abaadc02
Merge pull request #25842 from remilapeyre/consul-path-slash
Sanitize lock path for the Consul backend when it ends with a /
2020-09-11 11:14:49 -04:00
Alisdair McDiarmid 09d8355f43 command: Add experimental concise diff renderer
When rendering a diff between current state and projected state, we only
show resources and outputs which have changes. However, we show a full
structural diff for these values, which includes all attributes and
blocks for a changed resource or output. The result can be a very long
diff, which makes it difficult to verify what the changed fields are.

This commit adds an experimental concise diff renderer, which suppresses
most unchanged fields, only displaying the most relevant changes and
some identifying context. This means:

- Always show all identifying attributes, initially defined as `id`,
  `name`, and `tags`, even if unchanged;
- Only show changed, added, or removed primitive values: `string`,
  `number`, or `bool`;
- Only show added or removed elements in unordered collections and
  structural types: `map`, `set`, and `object`;
- Show added or removed elements with any surrounding unchanged elements
  for sequence types: `list` and `tuple`;
- Only show added or removed nested blocks, or blocks with changed
  attributes.

If any attributes, collection elements, or blocks are hidden, a count
is kept and displayed at the end of the parent scope. This ensures that
it is clear that the diff is only displaying a subset of the resource.

The experiment is currently enabled by default, but can be disabled by
setting the TF_X_CONCISE_DIFF environment variable to 0.
2020-09-10 10:35:55 -04:00
Kirill Zaborsky fbd3f191bd
Minor typo 2020-09-03 11:25:55 +03:00
Matt Kasa c2e35ae3b5
backend/http: add support for configuration by environment variable (#25439) 2020-09-02 08:35:08 -04:00
Kristin Laemmert 196c183dda
terraform: remove state from `validate` graph walk (#26063)
This pull reverts a recent change to backend/local which created two context, one with and one without state. Instead I have removed the state entirely from the validate graph (by explicitly passing a states.NewState() to the validate graph builder).

This changed caused a test failure, which (ty so much for the help) @jbardin discovered was inaccurate all along: the test's call to `Validate()` was actually what was removing the output from state. The new expected test output matches terraform's actual behavior on the command line: if you use -target to destroy a resource, an output that references only that resource is *not* removed from state even though that test would lead you to believe it did.

This includes two tests to cover the expected behavior:

TestPlan_varsUnset has been updated so it will panic if it gets more than one request to input a variable
TestPlan_providerArgumentUnset covers #26035

Fixes #26035, #26027
2020-08-31 15:45:39 -04:00
James Bardin 0c5430f88d
Merge pull request #25585 from chushenmeshile/master
fix ossbackend only returned 100 workspaces
2020-08-19 16:55:31 -04:00
James Bardin bd5ff98ca7
Merge pull request #25830 from hashicorp/kt/secret-metadata-host
backend/azurerm: support loading environments from a metadata host
2020-08-19 13:57:10 -04:00
James Bardin d46e9a4198
Merge pull request #25770 from hashicorp/f/azure-backend-spcert
backend/azurerm: support for authenticating using a Client Certificate
2020-08-19 13:56:30 -04:00