Commit Graph

559 Commits

Author SHA1 Message Date
Barrett Clark edbc84420c Run apply -refresh-state instead of refresh
When a user runs `terraform refresh` we give them an error message that
tells them to run `terraform apply -refresh-state`. We could just run
that command for them, though. That is what this PR does.
2021-10-28 19:29:12 -05:00
Brandon Croft bf02b5cb53 e2e test for cloud run variables 2021-10-28 19:29:12 -05:00
Brandon Croft 9963bf940a refactor: only fetch variables if unset variables are not allowed 2021-10-28 19:29:12 -05:00
Omar Ismail dd856f8a1b Add run variables to cloud plan operations from non-file sources
Run variables allow the run API to accept input variables not found
in the configuration slug (the file-based ones plus workspace vars)
2021-10-28 19:29:12 -05:00
Brandon Croft dbbfae5a1c refactor ParseVariableValues into separate operations
1. ParseDeclaredValues: parses unparsed variables into terraform.InputValues
2. ProbeUndeclaredVariableValues: compares variable declarations with unparsed values to warn/error about undeclared variables
2021-10-28 19:29:12 -05:00
Omar Ismail 55fc590904 Teraform Cloud Backend State Migration
* determining source or destination to cloud
* handling single to single state migrations to cloud,
using a name strategy or a tags strategy
* Add end-to-end tests for state migration.
2021-10-28 19:29:12 -05:00
Chris Arcand 7cc53fe163 cloud: Set minimum TFE version
These changes remove all of the preexisting version checking for
individual features, wiping the slate clean with an overall minimum
requirement of a future TFP-API-Version 2.5, which at the time of this
writing is expected to be TFE v202112-1.

It also actually provides that expected TFE version as an actionable
error message, rather than generically saying that it isn't supported or
using the somewhat opaque API version header.
2021-10-28 19:29:12 -05:00
Chris Arcand 46e47ed379 Use the 'tfe' service for everything.
The 'tfe' service was appended to with various versions to denote a new
'feature' implemented by a new 'service'. This quickly proved to not be
scalable, as adding an entry to the discovery document from every
feature is bad.

The new mechanism added was checking the TFP-API-Version header on
requests for a version, instead.

So we'll remove the separation here between different tfe service
'versions' and the separate 'state' service and Just Use TFE, as well as
the TFP-API-Version header for all feature versioning., as well as the
TFP-API-Version header for all feature versioning.
2021-10-28 19:29:12 -05:00
Chris Arcand 412679522d Remove all constraint checking
This is an outdated mechanism that isn't used anymore.
2021-10-28 19:29:11 -05:00
Chris Arcand a4e196d067 Port backend/local: Check dependency lock consistency before any operations
This is a port of the changes made in #29683
2021-10-28 19:29:11 -05:00
Omar Ismail d5567b479c Fix terraform version in cloud e2e tests to ignore prerelease. (#29) 2021-10-28 19:29:11 -05:00
Omar Ismail a387af6c61 Add auto-approve logic, e2e tests 2021-10-28 19:29:11 -05:00
Chris Arcand ee384e8716 Attempt to set current Terraform version on new workspaces, always 2021-10-28 19:29:11 -05:00
Chris Arcand aba7d96596 cloud: Support interop from 0.14 to 1.1
The previous conservative guarantee that we would not make backwards
incompatible changes to the state file format until at least Terraform
1.1 can now be extended. Terraform 0.14 through 1.1 will be able to
interoperably use state files, so we can update the remote backend
version compatibility check accordingly.

This is a port of https://github.com/hashicorp/terraform/pull/29645
2021-10-28 19:29:11 -05:00
Chris Arcand 74e087dc29 Rename cloud.testBackendDefault -> cloud.testBackendWithName 2021-10-28 19:29:11 -05:00
Chris Arcand d2b6b5f2b3 cloud: Align local and remote workspace name with 'name' strategy
This changes the 'name' strategy to always align the local configured
workspace name and the remote Terraform Cloud workspace, rather than the
implicit use of the 'default' unnamed workspace being used instead.

What this essentially means is that the Cloud integration does not fully
support workspaces when configured for a single TFC workspace (as was
the case with the 'remote' backend), but *does* use the
backend.Workspaces() interface to allow for normal local behaviors like
terraform.workspace to resolve to the correct name. It does this by
always setting the local workspace name when the 'name' strategy is
used, as a part of initialization.

Part of the diff here is exporting all the previously unexported types
for mapping workspaces. The command package (and init in particular)
needs to be able to handle setting the local workspace in this
particular scenario.
2021-10-28 19:29:11 -05:00
Omar Ismail a94b2405b1 static checker 2021-10-28 19:29:11 -05:00
Omar Ismail 594a390f84 Use context parallelism 2021-10-28 19:29:10 -05:00
Chris Arcand 1791b71196 cloud: TestCloud_backendWithTags
Implementing this test was quite a rabbithole, as in order to satisfy
backendTestBackendStates() the workspaces returned from
backend.Workspaces() must match exactly, and the shortcut taken to test
pagination in 3cc58813f0 created an
impossible circumstance that got plastered over with the fact that
prefix filtering is done clientside, not by the API as it should be.

Tagging does not rely on clientside filtering, and expects that the
request made to the TFC API returns exactly those workspaces with the
given tags.

These changes include a better way to test pagination, wherein we
actually create over a page worth of valid workspaces in the mock client
and implement a simplified pagination behavior to match how the TFC API
actually works.
2021-10-28 19:29:10 -05:00
Chris Arcand a97a1c8f66 cloud.MockClient: Use Contains() to implement Search
The TFC API doesn't behave this way; a search term just signals the
substring was found within the name.
2021-10-28 19:29:10 -05:00
Chris Arcand 9bdf1d2579 Rename testBackendNoDefault
"NoDefault" is now ambiguous with tags, and it does not imply using a
prefix.
2021-10-28 19:29:10 -05:00
Chris Arcand 6dcd0db265 Map CLI workspaces by TFC tags 2021-10-28 19:29:10 -05:00
Chris Arcand 7a243379fb cloud: Refactor workspaceMapping concerns into strategy() 2021-10-28 19:29:10 -05:00
Chris Arcand 922a8e4488 Refactor private workspace fields into workspaceMapping
A mostly cosemetic change; The fields 'workspace' and 'prefix' don't
really describe well what they are from a caller, so change these to use
a workspaceMapping struct to convey they are for implementing workspace
mapping strategies from CLI -> TFC
2021-10-28 19:29:10 -05:00
Omar Ismail 0a125c0400 Cloud: refactor setting attributes, add tests 2021-10-28 19:29:10 -05:00
Omar Ismail 8f49757f89 Add testing for Cloud PrepareConfig 2021-10-28 19:29:10 -05:00
Chris Arcand 471dd479e8 Update go-tfe to 26689e
These changes include additions to fulfill the interface for the client
mock, plus moving all that logic (which needn't be duplicated across
both the remote and cloud packages) over to the cloud package under a
dedicated mock client file.
2021-10-28 19:29:10 -05:00
Chris Arcand 34597c237f cloud: Don't ignore .log golden files 2021-10-28 19:29:10 -05:00
Chris Arcand 85f37f1aa5 Add missing log mocks
I missed adding these in the original porting commit from the remote
backend, because *.log is added to gitignore.
2021-10-28 19:29:10 -05:00
Martin Atkins f4758803fe cloud: Backend implements new version of the backend.Local interface
This makes the new backend compatible with the new terraform.Context API,
which has changed in main.
2021-10-28 19:29:09 -05:00
Chris Arcand 18d54c1129 Allow cloud block overrides
These changes allow cloud blocks to be overridden by backend blocks and
vice versa; the logic follows the current backend behavior of a block
overriding a preceding block in full, with no merges.
2021-10-28 19:29:09 -05:00
Chris Arcand 07b3d015d1 Defer cloud block overrides
This restriction is temporary. Overrides should be allowed, but have the
added complexity of needing to also override a 'backend' block, so this
work is being deferred for now.
2021-10-28 19:29:09 -05:00
Chris Arcand fccc873a3d Remove ability to declare a 'cloud' backend
With the alternative block introduced in 7bf9b2c7b, this removes the
ability to explicitly declare the 'cloud' backend. The literal backend
interface is an implementation detail and no longer a user-level
concept when using Terraform Cloud.
2021-10-28 19:29:09 -05:00
Chris Arcand a4c24e3147 Add cloud {} configuration block for Terraform Cloud
This is a replacement declaration for using Terraform Cloud as a remote
backend, leaving the literal backend as an implementation detail and not
a user-level concept.
2021-10-28 19:29:09 -05:00
Chris Arcand a5f063625a Replace generic host error messages for feature support 2021-10-28 19:29:09 -05:00
Chris Arcand ea8ad0b15a Initial commit of 'cloud' package
The cloud package intends to implement a new integration for
Terraform Cloud/Enterprise. The purpose of this integration is to better
support TFC users; it will shed some overly generic UX and architecture,
behavior changes that are otherwise backwards incompatible in the remote
backend, and technical debt - all of which are vestiges from before
Terraform Cloud existed.

This initial commit is largely a porting of the existing 'remote'
backend, which will serve as an underlying implementation detail and not
be a typical user-level backend. This is because to re-implement the
literal backend interface is orthogonal to the purpose of this
integration, and can always be migrated away from later.

As this backend is considered an implementation detail, it will not be
registered as a declarable backend. Within these changes it is, for easy
of initial development and a clean diff.
2021-10-28 19:29:09 -05:00
Alisdair McDiarmid ded4f1a0fd
Merge pull request #29805 from hashicorp/alisdair/fix-init-workspace-select-input-false
command/init: Fail if -input=false but required
2021-10-26 11:37:22 -04:00
Alisdair McDiarmid ecb98e1c43 command/init: Fail if -input=false but required
When running `terraform init` against a backend with multiple
workspaces, none of which are the currently indicated local workspace,
Terraform prompts the user to choose a workspace from the list. In
automation, using the `-input=false` argument should disable asking for
input, but previously would hang instead.
2021-10-25 15:08:10 -04:00
Alisdair McDiarmid 6d0521e6be
Merge pull request #29803 from hashicorp/alisdair/init-backend-provider-errors
cli: Fix backend init failure with deleted cache
2021-10-25 13:57:45 -04:00
Alisdair McDiarmid 22923f9873
Merge pull request #29793 from hashicorp/alisdair/fix-remote-backend-migrate-version-check
backend/remote: Fix version check when migrating
2021-10-25 12:50:27 -04:00
Alisdair McDiarmid d42d83572b cli: Fix backend init failure with deleted cache
When an explicit backend is configured with a configuration which has
not yet been initialized, running `terraform init` performs a state
migration to fetch the remotely stored state in order to operate on it.
Like the previous bug introduced by the recent provider diagnostics
change, this code path was not correctly configured to enable init mode
for the backend, which resulted in a fatal error during init when the
cache dir is deleted.

Setting the `Init` backend option allows this code path to continue
without error when first initializing the backend for state migration.
The new e2e test fails without this change.
2021-10-25 12:45:35 -04:00
Alisdair McDiarmid 1729431520 backend/remote: Fix version check when migrating
When migrating state to an existing Terraform Cloud workspace using the
remote backend, we check the remote version is compatible with the local
one by default.

This commit fixes two bugs in this code:

- If using the "name" strategy for the remote backend, the list of
  destination workspaces is empty. This resulted in no version checking
  of the remote workspace, and we fell back to the string equality
  check.
- The user-specified CLI flag `-ignore-remote-version` was not being
  applied for the state migration version checking.
2021-10-21 14:10:25 -04:00
Alisdair McDiarmid 39de3ebec7 cli: Fix init failure with deleted cache
The init command needs to initialize a backend, in order to access
state, in turn to derive provider requirements from state. The backend
initialization step requires building provider factories, which
previously would fail if a lockfile was present without a corresponding
local provider cache.

This commit ensures that in this situation only, errors with the
provider factories are temporarily ignored. This allows us to continue
to initialize the backend, fetch providers, and then report any errors
as necessary.
2021-10-21 08:44:26 -04:00
Alisdair McDiarmid 1190b95fe2 command/e2etest: Ensure init fixes missing cache
We test that a deleted provider cache results in an error when running
terraform plan, but previously did not test that running init (as
instructed) would resolve the issue. This (failing) e2e test adds that
step.
2021-10-21 08:41:01 -04:00
James Bardin 0fcb75020f
Merge pull request #29784 from xiaozhu36/xiaozhu
backend/oss: Fixes the nil pointer panic error when missing access key or secret key
2021-10-20 16:32:13 -04:00
Martin Atkins 5b266dd5ca command: Remove the experimental "terraform add" command
We introduced this experiment to gather feedback, and the feedback we saw
led to us deciding to do another round of design work before we move
forward with something to meet this use-case.

In addition to being experimental, this has only been included in alpha
releases so far, and so on both counts it is not protected by the
Terraform v1.0 Compatibility Promises.
2021-10-20 06:42:47 -07:00
xiaozhu36 7afaea4cf2 backend/oss: Fixes the nil pointer panic error when missing access key or secret key 2021-10-20 16:05:00 +08:00
Alisdair McDiarmid cdd5ee6fb3
Merge pull request #29773 from hashicorp/alisdair/init-lock-flags
cli: Restore -lock and -lock-timeout init flags
2021-10-19 09:45:15 -04:00
Alisdair McDiarmid c587384dff cli: Restore -lock and -lock-timeout init flags
The -lock and -lock-timeout flags were removed prior to the release of
1.0 as they were thought to have no effect. This is not true in the case
of state migrations when changing backends. This commit restores these
flags, and adds test coverage for locking during backend state
migration.

Also update the help output describing other boolean flags, showing the
argument as the user would type it rather than the default behavior.
2021-10-19 09:32:30 -04:00
Alisdair McDiarmid fb58f9e6d2 cli: Fix flaky init cancel test
There is a race between the MockSource and ShutdownCh which sometimes
causes this test to fail. Add a HangingSource implementation of Source
which hangs until the context is cancelled, so that there is always time
for a user-initiated shutdown to trigger the cancellation code path
under test.
2021-10-19 09:10:49 -04:00