ResiLien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
terraform/internal/command/init_test.go

2798 lines
84 KiB
Go
Raw Normal View History

command: init command start
2014-09-27 01:03:39 +02:00
package command
import (
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
"bytes"
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
"context"
failing test for backend re-init
2019-05-24 00:21:52 +02:00
"encoding/json"
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
"fmt"
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
"io/ioutil"
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
"log"
command: init works at a basic level
2014-09-27 01:30:49 +02:00
"os"
"path/filepath"
command: convert to use backends
2017-01-19 05:50:45 +01:00
"strings"
command: init command start
2014-09-27 01:03:39 +02:00
"testing"
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
"github.com/davecgh/go-spew/spew"
"github.com/google/go-cmp/cmp"
command: Fix TestInit_inputFalse This test was using old-style state files as its input, differing only by lineage. Since lineages are now managed within the state manager itself, the test can't use that to distinguish the two files and so we put a different output in each one instead. This also introduces some TRACE logging to the migration codepaths. There's some hard-to-follow control flow here and so this extra logging helps to understand the reason for a particular outcome, and since this codepath is visited only in "terraform init" anyway it doesn't hurt to be a bit more verbose here.
2018-11-09 23:26:01 +01:00
"github.com/mitchellh/cli"
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
"github.com/zclconf/go-cty/cty"
command/init: Be explicit that some options are not relevant for Cloud There are a few command line options for "terraform init" which are only relevant when working with traditional backends, with the Cloud integration previously just mostly ignoring them, or sometimes misbehaving slightly due to them creating an unreasonable situation. Now we'll catch these and return explicit errors, in order to be clear that these options are not needed nor supported in Cloud mode.
2021-11-13 02:07:10 +01:00
"github.com/hashicorp/go-version"
Move addrs/ to internal/addrs/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.
2021-05-17 21:00:50 +02:00
"github.com/hashicorp/terraform/internal/addrs"
Move configs/ to internal/configs/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.
2021-05-17 21:17:09 +02:00
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/configs/configschema"
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
"github.com/hashicorp/terraform/internal/depsfile"
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
"github.com/hashicorp/terraform/internal/getproviders"
"github.com/hashicorp/terraform/internal/providercache"
Move states/ to internal/states/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.
2021-05-17 21:43:35 +02:00
"github.com/hashicorp/terraform/internal/states"
command/init: Be explicit that some options are not relevant for Cloud There are a few command line options for "terraform init" which are only relevant when working with traditional backends, with the Cloud integration previously just mostly ignoring them, or sometimes misbehaving slightly due to them creating an unreasonable situation. Now we'll catch these and return explicit errors, in order to be clear that these options are not needed nor supported in Cloud mode.
2021-11-13 02:07:10 +01:00
"github.com/hashicorp/terraform/internal/states/statefile"
Move states/ to internal/states/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.
2021-05-17 21:43:35 +02:00
"github.com/hashicorp/terraform/internal/states/statemgr"
command: init command start
2014-09-27 01:03:39 +02:00
)
command: convert to use backends
2017-01-19 05:50:45 +01:00
func TestInit_empty(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
os.MkdirAll(td, 0755)
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: init command start
2014-09-27 01:03:39 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: init command start
2014-09-27 01:03:39 +02:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: init command start
2014-09-27 01:03:39 +02:00
},
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
command: init command start
2014-09-27 01:03:39 +02:00
}
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
func TestInit_multipleArgs(t *testing.T) {
command: tests should not leave dirs behind thank you (#24340)
2020-03-10 21:32:22 +01:00
// Create a temporary working directory that is empty
td := tempDir(t)
os.MkdirAll(td, 0755)
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: init command start
2014-09-27 01:03:39 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: init command start
2014-09-27 01:03:39 +02:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: init command start
2014-09-27 01:03:39 +02:00
},
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
args := []string{
"bad",
"bad",
}
command: init command start
2014-09-27 01:03:39 +02:00
if code := c.Run(args); code != 1 {
t.Fatalf("bad: \n%s", ui.OutputWriter.String())
}
}
command/init: test for issue 518
2014-11-04 21:19:39 +01:00
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
func TestInit_fromModule_cwdDest(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
os.MkdirAll(td, os.ModePerm)
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
},
}
args := []string{
"-from-module=" + testFixturePath("init"),
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if _, err := os.Stat(filepath.Join(td, "hello.tf")); err != nil {
t.Fatalf("err: %s", err)
}
}
// https://github.com/hashicorp/terraform/issues/518
func TestInit_fromModule_dstInSrc(t *testing.T) {
dir := tempDir(t)
if err := os.MkdirAll(dir, 0755); err != nil {
t.Fatalf("err: %s", err)
}
command: tests should not leave dirs behind thank you (#24340)
2020-03-10 21:32:22 +01:00
defer os.RemoveAll(dir)
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
// Change to the temporary directory
cwd, err := os.Getwd()
if err != nil {
t.Fatalf("err: %s", err)
}
if err := os.Chdir(dir); err != nil {
t.Fatalf("err: %s", err)
}
defer os.Chdir(cwd)
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if err := os.Mkdir("foo", os.ModePerm); err != nil {
t.Fatal(err)
}
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
if _, err := os.Create("issue518.tf"); err != nil {
t.Fatalf("err: %s", err)
}
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
if err := os.Chdir("foo"); err != nil {
t.Fatalf("err: %s", err)
}
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
},
}
args := []string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"-from-module=./..",
command: terraform init -from-module=... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.
2017-07-29 00:23:29 +02:00
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if _, err := os.Stat(filepath.Join(dir, "foo", "issue518.tf")); err != nil {
t.Fatalf("err: %s", err)
}
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
func TestInit_get(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get"), td)
command: convert to use backends
2017-01-19 05:50:45 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: convert to use backends
2017-01-19 05:50:45 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: convert to use backends
2017-01-19 05:50:45 +01:00
},
}
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Check output
output := ui.OutputWriter.String()
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if !strings.Contains(output, "foo in foo") {
t.Fatalf("doesn't look like we installed module 'foo': %s", output)
command: convert to use backends
2017-01-19 05:50:45 +01:00
}
}
command: terraform get -upgrade As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.
2017-06-12 23:14:40 +02:00
func TestInit_getUpgradeModules(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
testCopyDir(t, testFixturePath("init-get"), td)
command: terraform get -upgrade As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.
2017-06-12 23:14:40 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: terraform get -upgrade As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.
2017-06-12 23:14:40 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: terraform get -upgrade As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.
2017-06-12 23:14:40 +02:00
},
}
args := []string{
"-get=true",
"-upgrade",
}
if code := c.Run(args); code != 0 {
t.Fatalf("command did not complete successfully:\n%s", ui.ErrorWriter.String())
}
// Check output
output := ui.OutputWriter.String()
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if !strings.Contains(output, "Upgrading modules...") {
command: terraform get -upgrade As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.
2017-06-12 23:14:40 +02:00
t.Fatalf("doesn't look like get upgrade: %s", output)
}
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
func TestInit_backend(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend"), td)
command: convert to use backends
2017-01-19 05:50:45 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: convert to use backends
2017-01-19 05:50:45 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: convert to use backends
2017-01-19 05:50:45 +01:00
},
}
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if _, err := os.Stat(filepath.Join(DefaultDataDir, DefaultStateFilename)); err != nil {
t.Fatalf("err: %s", err)
}
}
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
func TestInit_backendUnset(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend"), td)
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
{
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
log.Printf("[TRACE] TestInit_backendUnset: beginning first init")
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
},
}
// Init
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
log.Printf("[TRACE] TestInit_backendUnset: first init complete")
t.Logf("First run output:\n%s", ui.OutputWriter.String())
t.Logf("First run errors:\n%s", ui.ErrorWriter.String())
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
if _, err := os.Stat(filepath.Join(DefaultDataDir, DefaultStateFilename)); err != nil {
t.Fatalf("err: %s", err)
}
}
{
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
log.Printf("[TRACE] TestInit_backendUnset: beginning second init")
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
// Unset
if err := ioutil.WriteFile("main.tf", []byte(""), 0644); err != nil {
t.Fatalf("err: %s", err)
}
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
},
}
add `-force-copy` option to init command The `-force-copy` option will suppress confirmation for copying state data. Modify some tests to use the option, making sure to leave coverage of the Input code path.
2017-03-21 20:05:51 +01:00
args := []string{"-force-copy"}
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
log.Printf("[TRACE] TestInit_backendUnset: second init complete")
t.Logf("Second run output:\n%s", ui.OutputWriter.String())
t.Logf("Second run errors:\n%s", ui.ErrorWriter.String())
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
s := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command/init: initialize backend even if not set in the config We need to initialize the backend even if the config has no backend set. This allows `init` to work when unsetting a previously set backend. Without this, there was no way to unset a backend.
2017-02-16 00:44:53 +01:00
if !s.Backend.Empty() {
t.Fatal("should not have backend config")
}
}
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
func TestInit_backendConfigFile(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-config-file"), td)
command: convert to use backends
2017-01-19 05:50:45 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
t.Run("good-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
},
}
args := []string{"-backend-config", "input.config"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
command: convert to use backends
2017-01-19 05:50:45 +01:00
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
// Read our saved backend config and verify we have our settings
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"hello","workspace_dir":null}`; got != want {
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
}
})
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
// the backend config file must not be a full terraform block
t.Run("full-backend-config-file", func(t *testing.T) {
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
},
}
args := []string{"-backend-config", "backend.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
if !strings.Contains(ui.ErrorWriter.String(), "Unsupported block type") {
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
// the backend config file must match the schema for the backend
t.Run("invalid-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
},
}
args := []string{"-backend-config", "invalid.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
if !strings.Contains(ui.ErrorWriter.String(), "Unsupported argument") {
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
// missing file is an error
t.Run("missing-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
},
}
args := []string{"-backend-config", "missing.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
if !strings.Contains(ui.ErrorWriter.String(), "Failed to read file") {
command/init: return an error with invalid -backend-config files (#25411) * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs
2020-06-26 18:49:31 +02:00
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
// blank filename clears the backend config
t.Run("blank-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
},
}
no longer automatically attempt state migration Add `init -migrate-state` flag to indicate automatic state migration is desired. This flag will be implied by the `-force-copy` flag, since that would indicate state migration is expected. If `init` encounters a change to the stored backend configuration, it will now always return an error when neither `-reconfigure` or `-migrate-state` is supplied. Turn the most common legacy output strings into diagnostics, removing the "see above text" error output.
2021-05-14 23:36:54 +02:00
args := []string{"-backend-config=", "-migrate-state"}
command: Fix backend config override validation When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.
2020-08-21 22:10:06 +02:00
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify the backend config is empty
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":null,"workspace_dir":null}`; got != want {
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
}
})
command: Add test for backend config validation
2020-08-26 17:37:11 +02:00
// simulate the local backend having a required field which is not
// specified in the override file
t.Run("required-argument", func(t *testing.T) {
c := &InitCommand{}
schema := &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"path": {
Type: cty.String,
Optional: true,
},
"workspace_dir": {
Type: cty.String,
Required: true,
},
},
}
flagConfigExtra := newRawFlags("-backend-config")
flagConfigExtra.Set("input.config")
_, diags := c.backendConfigOverrideBody(flagConfigExtra, schema)
if len(diags) != 0 {
t.Errorf("expected no diags, got: %s", diags.Err())
}
})
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
}
command: Fix init flags silent exit bug When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.
2020-06-18 23:56:05 +02:00
func TestInit_backendConfigFilePowershellConfusion(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-config-file"), td)
command: Fix init flags silent exit bug When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.
2020-06-18 23:56:05 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Fix init flags silent exit bug When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.
2020-06-18 23:56:05 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Fix init flags silent exit bug When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.
2020-06-18 23:56:05 +02:00
},
}
// SUBTLE: when using -flag=value with Powershell, unquoted values are
// broken into separate arguments. This results in the init command
// interpreting the flags as an empty backend-config setting (which is
// semantically valid!) followed by a custom configuration path.
//
// Adding the "=" here forces this codepath to be checked, and it should
// result in an early exit with a diagnostic that the provided
// configuration file is not a diretory.
args := []string{"-backend-config=", "./input.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
output := ui.ErrorWriter.String()
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
if got, want := output, `Too many command line arguments`; !strings.Contains(got, want) {
command: Fix init flags silent exit bug When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.
2020-06-18 23:56:05 +02:00
t.Fatalf("wrong output\ngot:\n%s\n\nwant: message containing %q", got, want)
}
}
command tests: plan and init (#28616) * command/init: add test for reconfigure * command/plan: adding tests * command/apply: tests * command: show and refresh tests
2021-05-05 20:13:20 +02:00
func TestInit_backendReconfigure(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
testCopyDir(t, testFixturePath("init-backend"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
providerSource, close := newMockProviderSource(t, map[string][]string{
"hashicorp/test": {"1.2.3"},
})
defer close()
ui := new(cli.MockUi)
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
ProviderSource: providerSource,
Ui: ui,
View: view,
},
}
// create some state, so the backend has something to migrate.
f, err := os.Create("foo") // this is the path" in the backend config
if err != nil {
t.Fatalf("err: %s", err)
}
err = writeStateForTesting(testState(), f)
f.Close()
if err != nil {
t.Fatalf("err: %s", err)
}
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// now run init again, changing the path.
// The -reconfigure flag prevents init from migrating
// Without -reconfigure, the test fails since the backend asks for input on migrating state
args = []string{"-reconfigure", "-backend-config", "path=changed"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
}
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
func TestInit_backendConfigFileChange(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-config-file-change"), td)
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
},
}
no longer automatically attempt state migration Add `init -migrate-state` flag to indicate automatic state migration is desired. This flag will be implied by the `-force-copy` flag, since that would indicate state migration is expected. If `init` encounters a change to the stored backend configuration, it will now always return an error when neither `-reconfigure` or `-migrate-state` is supplied. Turn the most common legacy output strings into diagnostics, removing the "see above text" error output.
2021-05-14 23:36:54 +02:00
args := []string{"-backend-config", "input.config", "-migrate-state"}
command: recompute config hash with ConfigFile set Fixes #12749 If we merge in an extra partial config we need to recompute the hash to compare with the old value to detect that change. This hash needs to NOT be stored and just used as a temporary. We want to keep the original hash in the state so that we don't detect a change from the config (since the config will always be partial).
2017-03-16 19:47:59 +01:00
if code := c.Run(args); code != 0 {
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify we have our settings
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"hello","workspace_dir":null}`; got != want {
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
}
}
cli: Restore -lock and -lock-timeout init flags The -lock and -lock-timeout flags were removed prior to the release of 1.0 as they were thought to have no effect. This is not true in the case of state migrations when changing backends. This commit restores these flags, and adds test coverage for locking during backend state migration. Also update the help output describing other boolean flags, showing the argument as the user would type it rather than the default behavior.
2021-10-18 20:41:04 +02:00
func TestInit_backendMigrateWhileLocked(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
testCopyDir(t, testFixturePath("init-backend-migrate-while-locked"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
providerSource, close := newMockProviderSource(t, map[string][]string{
"hashicorp/test": {"1.2.3"},
})
defer close()
ui := new(cli.MockUi)
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
ProviderSource: providerSource,
Ui: ui,
View: view,
},
}
// Create some state, so the backend has something to migrate from
f, err := os.Create("local-state.tfstate")
if err != nil {
t.Fatalf("err: %s", err)
}
err = writeStateForTesting(testState(), f)
f.Close()
if err != nil {
t.Fatalf("err: %s", err)
}
// Lock the source state
unlock, err := testLockState(testDataDir, "local-state.tfstate")
if err != nil {
t.Fatal(err)
}
defer unlock()
// Attempt to migrate
args := []string{"-backend-config", "input.config", "-migrate-state", "-force-copy"}
if code := c.Run(args); code == 0 {
t.Fatalf("expected nonzero exit code: %s", ui.OutputWriter.String())
}
// Disabling locking should work
args = []string{"-backend-config", "input.config", "-migrate-state", "-force-copy", "-lock=false"}
if code := c.Run(args); code != 0 {
t.Fatalf("expected zero exit code, got %d: %s", code, ui.ErrorWriter.String())
}
}
Test case for changing backend hash during aborted state migration Pulled and updated from https://github.com/hashicorp/terraform/pull/26260
2020-09-16 12:26:46 +02:00
func TestInit_backendConfigFileChangeWithExistingState(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
testCopyDir(t, testFixturePath("init-backend-config-file-change-migrate-existing"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
oldState := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
// we deliberately do not provide the answer for backend-migrate-copy-to-empty to trigger error
args := []string{"-migrate-state", "-backend-config", "input.config", "-input=true"}
if code := c.Run(args); code == 0 {
t.Fatal("expected error")
}
// Read our backend config and verify new settings are not saved
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"local-state.tfstate"}`; got != want {
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
}
// without changing config, hash should not change
if oldState.Backend.Hash != state.Backend.Hash {
t.Errorf("backend hash should not have changed\ngot: %d\nwant: %d", state.Backend.Hash, oldState.Backend.Hash)
}
}
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
func TestInit_backendConfigKV(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-config-kv"), td)
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: add test for -backend-config k/v
2017-03-17 18:22:48 +01:00
},
}
args := []string{"-backend-config", "path=hello"}
if code := c.Run(args); code != 0 {
command: convert to use backends
2017-01-19 05:50:45 +01:00
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify we have our settings
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"hello","workspace_dir":null}`; got != want {
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
command: convert to use backends
2017-01-19 05:50:45 +01:00
}
}
failing test for backend re-init
2019-05-24 00:21:52 +02:00
func TestInit_backendConfigKVReInit(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-config-kv"), td)
failing test for backend re-init
2019-05-24 00:21:52 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
failing test for backend re-init
2019-05-24 00:21:52 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
failing test for backend re-init
2019-05-24 00:21:52 +02:00
},
}
args := []string{"-backend-config", "path=test"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
ui = new(cli.MockUi)
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
failing test for backend re-init
2019-05-24 00:21:52 +02:00
},
}
// a second init should require no changes, nor should it change the backend.
args = []string{"-input=false"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// make sure the backend is configured how we expect
configState := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
cfg := map[string]interface{}{}
if err := json.Unmarshal(configState.Backend.ConfigRaw, &cfg); err != nil {
t.Fatal(err)
}
if cfg["path"] != "test" {
t.Fatalf(`expected backend path="test", got path="%v"`, cfg["path"])
}
allow setting -backend-config='' to unset override There is currently no way to unset -backend-config during init, since not setting that option assumes the user will use the saved config. Allow setting `-backend-config=""` to specify no overrides.
2019-05-29 19:58:04 +02:00
// override the -backend-config options by settings
no longer automatically attempt state migration Add `init -migrate-state` flag to indicate automatic state migration is desired. This flag will be implied by the `-force-copy` flag, since that would indicate state migration is expected. If `init` encounters a change to the stored backend configuration, it will now always return an error when neither `-reconfigure` or `-migrate-state` is supplied. Turn the most common legacy output strings into diagnostics, removing the "see above text" error output.
2021-05-14 23:36:54 +02:00
args = []string{"-input=false", "-backend-config", "", "-migrate-state"}
allow setting -backend-config='' to unset override There is currently no way to unset -backend-config during init, since not setting that option assumes the user will use the saved config. Allow setting `-backend-config=""` to specify no overrides.
2019-05-29 19:58:04 +02:00
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// make sure the backend is configured how we expect
configState = testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
cfg = map[string]interface{}{}
if err := json.Unmarshal(configState.Backend.ConfigRaw, &cfg); err != nil {
t.Fatal(err)
}
if cfg["path"] != nil {
t.Fatalf(`expected backend path="<nil>", got path="%v"`, cfg["path"])
}
failing test for backend re-init
2019-05-24 00:21:52 +02:00
}
don't store prepared backend config The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.
2019-05-24 20:51:18 +02:00
func TestInit_backendConfigKVReInitWithConfigDiff(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend"), td)
don't store prepared backend config The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.
2019-05-24 20:51:18 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
don't store prepared backend config The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.
2019-05-24 20:51:18 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
don't store prepared backend config The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.
2019-05-24 20:51:18 +02:00
},
}
args := []string{"-input=false"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
ui = new(cli.MockUi)
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
don't store prepared backend config The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.
2019-05-24 20:51:18 +02:00
},
}
// a second init with identical config should require no changes, nor
// should it change the backend.
args = []string{"-input=false", "-backend-config", "path=foo"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// make sure the backend is configured how we expect
configState := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
cfg := map[string]interface{}{}
if err := json.Unmarshal(configState.Backend.ConfigRaw, &cfg); err != nil {
t.Fatal(err)
}
if cfg["path"] != "foo" {
t.Fatalf(`expected backend path="foo", got path="%v"`, cfg["foo"])
}
}
command/init: omit a warning if -backend-config is used with no backend (#22164) * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```
2019-07-23 14:08:28 +02:00
func TestInit_backendCli_no_config_block(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init"), td)
command/init: omit a warning if -backend-config is used with no backend (#22164) * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```
2019-07-23 14:08:28 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: omit a warning if -backend-config is used with no backend (#22164) * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```
2019-07-23 14:08:28 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: omit a warning if -backend-config is used with no backend (#22164) * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```
2019-07-23 14:08:28 +02:00
},
}
args := []string{"-backend-config", "path=test"}
if code := c.Run(args); code != 0 {
t.Fatalf("got exit status %d; want 0\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
errMsg := ui.ErrorWriter.String()
if !strings.Contains(errMsg, "Warning: Missing backend configuration") {
t.Fatal("expected missing backend block warning, got", errMsg)
}
}
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
func TestInit_backendReinitWithExtra(t *testing.T) {
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend-empty"), td)
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
m := testMetaBackend(t, nil)
opts := &BackendOpts{
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
ConfigOverride: configs.SynthBody("synth", map[string]cty.Value{
"path": cty.StringVal("hello"),
}),
Init: true,
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
}
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
_, cHash, err := m.backendConfig(opts)
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
if err != nil {
t.Fatal(err)
}
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
},
}
args := []string{"-backend-config", "path=hello"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify we have our settings
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"hello","workspace_dir":null}`; got != want {
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
}
missing commits from 19688
2018-12-19 01:06:49 +01:00
if state.Backend.Hash != uint64(cHash) {
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
t.Fatal("mismatched state and config backend hashes")
}
// init again and make sure nothing changes
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state = testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"hello","workspace_dir":null}`; got != want {
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
}
missing commits from 19688
2018-12-19 01:06:49 +01:00
if state.Backend.Hash != uint64(cHash) {
add Rehash to terraform.BackendState This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.
2017-03-29 18:50:20 +02:00
t.Fatal("mismatched state and config backend hashes")
}
}
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
// move option from config to -backend-config args
func TestInit_backendReinitConfigToExtra(t *testing.T) {
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend"), td)
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
},
}
if code := c.Run([]string{"-input=false"}); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify we have our settings
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"foo","workspace_dir":null}`; got != want {
command: Various updates for the new backend package API This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.
2018-03-28 00:31:05 +02:00
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
}
backendHash := state.Backend.Hash
// init again but remove the path option from the config
cfg := "terraform {\n backend \"local\" {}\n}\n"
if err := ioutil.WriteFile("main.tf", []byte(cfg), 0644); err != nil {
t.Fatal(err)
}
command: Fix TestInit_backendReinitConfigToExtra This test was re-using the same InitCommand value to run multiple times, which is not realistic. Since we now cache configuration source code inside command.Meta on load, it's important that we use a fresh InitCommand instance here so it'll see the modified configuration file we've left on disk.
2018-11-09 02:26:15 +01:00
// We need a fresh InitCommand here because the old one now has our configuration
// file cached inside it, so it won't re-read the modification we just made.
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Fix TestInit_backendReinitConfigToExtra This test was re-using the same InitCommand value to run multiple times, which is not realistic. Since we now cache configuration source code inside command.Meta on load, it's important that we use a fresh InitCommand instance here so it'll see the modified configuration file we've left on disk.
2018-11-09 02:26:15 +01:00
},
}
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
args := []string{"-input=false", "-backend-config=path=foo"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
terraform: Ugly huge change to weave in new State and Plan types Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-08-14 23:24:45 +02:00
state = testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
command: Fix TestInit_backendReinitConfigToExtra This test was re-using the same InitCommand value to run multiple times, which is not realistic. Since we now cache configuration source code inside command.Meta on load, it's important that we use a fresh InitCommand instance here so it'll see the modified configuration file we've left on disk.
2018-11-09 02:26:15 +01:00
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":"foo","workspace_dir":null}`; got != want {
t.Errorf("wrong config after moving to arg\ngot: %s\nwant: %s", got, want)
}
detect when backend.Hash needs update It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.
2017-03-29 21:51:24 +02:00
if state.Backend.Hash == backendHash {
t.Fatal("state.Backend.Hash was not updated")
}
}
command/init: Be explicit that some options are not relevant for Cloud There are a few command line options for "terraform init" which are only relevant when working with traditional backends, with the Cloud integration previously just mostly ignoring them, or sometimes misbehaving slightly due to them creating an unreasonable situation. Now we'll catch these and return explicit errors, in order to be clear that these options are not needed nor supported in Cloud mode.
2021-11-13 02:07:10 +01:00
func TestInit_backendCloudInvalidOptions(t *testing.T) {
// There are various "terraform init" options that are only for
// traditional backends and not applicable to Terraform Cloud mode.
// For those, we want to return an explicit error rather than
// just silently ignoring them, so that users will be aware that
// Cloud mode has more of an expected "happy path" than the
// less-vertically-integrated backends do, and to avoid these
// unapplicable options becoming compatibility constraints for
// future evolution of Cloud mode.
// We use the same starting fixture for all of these tests, but some
// of them will customize it a bit as part of their work.
setupTempDir := func(t *testing.T) func() {
t.Helper()
td := tempDir(t)
testCopyDir(t, testFixturePath("init-cloud-simple"), td)
unChdir := testChdir(t, td)
return func() {
unChdir()
os.RemoveAll(td)
}
}
// Some of the tests need a non-empty placeholder state file to work
// with.
fakeState := states.BuildState(func(cb *states.SyncState) {
// Having a root module output value should be enough for this
// state file to be considered "non-empty" and thus a candidate
// for migration.
cb.SetOutputValue(
addrs.OutputValue{Name: "a"}.Absolute(addrs.RootModuleInstance),
cty.True,
false,
)
})
fakeStateFile := &statefile.File{
Lineage: "boop",
Serial: 4,
TerraformVersion: version.Must(version.NewVersion("1.0.0")),
State: fakeState,
}
var fakeStateBuf bytes.Buffer
err := statefile.WriteForTest(fakeStateFile, &fakeStateBuf)
if err != nil {
t.Error(err)
}
fakeStateBytes := fakeStateBuf.Bytes()
t.Run("-backend-config", func(t *testing.T) {
defer setupTempDir(t)()
// We have -backend-config as a pragmatic way to dynamically set
// certain settings of backends that tend to vary depending on
// where Terraform is running, such as AWS authentication profiles
// that are naturally local only to the machine where Terraform is
// running. Those needs don't apply to Terraform Cloud, because
// the remote workspace encapsulates all of the details of how
// operations and state work in that case, and so the Cloud
// configuration is only about which workspaces we'll be working
// with.
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-backend-config=anything"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -backend-config=... command line option is only for state backends, and
is not applicable to Terraform Cloud-based configurations.
To change the set of workspaces associated with this configuration, edit the
Cloud configuration block in the root module.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-reconfigure", func(t *testing.T) {
defer setupTempDir(t)()
// The -reconfigure option was originally imagined as a way to force
// skipping state migration when migrating between backends, but it
// has a historical flaw that it doesn't work properly when the
// initial situation is the implicit local backend with a state file
// present. The Terraform Cloud migration path has some additional
// steps to take care of more details automatically, and so
// -reconfigure doesn't really make sense in that context, particularly
// with its design bug with the handling of the implicit local backend.
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-reconfigure"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -reconfigure option is for in-place reconfiguration of state backends
only, and is not needed when changing Terraform Cloud settings.
When using Terraform Cloud, initialization automatically activates any new
Cloud configuration settings.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-reconfigure when migrating in", func(t *testing.T) {
defer setupTempDir(t)()
// We have a slightly different error message for the case where we
// seem to be trying to migrate to Terraform Cloud with existing
// state or explicit backend already present.
if err := os.WriteFile("terraform.tfstate", fakeStateBytes, 0644); err != nil {
t.Fatal(err)
}
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-reconfigure"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -reconfigure option is unsupported when migrating to Terraform Cloud,
because activating Terraform Cloud involves some additional steps.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-migrate-state", func(t *testing.T) {
defer setupTempDir(t)()
// In Cloud mode, migrating in or out always proposes migrating state
// and changing configuration while staying in cloud mode never migrates
// state, so this special option isn't relevant.
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-migrate-state"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -migrate-state option is for migration between state backends only, and
is not applicable when using Terraform Cloud.
State storage is handled automatically by Terraform Cloud and so the state
storage location is not configurable.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-migrate-state when migrating in", func(t *testing.T) {
defer setupTempDir(t)()
// We have a slightly different error message for the case where we
// seem to be trying to migrate to Terraform Cloud with existing
// state or explicit backend already present.
if err := os.WriteFile("terraform.tfstate", fakeStateBytes, 0644); err != nil {
t.Fatal(err)
}
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-migrate-state"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -migrate-state option is for migration between state backends only, and
is not applicable when using Terraform Cloud.
Terraform Cloud migration has additional steps, configured by interactive
prompts.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-force-copy", func(t *testing.T) {
defer setupTempDir(t)()
// In Cloud mode, migrating in or out always proposes migrating state
// and changing configuration while staying in cloud mode never migrates
// state, so this special option isn't relevant.
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-force-copy"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -force-copy option is for migration between state backends only, and is
not applicable when using Terraform Cloud.
State storage is handled automatically by Terraform Cloud and so the state
storage location is not configurable.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
t.Run("-force-copy when migrating in", func(t *testing.T) {
defer setupTempDir(t)()
// We have a slightly different error message for the case where we
// seem to be trying to migrate to Terraform Cloud with existing
// state or explicit backend already present.
if err := os.WriteFile("terraform.tfstate", fakeStateBytes, 0644); err != nil {
t.Fatal(err)
}
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
Ui: ui,
View: view,
},
}
args := []string{"-force-copy"}
if code := c.Run(args); code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
}
gotStderr := ui.ErrorWriter.String()
wantStderr := `
Error: Invalid command-line option
The -force-copy option is for migration between state backends only, and is
not applicable when using Terraform Cloud.
Terraform Cloud migration has additional steps, configured by interactive
prompts.
`
if diff := cmp.Diff(wantStderr, gotStderr); diff != "" {
t.Errorf("wrong error output\n%s", diff)
}
})
}
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
// make sure inputFalse stops execution on migrate
func TestInit_inputFalse(t *testing.T) {
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-backend"), td)
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
c := &InitCommand{
Meta: Meta{
Push plugin discovery down into command package Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.
2017-04-14 03:05:58 +02:00
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
},
}
args := []string{"-input=false", "-backend-config=path=foo"}
command staticcheck
2020-12-01 18:34:50 +01:00
if code := c.Run(args); code != 0 {
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
t.Fatalf("bad: \n%s", ui.ErrorWriter)
}
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
// write different states for foo and bar
command: Fix TestInit_inputFalse This test was using old-style state files as its input, differing only by lineage. Since lineages are now managed within the state manager itself, the test can't use that to distinguish the two files and so we put a different output in each one instead. This also introduces some TRACE logging to the migration codepaths. There's some hard-to-follow control flow here and so this extra logging helps to understand the reason for a particular outcome, and since this codepath is visited only in "terraform init" anyway it doesn't hurt to be a bit more verbose here.
2018-11-09 23:26:01 +01:00
fooState := states.BuildState(func(s *states.SyncState) {
s.SetOutputValue(
addrs.OutputValue{Name: "foo"}.Absolute(addrs.RootModuleInstance),
cty.StringVal("foo"),
false, // not sensitive
)
})
if err := statemgr.NewFilesystem("foo").WriteState(fooState); err != nil {
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
t.Fatal(err)
}
command: Fix TestInit_inputFalse This test was using old-style state files as its input, differing only by lineage. Since lineages are now managed within the state manager itself, the test can't use that to distinguish the two files and so we put a different output in each one instead. This also introduces some TRACE logging to the migration codepaths. There's some hard-to-follow control flow here and so this extra logging helps to understand the reason for a particular outcome, and since this codepath is visited only in "terraform init" anyway it doesn't hurt to be a bit more verbose here.
2018-11-09 23:26:01 +01:00
barState := states.BuildState(func(s *states.SyncState) {
s.SetOutputValue(
addrs.OutputValue{Name: "bar"}.Absolute(addrs.RootModuleInstance),
cty.StringVal("bar"),
false, // not sensitive
)
})
if err := statemgr.NewFilesystem("bar").WriteState(barState); err != nil {
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
t.Fatal(err)
}
ui = new(cli.MockUi)
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
},
}
no longer automatically attempt state migration Add `init -migrate-state` flag to indicate automatic state migration is desired. This flag will be implied by the `-force-copy` flag, since that would indicate state migration is expected. If `init` encounters a change to the stored backend configuration, it will now always return an error when neither `-reconfigure` or `-migrate-state` is supplied. Turn the most common legacy output strings into diagnostics, removing the "see above text" error output.
2021-05-14 23:36:54 +02:00
args = []string{"-input=false", "-backend-config=path=bar", "-migrate-state"}
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
if code := c.Run(args); code == 0 {
t.Fatal("init should have failed", ui.OutputWriter)
}
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
errMsg := ui.ErrorWriter.String()
Cloud integration requires input for migrations We cannot programmatically migrate workspaces to Terraform Cloud without prompts, so `-input=false` should not be allowed in those cases. There are 4 scenarios where we need input from a user to complete migrating workspaces to Terraform Cloud. 1.) Migrate from a single local workspace to Terraform Cloud * Terraform config for a local backend. Implicit local (no backend specified) is fine. * `terraform init` and `terraform apply` * Change the Terraform config to use the cloud block * `terraform init -input=false` * You should now see an error message 2.) Migrate from a remote backend with a prefix to Terraform Cloud with tags * Create a workspace in Terraform Cloud manually. The name should include a prefix, like "app-one" * Have the terraform config use `backend "remote"` with a prefix set to "app-" * `terraform init` and `terraform apply` * Update the Terraform config to use a cloud block with `tags = ["app"]`. There should not be a prefix defined in the config now. * `terraform init -input=false` * You should now see an error message 3.) Migrate from multiple local workspaces to a single Terraform Cloud workspace * Create one or many local workspaces * `terraform init` and `terraform apply` in each * Change the Terraform config to use the cloud block * `terraform init -input=false` * You should now see an error message 4.) Migrate to Terraform Cloud and ask for a workspace name * Create several local workspaces * `terraform init` and `terraform apply` in each * Change the Terraform config to use the cloud block with tags * `terraform init -input=false` * You should now see an error message
2021-11-22 20:17:04 +01:00
if !strings.Contains(errMsg, "interactive input is disabled") {
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
t.Fatal("expected input disabled error, got", errMsg)
}
ui = new(cli.MockUi)
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
},
}
// A missing input=false should abort rather than loop infinitely
don't migrate backend during init without override If the backend config hashes match during init, and there are no new backend override options, then we assume the existing config is OK. Since init should be idempotent, we should be able to run init with no options or config changes, and not effect the backends at all.
2019-05-24 17:31:04 +02:00
args = []string{"-backend-config=path=baz"}
inputFalse test should attempt migration and check error Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 16:37:15 +01:00
if code := c.Run(args); code == 0 {
t.Fatal("init should have failed", ui.OutputWriter)
}
honor `input=false` in state migration return an error when confirming a copy if -input=false
2017-03-29 22:45:25 +02:00
}
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
func TestInit_getProvider(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-providers"), td)
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
overrides := metaOverridesForProvider(testProvider())
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
// looking for an exact version
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"exact": {"1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// config requires >= 2.3.3
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"greater-than": {"2.3.4", "2.3.3", "2.3.0"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// config specifies
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"between": {"3.4.5", "2.3.4", "1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
})
defer close()
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
m := Meta{
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
testingOverrides: overrides,
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
command: init to allow plugin init without backend init Previously init would crash if given these options: -backend=false -get-plugins=true This is because the state is used as a source of provider dependency information, and we need to instantiate the backend to get the state. To avoid the crash, we now use the following adjusted behavior: - if -backend=true, we behave as before - if -backend=false, we instead try to instantiate the backend the same way any other command would, without modifying its configuration - if we're able to instantiate the backend, we use it to fetch state for dependency resolution purposes - if the backend is not instantiable then we assume it's not yet configured and proceed with a nil state, which may cause us to see an incomplete picture of the dependencies but still allows the install to succeed. Subsequently running "terraform plan" will not work until the backend is (re-)initialized, so the incomplete picture of required plugins is safe.
2017-06-21 19:32:13 +02:00
args := []string{
"-backend=false", // should be possible to install plugins without backend init
}
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// check that we got the providers for our config
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
exactPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/hashicorp/exact/1.2.3/%s", getproviders.CurrentPlatform)
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
if _, err := os.Stat(exactPath); os.IsNotExist(err) {
t.Fatal("provider 'exact' not downloaded")
}
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
greaterThanPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/hashicorp/greater-than/2.3.4/%s", getproviders.CurrentPlatform)
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
if _, err := os.Stat(greaterThanPath); os.IsNotExist(err) {
rename provider-specific functions (#24417) missingPlugins was hard-coded to work only with provider plugins, so I renamed it to clarify the usage. Also renamed a test provider from greater_than to greater-than as the underscore is an invalid provider name character and this will become a hard error in the near future.
2020-03-20 18:59:59 +01:00
t.Fatal("provider 'greater-than' not downloaded")
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
betweenPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/hashicorp/between/2.3.4/%s", getproviders.CurrentPlatform)
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
if _, err := os.Stat(betweenPath); os.IsNotExist(err) {
t.Fatal("provider 'between' not downloaded")
}
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
t.Run("future-state", func(t *testing.T) {
// getting providers should fail if a state from a newer version of
// terraform exists, since InitCommand.getProviders needs to inspect that
// state.
remove LegacyProvider (#26433)
2020-10-05 14:33:49 +02:00
f, err := os.Create(DefaultStateFilename)
if err != nil {
t.Fatalf("err: %s", err)
}
defer f.Close()
states: Disable Terraform version check For this version of Terraform and forward, we no longer refuse to read compatible state files written by future versions of Terraform. This is a commitment that any changes to the semantics or format of the state file after this commit will require a new state file version 5. The result of this is that users of this Terraform version will be able to share remote state with users of future versions, and all users will be able to read and write state. This will be true until the next major state file version is required. This does not affect users of previous versions of Terraform, which will continue to refuse to read state written by later versions.
2020-10-28 17:41:11 +01:00
// Construct a mock state file from the far future
type FutureState struct {
Version uint `json:"version"`
Lineage string `json:"lineage"`
TerraformVersion string `json:"terraform_version"`
Outputs map[string]interface{} `json:"outputs"`
Resources []map[string]interface{} `json:"resources"`
remove LegacyProvider (#26433)
2020-10-05 14:33:49 +02:00
}
states: Disable Terraform version check For this version of Terraform and forward, we no longer refuse to read compatible state files written by future versions of Terraform. This is a commitment that any changes to the semantics or format of the state file after this commit will require a new state file version 5. The result of this is that users of this Terraform version will be able to share remote state with users of future versions, and all users will be able to read and write state. This will be true until the next major state file version is required. This does not affect users of previous versions of Terraform, which will continue to refuse to read state written by later versions.
2020-10-28 17:41:11 +01:00
fs := &FutureState{
Version: 999,
Lineage: "123-456-789",
TerraformVersion: "999.0.0",
command staticcheck
2020-12-01 18:34:50 +01:00
Outputs: make(map[string]interface{}),
states: Disable Terraform version check For this version of Terraform and forward, we no longer refuse to read compatible state files written by future versions of Terraform. This is a commitment that any changes to the semantics or format of the state file after this commit will require a new state file version 5. The result of this is that users of this Terraform version will be able to share remote state with users of future versions, and all users will be able to read and write state. This will be true until the next major state file version is required. This does not affect users of previous versions of Terraform, which will continue to refuse to read state written by later versions.
2020-10-28 17:41:11 +01:00
Resources: make([]map[string]interface{}, 0),
}
src, err := json.MarshalIndent(fs, "", " ")
if err != nil {
t.Fatalf("failed to marshal future state: %s", err)
}
src = append(src, '\n')
_, err = f.Write(src)
command staticcheck
2020-12-01 18:34:50 +01:00
if err != nil {
t.Fatal(err)
}
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
m.Ui = ui
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
m.View = view
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
}
if code := c.Run(nil); code == 0 {
t.Fatal("expected error, got:", ui.OutputWriter)
}
errMsg := ui.ErrorWriter.String()
states: Disable Terraform version check For this version of Terraform and forward, we no longer refuse to read compatible state files written by future versions of Terraform. This is a commitment that any changes to the semantics or format of the state file after this commit will require a new state file version 5. The result of this is that users of this Terraform version will be able to share remote state with users of future versions, and all users will be able to read and write state. This will be true until the next major state file version is required. This does not affect users of previous versions of Terraform, which will continue to refuse to read state written by later versions.
2020-10-28 17:41:11 +01:00
if !strings.Contains(errMsg, "Unsupported state file format") {
check state version during init The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.
2017-12-14 21:46:43 +01:00
t.Fatal("unexpected error:", errMsg)
}
})
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
func TestInit_getProviderSource(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-provider-source"), td)
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
overrides := metaOverridesForProvider(testProvider())
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
// looking for an exact version
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"acme/alpha": {"1.2.3"},
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
// config doesn't specify versions for other providers
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"registry.example.com/acme/beta": {"1.0.0"},
"gamma": {"2.0.0"},
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
})
defer close()
m := Meta{
testingOverrides: overrides,
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
args := []string{
"-backend=false", // should be possible to install plugins without backend init
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// check that we got the providers for our config
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
exactPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/acme/alpha/1.2.3/%s", getproviders.CurrentPlatform)
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
if _, err := os.Stat(exactPath); os.IsNotExist(err) {
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
t.Error("provider 'alpha' not downloaded")
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
}
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
greaterThanPath := fmt.Sprintf(".terraform/providers/registry.example.com/acme/beta/1.0.0/%s", getproviders.CurrentPlatform)
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
if _, err := os.Stat(greaterThanPath); os.IsNotExist(err) {
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
t.Error("provider 'beta' not downloaded")
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
}
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
betweenPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/hashicorp/gamma/2.0.0/%s", getproviders.CurrentPlatform)
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
if _, err := os.Stat(betweenPath); os.IsNotExist(err) {
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
t.Error("provider 'gamma' not downloaded")
}
}
command: Better in-house provider install errors When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.
2020-08-31 23:02:05 +02:00
func TestInit_getProviderLegacyFromState(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-provider-legacy-from-state"), td)
command: Better in-house provider install errors When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.
2020-08-31 23:02:05 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
overrides := metaOverridesForProvider(testProvider())
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Better in-house provider install errors When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.
2020-08-31 23:02:05 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
"acme/alpha": {"1.2.3"},
})
defer close()
m := Meta{
testingOverrides: overrides,
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Better in-house provider install errors When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.
2020-08-31 23:02:05 +02:00
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
if code := c.Run(nil); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
// Expect this diagnostic output
wants := []string{
command/init: Remove support for legacy provider addresses We no longer need to support 0.12-and-earlier-style provider addresses because users should've upgraded their existing configurations and states on Terraform 0.13 already. For now this is only checked in the "init" command, because various test shims are still relying on the idea of legacy providers the core layer. However, rejecting these during init is sufficient grounds to avoid supporting legacy provider addresses in the new dependency lock file format, and thus sets the stage for a more severe removal of legacy provider support in a later commit.
2020-09-30 02:51:39 +02:00
"Invalid legacy provider address",
"You must complete the Terraform 0.13 upgrade process",
command: Better in-house provider install errors When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.
2020-08-31 23:02:05 +02:00
}
got := ui.ErrorWriter.String()
for _, want := range wants {
if !strings.Contains(got, want) {
t.Fatalf("expected output to contain %q, got:\n\n%s", want, got)
}
}
}
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
func TestInit_getProviderInvalidPackage(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-provider-invalid-package"), td)
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
overrides := metaOverridesForProvider(testProvider())
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
// create a provider source which allows installing an invalid package
addr := addrs.MustParseProviderSourceString("invalid/package")
version := getproviders.MustParseVersion("1.0.0")
meta, close, err := getproviders.FakeInstallablePackageMeta(
addr,
version,
getproviders.VersionList{getproviders.MustParseVersion("5.0")},
getproviders.CurrentPlatform,
"terraform-package", // should be "terraform-provider-package"
)
defer close()
if err != nil {
t.Fatalf("failed to prepare fake package for %s %s: %s", addr.ForDisplay(), version, err)
}
providerSource := getproviders.NewMockSource([]getproviders.PackageMeta{meta}, nil)
m := Meta{
testingOverrides: overrides,
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
args := []string{
"-backend=false", // should be possible to install plugins without backend init
}
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
// invalid provider should be installed
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
packagePath := fmt.Sprintf(".terraform/providers/registry.terraform.io/invalid/package/1.0.0/%s/terraform-package", getproviders.CurrentPlatform)
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
if _, err := os.Stat(packagePath); os.IsNotExist(err) {
t.Fatal("provider 'invalid/package' not downloaded")
}
wantErrors := []string{
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
"Failed to install provider",
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
"could not find executable file starting with terraform-provider-package",
}
got := ui.ErrorWriter.String()
for _, wantError := range wantErrors {
if !strings.Contains(got, wantError) {
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
t.Fatalf("missing error:\nwant: %q\ngot:\n%s", wantError, got)
command/init: Display provider validation errors After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.
2020-07-07 20:48:45 +02:00
}
}
}
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
func TestInit_getProviderDetectedLegacy(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-provider-detected-legacy"), td)
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
// We need to construct a multisource with a mock source and a registry
// source: the mock source will return ErrRegistryProviderNotKnown for an
// unknown provider, and the registry source will allow us to look up the
// appropriate namespace if possible.
providerSource, psClose := newMockProviderSource(t, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"hashicorp/foo": {"1.2.3"},
"terraform-providers/baz": {"2.3.4"}, // this will not be installed
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
})
defer psClose()
registrySource, rsClose := testRegistrySource(t)
defer rsClose()
multiSource := getproviders.MultiSource{
{Source: providerSource},
{Source: registrySource},
}
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
m := Meta{
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
ProviderSource: multiSource,
}
c := &InitCommand{
Meta: m,
}
args := []string{
"-backend=false", // should be possible to install plugins without backend init
}
if code := c.Run(args); code == 0 {
t.Fatalf("expected error, got output: \n%s", ui.OutputWriter.String())
}
// foo should be installed
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
fooPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/hashicorp/foo/1.2.3/%s", getproviders.CurrentPlatform)
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
if _, err := os.Stat(fooPath); os.IsNotExist(err) {
t.Error("provider 'foo' not installed")
}
// baz should not be installed
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
bazPath := fmt.Sprintf(".terraform/providers/registry.terraform.io/terraform-providers/baz/2.3.4/%s", getproviders.CurrentPlatform)
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
if _, err := os.Stat(bazPath); !os.IsNotExist(err) {
t.Error("provider 'baz' installed, but should not be")
}
// error output is the main focus of this test
errOutput := ui.ErrorWriter.String()
command/init: Improve diags for legacy providers When initializing a configuration which refers to re-namespaced legacy providers, we attempt to detect this and display a diagnostic message. Previously this message would direct the user to run the 0.13upgrade command, but without specifying in which directories. This commit detects which modules are using the providers in question, and for local modules displays a list of upgrade commands which specify the source directories of these modules. For remote modules, we display a separate list noting that they need to be upgraded elsewhere, providing both the local module call name and the module source address.
2020-06-09 19:33:07 +02:00
errors := []string{
command/init: Remove support for legacy provider addresses We no longer need to support 0.12-and-earlier-style provider addresses because users should've upgraded their existing configurations and states on Terraform 0.13 already. For now this is only checked in the "init" command, because various test shims are still relying on the idea of legacy providers the core layer. However, rejecting these during init is sufficient grounds to avoid supporting legacy provider addresses in the new dependency lock file format, and thus sets the stage for a more severe removal of legacy provider support in a later commit.
2020-09-30 02:51:39 +02:00
"Failed to query available provider packages",
"Could not retrieve the list of available versions",
"registry.terraform.io/hashicorp/baz",
"registry.terraform.io/hashicorp/frob",
command/init: Improve diags for legacy providers When initializing a configuration which refers to re-namespaced legacy providers, we attempt to detect this and display a diagnostic message. Previously this message would direct the user to run the 0.13upgrade command, but without specifying in which directories. This commit detects which modules are using the providers in question, and for local modules displays a list of upgrade commands which specify the source directories of these modules. For remote modules, we display a separate list noting that they need to be upgraded elsewhere, providing both the local module call name and the module source address.
2020-06-09 19:33:07 +02:00
}
for _, want := range errors {
if !strings.Contains(errOutput, want) {
t.Fatalf("expected error %q: %s", want, errOutput)
}
command/init: Better diagnostics for provider 404s Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.
2020-05-25 21:24:35 +02:00
}
}
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
func TestInit_providerSource(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
testCopyDir(t, testFixturePath("init-required-providers"), td)
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"test": {"1.2.3", "1.2.4"},
"test-beta": {"1.2.4"},
"source": {"1.2.2", "1.2.3", "1.2.1"},
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
})
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
defer close()
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
}
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
}
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
args := []string{}
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if strings.Contains(ui.OutputWriter.String(), "Terraform has initialized, but configuration upgrades may be needed") {
t.Fatalf("unexpected \"configuration upgrade\" warning in output")
}
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
cacheDir := m.providerLocalCacheDir()
gotPackages := cacheDir.AllAvailablePackages()
wantPackages := map[addrs.Provider][]providercache.CachedProvider{
addrs.NewDefaultProvider("test"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("test"),
Version: getproviders.MustParseVersion("1.2.3"),
PackageDir: expectedPackageInstallPath("test", "1.2.3", false),
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
},
},
provider source tests: added test suite to exercise hyphenated providers (#24685)
2020-04-16 21:54:33 +02:00
addrs.NewDefaultProvider("test-beta"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("test-beta"),
Version: getproviders.MustParseVersion("1.2.4"),
PackageDir: expectedPackageInstallPath("test-beta", "1.2.4", false),
provider source tests: added test suite to exercise hyphenated providers (#24685)
2020-04-16 21:54:33 +02:00
},
},
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
addrs.NewDefaultProvider("source"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("source"),
Version: getproviders.MustParseVersion("1.2.3"),
PackageDir: expectedPackageInstallPath("source", "1.2.3", false),
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
},
},
}
if diff := cmp.Diff(wantPackages, gotPackages); diff != "" {
t.Errorf("wrong cache directory contents after upgrade\n%s", diff)
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
locks, err := m.lockedDependencies()
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
if err != nil {
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
t.Fatalf("failed to get locked dependencies: %s", err)
}
gotProviderLocks := locks.AllProviders()
wantProviderLocks := map[addrs.Provider]*depsfile.ProviderLock{
addrs.NewDefaultProvider("test-beta"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("test-beta"),
getproviders.MustParseVersion("1.2.4"),
getproviders.MustParseVersionConstraints("= 1.2.4"),
[]getproviders.Hash{
getproviders.HashScheme1.New("see6W06w09Ea+AobFJ+mbvPTie6ASqZAAdlFZbs8BSM="),
},
),
addrs.NewDefaultProvider("test"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("test"),
getproviders.MustParseVersion("1.2.3"),
getproviders.MustParseVersionConstraints("= 1.2.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("wlbEC2mChQZ2hhgUhl6SeVLPP7fMqOFUZAQhQ9GIIno="),
},
),
addrs.NewDefaultProvider("source"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("source"),
getproviders.MustParseVersion("1.2.3"),
getproviders.MustParseVersionConstraints("= 1.2.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("myS3qb3px3tRBq1ZWRYJeUH+kySWpBc0Yy8rw6W7/p4="),
},
),
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
if diff := cmp.Diff(gotProviderLocks, wantProviderLocks, depsfile.ProviderLockComparer); diff != "" {
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
t.Errorf("wrong version selections after upgrade\n%s", diff)
}
internal: Verify provider signatures on install Providers installed from the registry are accompanied by a list of checksums (the "SHA256SUMS" file), which is cryptographically signed to allow package authentication. The process of verifying this has multiple steps: - First we must verify that the SHA256 hash of the package archive matches the expected hash. This could be done for local installations too, in the future. - Next we ensure that the expected hash returned as part of the registry API response matches an entry in the checksum list. - Finally we verify the cryptographic signature of the checksum list, using the public keys provided by the registry. Each of these steps is implemented as a separate PackageAuthentication type. The local archive installation mechanism uses only the archive checksum authenticator, and the HTTP installation uses all three in the order given. The package authentication system now also returns a result value, which is used by command/init to display the result of the authentication process. There are three tiers of signature, each of which is presented differently to the user: - Signatures from the embedded HashiCorp public key indicate that the provider is officially supported by HashiCorp; - If the signing key is not from HashiCorp, it may have an associated trust signature, which indicates that the provider is from one of HashiCorp's trusted partners; - Otherwise, if the signature is valid, this is a community provider.
2020-04-08 22:22:07 +02:00
outputStr := ui.OutputWriter.String()
if want := "Installed hashicorp/test v1.2.3 (verified checksum)"; !strings.Contains(outputStr, want) {
t.Fatalf("unexpected output: %s\nexpected to include %q", outputStr, want)
}
Mildwonkey/tfconfig upgrade (#23670) * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.
2020-01-10 17:54:53 +01:00
}
command: make module installation interruptible Earlier work to make "terraform init" interruptible made the getproviders package context-aware in order to allow provider installation to be cancelled. Here we make a similar change for module installation, which is now also cancellable with SIGINT. This involves plumbing context through initwd and getmodules. Functions which can make network requests now include a context parameter whose cancellation cancels those requests. Since the module installation code is shared, "terraform get" is now also interruptible during module installation.
2021-11-01 21:09:16 +01:00
func TestInit_cancelModules(t *testing.T) {
// This test runs `terraform init` as if SIGINT (or similar on other
// platforms) were sent to it, testing that it is interruptible.
td := tempDir(t)
testCopyDir(t, testFixturePath("init-registry-module"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
// Our shutdown channel is pre-closed so init will exit as soon as it
// starts a cancelable portion of the process.
shutdownCh := make(chan struct{})
close(shutdownCh)
ui := cli.NewMockUi()
view, _ := testView(t)
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
View: view,
ShutdownCh: shutdownCh,
}
c := &InitCommand{
Meta: m,
}
args := []string{}
if code := c.Run(args); code == 0 {
t.Fatalf("succeeded; wanted error\n%s", ui.OutputWriter.String())
}
if got, want := ui.ErrorWriter.String(), `Module installation was canceled by an interrupt signal`; !strings.Contains(got, want) {
t.Fatalf("wrong error message\nshould contain: %s\ngot:\n%s", want, got)
}
}
func TestInit_cancelProviders(t *testing.T) {
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
// This test runs `terraform init` as if SIGINT (or similar on other
// platforms) were sent to it, testing that it is interruptible.
td := tempDir(t)
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
testCopyDir(t, testFixturePath("init-required-providers"), td)
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
cli: Fix flaky init cancel test There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.
2021-10-18 22:54:31 +02:00
// Use a provider source implementation which is designed to hang indefinitely,
// to avoid a race between the closed shutdown channel and the provider source
// operations.
providerSource := &getproviders.HangingSource{}
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
cli: Fix flaky init cancel test There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.
2021-10-18 22:54:31 +02:00
// Our shutdown channel is pre-closed so init will exit as soon as it
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
// starts a cancelable portion of the process.
shutdownCh := make(chan struct{})
close(shutdownCh)
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
ProviderSource: providerSource,
ShutdownCh: shutdownCh,
}
c := &InitCommand{
Meta: m,
}
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
args := []string{}
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
if code := c.Run(args); code == 0 {
cli: Fix flaky init cancel test There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.
2021-10-18 22:54:31 +02:00
t.Fatalf("succeeded; wanted error\n%s", ui.OutputWriter.String())
command: Make provider installation interruptible In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.
2020-09-29 02:13:32 +02:00
}
// Currently the first operation that is cancelable is provider
// installation, so our error message comes from there. If we
// make the earlier steps cancelable in future then it'd be
// expected for this particular message to change.
if got, want := ui.ErrorWriter.String(), `Provider installation was canceled by an interrupt signal`; !strings.Contains(got, want) {
t.Fatalf("wrong error message\nshould contain: %s\ngot:\n%s", want, got)
}
}
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
func TestInit_getUpgradePlugins(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-providers"), td)
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
// looking for an exact version
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"exact": {"1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// config requires >= 2.3.3
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"greater-than": {"2.3.4", "2.3.3", "2.3.0"},
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
// config specifies > 1.0.0 , < 3.0.0
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"between": {"3.4.5", "2.3.4", "1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
})
defer close()
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
}
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
installFakeProviderPackages(t, &m, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"exact": {"0.0.1"},
"greater-than": {"2.3.3"},
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
})
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
}
args := []string{
"-upgrade=true",
}
if code := c.Run(args); code != 0 {
t.Fatalf("command did not complete successfully:\n%s", ui.ErrorWriter.String())
}
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
cacheDir := m.providerLocalCacheDir()
gotPackages := cacheDir.AllAvailablePackages()
wantPackages := map[addrs.Provider][]providercache.CachedProvider{
// "between" wasn't previously installed at all, so we installed
// the newest available version that matched the version constraints.
addrs.NewDefaultProvider("between"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("between"),
Version: getproviders.MustParseVersion("2.3.4"),
PackageDir: expectedPackageInstallPath("between", "2.3.4", false),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
},
},
// The existing version of "exact" did not match the version constraints,
// so we installed what the configuration selected as well.
addrs.NewDefaultProvider("exact"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("exact"),
Version: getproviders.MustParseVersion("1.2.3"),
PackageDir: expectedPackageInstallPath("exact", "1.2.3", false),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
},
// Previous version is still there, but not selected
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("exact"),
Version: getproviders.MustParseVersion("0.0.1"),
PackageDir: expectedPackageInstallPath("exact", "0.0.1", false),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
},
},
// The existing version of "greater-than" _did_ match the constraints,
// but a newer version was available and the user specified
// -upgrade and so we upgraded it anyway.
addrs.NewDefaultProvider("greater-than"): {
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("greater-than"),
Version: getproviders.MustParseVersion("2.3.4"),
PackageDir: expectedPackageInstallPath("greater-than", "2.3.4", false),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
},
// Previous version is still there, but not selected
{
providercache: Lazily detect executable file Instead of searching the installed provider package directory for a binary as we install it, we can lazily detect the executable as it is required. Doing so allows us to separately report an invalid unpacked package, giving the user more actionable error messages.
2020-07-07 20:36:04 +02:00
Provider: addrs.NewDefaultProvider("greater-than"),
Version: getproviders.MustParseVersion("2.3.3"),
PackageDir: expectedPackageInstallPath("greater-than", "2.3.3", false),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
},
},
}
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
if diff := cmp.Diff(wantPackages, gotPackages); diff != "" {
t.Errorf("wrong cache directory contents after upgrade\n%s", diff)
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
locks, err := m.lockedDependencies()
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
if err != nil {
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
t.Fatalf("failed to get locked dependencies: %s", err)
}
gotProviderLocks := locks.AllProviders()
wantProviderLocks := map[addrs.Provider]*depsfile.ProviderLock{
addrs.NewDefaultProvider("between"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("between"),
getproviders.MustParseVersion("2.3.4"),
getproviders.MustParseVersionConstraints("> 1.0.0, < 3.0.0"),
[]getproviders.Hash{
getproviders.HashScheme1.New("JVqAvZz88A+hS2wHVtTWQkHaxoA/LrUAz0H3jPBWPIA="),
},
),
addrs.NewDefaultProvider("exact"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("exact"),
getproviders.MustParseVersion("1.2.3"),
getproviders.MustParseVersionConstraints("= 1.2.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("H1TxWF8LyhBb6B4iUdKhLc/S9sC/jdcrCykpkbGcfbg="),
},
),
addrs.NewDefaultProvider("greater-than"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("greater-than"),
getproviders.MustParseVersion("2.3.4"),
getproviders.MustParseVersionConstraints(">= 2.3.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("SJPpXx/yoFE/W+7eCipjJ+G21xbdnTBD7lWodZ8hWkU="),
},
),
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
if diff := cmp.Diff(gotProviderLocks, wantProviderLocks, depsfile.ProviderLockComparer); diff != "" {
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
t.Errorf("wrong version selections after upgrade\n%s", diff)
}
command: init -upgrade for provider plugins Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.
2017-06-13 03:32:42 +02:00
}
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
func TestInit_getProviderMissing(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-providers"), td)
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
// looking for exact version 1.2.3
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"exact": {"1.2.4"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// config requires >= 2.3.3
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"greater-than": {"2.3.4", "2.3.3", "2.3.0"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// config specifies
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"between": {"3.4.5", "2.3.4", "1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
})
defer close()
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
args := []string{}
if code := c.Run(args); code == 0 {
internal: Verify provider signatures on install Providers installed from the registry are accompanied by a list of checksums (the "SHA256SUMS" file), which is cryptographically signed to allow package authentication. The process of verifying this has multiple steps: - First we must verify that the SHA256 hash of the package archive matches the expected hash. This could be done for local installations too, in the future. - Next we ensure that the expected hash returned as part of the registry API response matches an entry in the checksum list. - Finally we verify the cryptographic signature of the checksum list, using the public keys provided by the registry. Each of these steps is implemented as a separate PackageAuthentication type. The local archive installation mechanism uses only the archive checksum authenticator, and the HTTP installation uses all three in the order given. The package authentication system now also returns a result value, which is used by command/init to display the result of the authentication process. There are three tiers of signature, each of which is presented differently to the user: - Signatures from the embedded HashiCorp public key indicate that the provider is officially supported by HashiCorp; - If the signing key is not from HashiCorp, it may have an associated trust signature, which indicates that the provider is from one of HashiCorp's trusted partners; - Otherwise, if the signature is valid, this is a community provider.
2020-04-08 22:22:07 +02:00
t.Fatalf("expected error, got output: \n%s", ui.OutputWriter.String())
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
}
command: Fix TestInit_getProviderMissing This one just needs a tweak for the new wording of the error message.
2020-04-01 19:51:26 +02:00
if !strings.Contains(ui.ErrorWriter.String(), "no available releases match") {
add init getPlugin test add a mock plugin getter, and test that we can fetch requested version of the plugins.
2017-05-04 20:03:57 +02:00
t.Fatalf("unexpected error output: %s", ui.ErrorWriter)
}
}
command: Fix TestInit_checkRequiredVersion In prior refactoring we lost the required core version check from "terraform init", which we restore here. Additionally, this test used to have an incorrect name that suggested it was testing something in the "getProvider" codepath, but version checking happens regardless of what other options are selected.
2018-11-10 00:08:39 +01:00
func TestInit_checkRequiredVersion(t *testing.T) {
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-check-required-version"), td)
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
},
}
args := []string{}
if code := c.Run(args); code != 1 {
command: Fix several "terraform init" tests
2018-10-15 01:35:59 +02:00
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
}
terraform: Fix required version constraint diags If a module has multiple terraform.required_version constraints, any failures would point at the last constraint in the error diagnostics. If an earlier constraint was the actual problem, this leads to confusing errors like this: Error: Unsupported Terraform Core version on main.tf line 6, in terraform: 6: required_version = ">= 0.13.0" This configuration does not support Terraform version 0.13.0. The error was due to storing the declaration range of the constraint as a pointer to the contents of a loop variable, which was later overwritten in later iterations of the loop. Instead we now use HCL's handy Ptr() method to create a direct pointer to the range struct.
2020-08-18 15:18:00 +02:00
errStr := ui.ErrorWriter.String()
if !strings.Contains(errStr, `required_version = "~> 0.9.0"`) {
t.Fatalf("output should point to unmet version constraint, but is:\n\n%s", errStr)
}
if strings.Contains(errStr, `required_version = ">= 0.13.0"`) {
t.Fatalf("output should not point to met version constraint, but is:\n\n%s", errStr)
}
command/init: check required_version Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 20:25:16 +02:00
}
make sure required_version is checked before diags We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.
2021-09-28 18:38:40 +02:00
// Verify that init will error out with an invalid version constraint, even if
// there are other invalid configuration constructs.
func TestInit_checkRequiredVersionFirst(t *testing.T) {
LoadModule now always returns the module We don't need to load the configuration twice, since configload can return the module for us.
2021-09-28 19:06:22 +02:00
t.Run("root_module", func(t *testing.T) {
td := t.TempDir()
testCopyDir(t, testFixturePath("init-check-required-version-first"), td)
defer testChdir(t, td)()
make sure required_version is checked before diags We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.
2021-09-28 18:38:40 +02:00
LoadModule now always returns the module We don't need to load the configuration twice, since configload can return the module for us.
2021-09-28 19:06:22 +02:00
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
View: view,
},
}
make sure required_version is checked before diags We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.
2021-09-28 18:38:40 +02:00
LoadModule now always returns the module We don't need to load the configuration twice, since configload can return the module for us.
2021-09-28 19:06:22 +02:00
args := []string{}
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
errStr := ui.ErrorWriter.String()
if !strings.Contains(errStr, `Unsupported Terraform Core version`) {
t.Fatalf("output should point to unmet version constraint, but is:\n\n%s", errStr)
}
})
t.Run("sub_module", func(t *testing.T) {
td := t.TempDir()
testCopyDir(t, testFixturePath("init-check-required-version-first-module"), td)
defer testChdir(t, td)()
ui := cli.NewMockUi()
view, _ := testView(t)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
View: view,
},
}
args := []string{}
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
errStr := ui.ErrorWriter.String()
if !strings.Contains(errStr, `Unsupported Terraform Core version`) {
t.Fatalf("output should point to unmet version constraint, but is:\n\n%s", errStr)
}
})
make sure required_version is checked before diags We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.
2021-09-28 18:38:40 +02:00
}
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
func TestInit_providerLockFile(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-provider-lock-file"), td)
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
providerSource, close := newMockProviderSource(t, map[string][]string{
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
"test": {"1.2.3"},
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
})
defer close()
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
plugin/discovery: Installer interface, and provider implementation Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.
2017-06-13 03:22:47 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
args := []string{}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
lockFile := ".terraform.lock.hcl"
buf, err := ioutil.ReadFile(lockFile)
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
if err != nil {
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
t.Fatalf("failed to read dependency lock file %s: %s", lockFile, err)
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
buf = bytes.TrimSpace(buf)
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// The hash in here is for the fake package that newMockProviderSource produces
// (so it'll change if newMockProviderSource starts producing different contents)
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
wantLockFile := strings.TrimSpace(`
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/test" {
version = "1.2.3"
constraints = "1.2.3"
internal/depsfile: Control how the "hashes" value is formatted Previously we were just letting hclwrite do its default formatting behavior here. The current behavior there isn't ideal anyway -- it puts big data structures all on one line -- but even ignoring that our goal for this file format is to keep things in a highly-normalized shape so that diffs against the file are clear and easy to read. With that in mind, here we directly control how we write that value into the file, which means that later changes to hclwrite's list/set presentation won't affect it, regardless of what form they take.
2020-10-08 20:11:24 +02:00
hashes = [
"h1:wlbEC2mChQZ2hhgUhl6SeVLPP7fMqOFUZAQhQ9GIIno=",
]
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
`)
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
if diff := cmp.Diff(wantLockFile, string(buf)); diff != "" {
t.Errorf("wrong dependency lock file contents\n%s", diff)
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
cli: Only rewrite provider locks file if changed If the provider locks have not changed, there is no need to rewrite the locks file. Preventing this needless rewrite should allow Terraform to operate in a read-only directory, so long as the provider requirements don't change.
2021-03-29 22:03:29 +02:00
// Make the local directory read-only, and verify that rerunning init
// succeeds, to ensure that we don't try to rewrite an unchanged lock file
os.Chmod(".", 0555)
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
command: produce provider lock file during "terraform init" Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.
2017-05-25 02:35:46 +02:00
}
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
command/init: Add a new flag `-lockfile=readonly` (#27630) Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-03-09 17:12:00 +01:00
func TestInit_providerLockFileReadonly(t *testing.T) {
// The hash in here is for the fake package that newMockProviderSource produces
// (so it'll change if newMockProviderSource starts producing different contents)
inputLockFile := strings.TrimSpace(`
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/test" {
version = "1.2.3"
constraints = "1.2.3"
hashes = [
"zh:e919b507a91e23a00da5c2c4d0b64bcc7900b68d43b3951ac0f6e5d80387fbdc",
]
}
`)
badLockFile := strings.TrimSpace(`
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/test" {
version = "1.2.3"
constraints = "1.2.3"
hashes = [
"zh:0000000000000000000000000000000000000000000000000000000000000000",
]
}
`)
updatedLockFile := strings.TrimSpace(`
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/test" {
version = "1.2.3"
constraints = "1.2.3"
hashes = [
"h1:wlbEC2mChQZ2hhgUhl6SeVLPP7fMqOFUZAQhQ9GIIno=",
"zh:e919b507a91e23a00da5c2c4d0b64bcc7900b68d43b3951ac0f6e5d80387fbdc",
]
}
providercache: Discard lock entries for unused providers Previously we would only ever add new lock entries or update existing ones. However, it's possible that over time a module may _cease_ using a particular provider, at which point we ought to remove it from the lock file so that operations won't fail when seeing that the provider cache directory is inconsistent with the lock file. Now the provider installer (EnsureProviderVersions) will remove any lock file entries that relate to providers not included in the given requirements, which therefore makes the resulting lock file properly match the set of packages the installer wrote into the cache. This does potentially mean that someone could inadvertently defeat the lock by removing a provider dependency, running "terraform init", then undoing that removal, and finally running "terraform init" again. However, that seems relatively unlikely compared to the likelihood of removing a provider and keeping it removed, and in the event it _did_ happen the changes to the lock entry for that provider would be visible in the diff of the provider lock file as usual, and so could be noticed in code review just as for any other change to dependencies.
2021-12-17 00:57:47 +01:00
`)
emptyUpdatedLockFile := strings.TrimSpace(`
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
command/init: Add a new flag `-lockfile=readonly` (#27630) Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-03-09 17:12:00 +01:00
`)
cases := []struct {
desc string
fixture string
providers map[string][]string
input string
args []string
ok bool
want string
}{
{
desc: "default",
fixture: "init-provider-lock-file",
providers: map[string][]string{"test": {"1.2.3"}},
input: inputLockFile,
args: []string{},
ok: true,
want: updatedLockFile,
},
providercache: Discard lock entries for unused providers Previously we would only ever add new lock entries or update existing ones. However, it's possible that over time a module may _cease_ using a particular provider, at which point we ought to remove it from the lock file so that operations won't fail when seeing that the provider cache directory is inconsistent with the lock file. Now the provider installer (EnsureProviderVersions) will remove any lock file entries that relate to providers not included in the given requirements, which therefore makes the resulting lock file properly match the set of packages the installer wrote into the cache. This does potentially mean that someone could inadvertently defeat the lock by removing a provider dependency, running "terraform init", then undoing that removal, and finally running "terraform init" again. However, that seems relatively unlikely compared to the likelihood of removing a provider and keeping it removed, and in the event it _did_ happen the changes to the lock entry for that provider would be visible in the diff of the provider lock file as usual, and so could be noticed in code review just as for any other change to dependencies.
2021-12-17 00:57:47 +01:00
{
desc: "unused provider",
fixture: "init-provider-now-unused",
providers: map[string][]string{"test": {"1.2.3"}},
input: inputLockFile,
args: []string{},
ok: true,
want: emptyUpdatedLockFile,
},
command/init: Add a new flag `-lockfile=readonly` (#27630) Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-03-09 17:12:00 +01:00
{
desc: "readonly",
fixture: "init-provider-lock-file",
providers: map[string][]string{"test": {"1.2.3"}},
input: inputLockFile,
args: []string{"-lockfile=readonly"},
ok: true,
want: inputLockFile,
},
providercache: Discard lock entries for unused providers Previously we would only ever add new lock entries or update existing ones. However, it's possible that over time a module may _cease_ using a particular provider, at which point we ought to remove it from the lock file so that operations won't fail when seeing that the provider cache directory is inconsistent with the lock file. Now the provider installer (EnsureProviderVersions) will remove any lock file entries that relate to providers not included in the given requirements, which therefore makes the resulting lock file properly match the set of packages the installer wrote into the cache. This does potentially mean that someone could inadvertently defeat the lock by removing a provider dependency, running "terraform init", then undoing that removal, and finally running "terraform init" again. However, that seems relatively unlikely compared to the likelihood of removing a provider and keeping it removed, and in the event it _did_ happen the changes to the lock entry for that provider would be visible in the diff of the provider lock file as usual, and so could be noticed in code review just as for any other change to dependencies.
2021-12-17 00:57:47 +01:00
{
desc: "unused provider readonly",
fixture: "init-provider-now-unused",
providers: map[string][]string{"test": {"1.2.3"}},
input: inputLockFile,
args: []string{"-lockfile=readonly"},
ok: false,
want: inputLockFile,
},
command/init: Add a new flag `-lockfile=readonly` (#27630) Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-03-09 17:12:00 +01:00
{
desc: "conflict",
fixture: "init-provider-lock-file",
providers: map[string][]string{"test": {"1.2.3"}},
input: inputLockFile,
args: []string{"-lockfile=readonly", "-upgrade"},
ok: false,
want: inputLockFile,
},
{
desc: "checksum mismatch",
fixture: "init-provider-lock-file",
providers: map[string][]string{"test": {"1.2.3"}},
input: badLockFile,
args: []string{"-lockfile=readonly"},
ok: false,
want: badLockFile,
},
{
desc: "reject to change required provider dependences",
fixture: "init-provider-lock-file-readonly-add",
providers: map[string][]string{
"test": {"1.2.3"},
"foo": {"1.0.0"},
},
input: inputLockFile,
args: []string{"-lockfile=readonly"},
ok: false,
want: inputLockFile,
},
}
for _, tc := range cases {
t.Run(tc.desc, func(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
testCopyDir(t, testFixturePath(tc.fixture), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
providerSource, close := newMockProviderSource(t, tc.providers)
defer close()
ui := new(cli.MockUi)
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
// write input lockfile
lockFile := ".terraform.lock.hcl"
if err := ioutil.WriteFile(lockFile, []byte(tc.input), 0644); err != nil {
t.Fatalf("failed to write input lockfile: %s", err)
}
code := c.Run(tc.args)
if tc.ok && code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if !tc.ok && code == 0 {
t.Fatalf("expected error, got output: \n%s", ui.OutputWriter.String())
}
buf, err := ioutil.ReadFile(lockFile)
if err != nil {
t.Fatalf("failed to read dependency lock file %s: %s", lockFile, err)
}
buf = bytes.TrimSpace(buf)
if diff := cmp.Diff(tc.want, string(buf)); diff != "" {
t.Errorf("wrong dependency lock file contents\n%s", diff)
}
})
}
}
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
func TestInit_pluginDirReset(t *testing.T) {
cleanup temp files from command tests Rather than try to modify all the hundreds of calls to the temp helper functions, and cleanup the temp files at every call site, have all tests work within a single temp directory that is removed at the end of TestMain.
2018-03-28 19:08:38 +02:00
td := testTempDir(t)
command/show: fix issue with show and aliased provider (#23848) The formatter in `command/format/state.go`, when formatting a resource with an aliased provider, was looking for a schema with the alias (ie, test.foo), but the schemas are only listed by provider type (test). Update the state formatter to lookup schemas by provider type only. Some of the show tests (and a couple others) were not properly cleaning up the created tmpdirs, so I fixed those. Also, the show tests are using a statefile named `state.tfstate`, but were not passing that path to the show command, so we were getting some false positives (a `show` command that returns `no state` exits 0). Fixes #21462
2020-01-13 21:10:00 +01:00
defer os.RemoveAll(td)
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// An empty provider source
providerSource, close := newMockProviderSource(t, nil)
defer close()
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
},
}
// make our vendor paths
pluginPath := []string{"a", "b", "c"}
for _, p := range pluginPath {
if err := os.MkdirAll(p, 0755); err != nil {
t.Fatal(err)
}
}
// run once and save the -plugin-dir
args := []string{"-plugin-dir", "a"}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter)
}
pluginDirs, err := c.loadPluginPath()
if err != nil {
t.Fatal(err)
}
if len(pluginDirs) != 1 || pluginDirs[0] != "a" {
t.Fatalf(`expected plugin dir ["a"], got %q`, pluginDirs)
}
ui = new(cli.MockUi)
c = &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource, // still empty
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
},
}
// make sure we remove the plugin-dir record
remove the plugin path only for an empty string To avoid breaking automation where plugin-path was assumed to be set permanently, only remove the plugin-path record if it was explicitly set to and empty string.
2018-01-04 22:49:45 +01:00
args = []string{"-plugin-dir="}
if code := c.Run(args); code != 0 {
allow init to reset -plugin-dir Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 17:21:07 +01:00
t.Fatalf("bad: \n%s", ui.ErrorWriter)
}
pluginDirs, err = c.loadPluginPath()
if err != nil {
t.Fatal(err)
}
if len(pluginDirs) != 0 {
t.Fatalf("expected no plugin dirs got %q", pluginDirs)
}
}
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
// Test user-supplied -plugin-dir
func TestInit_pluginDirProviders(t *testing.T) {
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-providers"), td)
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// An empty provider source
providerSource, close := newMockProviderSource(t, nil)
defer close()
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
ui := new(cli.MockUi)
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
c := &InitCommand{
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
Meta: m,
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
// make our vendor paths
pluginPath := []string{"a", "b", "c"}
for _, p := range pluginPath {
if err := os.MkdirAll(p, 0755); err != nil {
t.Fatal(err)
}
}
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
// We'll put some providers in our plugin dirs. To do this, we'll pretend
// for a moment that they are provider cache directories just because that
// allows us to lean on our existing test helper functions to do this.
for i, def := range [][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
{"exact", "1.2.3"},
{"greater-than", "2.3.4"},
{"between", "2.3.4"},
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
} {
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
name, version := def[0], def[1]
dir := providercache.NewDir(pluginPath[i])
installFakeProviderPackagesElsewhere(t, dir, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
name: {version},
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
})
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
args := []string{
"-plugin-dir", "a",
"-plugin-dir", "b",
"-plugin-dir", "c",
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter)
}
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
locks, err := m.lockedDependencies()
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
if err != nil {
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
t.Fatalf("failed to get locked dependencies: %s", err)
}
gotProviderLocks := locks.AllProviders()
wantProviderLocks := map[addrs.Provider]*depsfile.ProviderLock{
addrs.NewDefaultProvider("between"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("between"),
getproviders.MustParseVersion("2.3.4"),
getproviders.MustParseVersionConstraints("> 1.0.0, < 3.0.0"),
[]getproviders.Hash{
getproviders.HashScheme1.New("JVqAvZz88A+hS2wHVtTWQkHaxoA/LrUAz0H3jPBWPIA="),
},
),
addrs.NewDefaultProvider("exact"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("exact"),
getproviders.MustParseVersion("1.2.3"),
getproviders.MustParseVersionConstraints("= 1.2.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("H1TxWF8LyhBb6B4iUdKhLc/S9sC/jdcrCykpkbGcfbg="),
},
),
addrs.NewDefaultProvider("greater-than"): depsfile.NewProviderLock(
addrs.NewDefaultProvider("greater-than"),
getproviders.MustParseVersion("2.3.4"),
getproviders.MustParseVersionConstraints(">= 2.3.3"),
[]getproviders.Hash{
getproviders.HashScheme1.New("SJPpXx/yoFE/W+7eCipjJ+G21xbdnTBD7lWodZ8hWkU="),
},
),
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
if diff := cmp.Diff(gotProviderLocks, wantProviderLocks, depsfile.ProviderLockComparer); diff != "" {
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
t.Errorf("wrong version selections after upgrade\n%s", diff)
}
// -plugin-dir overrides the normal provider source, so it should not have
// seen any calls at all.
if calls := providerSource.CallLog(); len(calls) > 0 {
t.Errorf("unexpected provider source calls (want none)\n%s", spew.Sdump(calls))
}
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
// Test user-supplied -plugin-dir doesn't allow auto-install
func TestInit_pluginDirProvidersDoesNotGet(t *testing.T) {
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-get-providers"), td)
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
// Our provider source has a suitable package for "between" available,
// but we should ignore it because -plugin-dir is set and thus this
// source is temporarily overridden during install.
providerSource, close := newMockProviderSource(t, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
"between": {"2.3.4"},
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
})
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
defer close()
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
c := &InitCommand{
Meta: m,
}
// make our vendor paths
pluginPath := []string{"a", "b"}
for _, p := range pluginPath {
if err := os.MkdirAll(p, 0755); err != nil {
t.Fatal(err)
}
}
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
// We'll put some providers in our plugin dirs. To do this, we'll pretend
// for a moment that they are provider cache directories just because that
// allows us to lean on our existing test helper functions to do this.
for i, def := range [][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
{"exact", "1.2.3"},
{"greater-than", "2.3.4"},
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
} {
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
name, version := def[0], def[1]
dir := providercache.NewDir(pluginPath[i])
installFakeProviderPackagesElsewhere(t, dir, map[string][]string{
cli: Remove legacy positional path arguments Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!
2021-02-02 16:35:45 +01:00
name: {version},
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
})
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
args := []string{
"-plugin-dir", "a",
"-plugin-dir", "b",
}
if code := c.Run(args); code == 0 {
// should have been an error
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
t.Fatalf("succeeded; want error\nstdout:\n%s\nstderr\n%s", ui.OutputWriter, ui.ErrorWriter)
}
// The error output should mention the "between" provider but should not
// mention either the "exact" or "greater-than" provider, because the
// latter two are available via the -plugin-dir directories.
errStr := ui.ErrorWriter.String()
if subStr := "hashicorp/between"; !strings.Contains(errStr, subStr) {
t.Errorf("error output should mention the 'between' provider\nwant substr: %s\ngot:\n%s", subStr, errStr)
}
if subStr := "hashicorp/exact"; strings.Contains(errStr, subStr) {
t.Errorf("error output should not mention the 'exact' provider\ndo not want substr: %s\ngot:\n%s", subStr, errStr)
}
if subStr := "hashicorp/greater-than"; strings.Contains(errStr, subStr) {
t.Errorf("error output should not mention the 'greater-than' provider\ndo not want substr: %s\ngot:\n%s", subStr, errStr)
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
if calls := providerSource.CallLog(); len(calls) > 0 {
t.Errorf("unexpected provider source calls (want none)\n%s", spew.Sdump(calls))
}
add -plugin-dir option The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 21:23:16 +02:00
}
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
// Verify that plugin-dir doesn't prevent discovery of internal providers
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
func TestInit_pluginDirWithBuiltIn(t *testing.T) {
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-internal"), td)
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// An empty provider source
providerSource, close := newMockProviderSource(t, nil)
defer close()
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
ProviderSource: providerSource,
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
}
c := &InitCommand{
Meta: m,
}
args := []string{"-plugin-dir", "./"}
if code := c.Run(args); code != 0 {
t.Fatalf("error: %s", ui.ErrorWriter)
}
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
outputStr := ui.OutputWriter.String()
if subStr := "terraform.io/builtin/terraform is built in to Terraform"; !strings.Contains(outputStr, subStr) {
t.Errorf("output should mention the terraform provider\nwant substr: %s\ngot:\n%s", subStr, outputStr)
}
}
func TestInit_invalidBuiltInProviders(t *testing.T) {
// This test fixture includes two invalid provider dependencies:
// - an implied dependency on terraform.io/builtin/terraform with an
// explicit version number, which is not allowed because it's builtin.
// - an explicit dependency on terraform.io/builtin/nonexist, which does
// not exist at all.
td := tempDir(t)
deprecate helper/copy helper/copy CopyDir was used heavily in tests. It differes from internal/copydir in a few ways, the main one being that it creates the dst directory while the internal version expected the dst to exist (there are other differences, which is why I did not just switch tests to using internal's CopyDir). I moved the CopyDir func from helper/copy into command_test.go; I could also have moved it into internal/copy and named it something like CreateDirAndCopy so if that seems like a better option please let me know. helper/copy/CopyFile was used in a couple of spots so I moved it into internal, at which point I thought it made more sense to rename the package copy (instead of copydir). There's also a `go mod tidy` included.
2020-10-07 18:48:25 +02:00
testCopyDir(t, testFixturePath("init-internal-invalid"), td)
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
defer os.RemoveAll(td)
defer testChdir(t, td)()
// An empty provider source
providerSource, close := newMockProviderSource(t, nil)
defer close()
ui := cli.NewMockUi()
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
view, _ := testView(t)
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
clistate: Update clistate.Locker for command views The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.
2021-02-16 13:19:22 +01:00
View: view,
internal/providercache: Handle built-in providers Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.
2020-04-02 01:44:50 +02:00
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
if code := c.Run(nil); code == 0 {
t.Fatalf("succeeded, but was expecting error\nstdout:\n%s\nstderr:\n%s", ui.OutputWriter, ui.ErrorWriter)
}
errStr := ui.ErrorWriter.String()
if subStr := "Cannot use terraform.io/builtin/terraform: built-in"; !strings.Contains(errStr, subStr) {
t.Errorf("error output should mention the terraform provider\nwant substr: %s\ngot:\n%s", subStr, errStr)
}
if subStr := "Cannot use terraform.io/builtin/nonexist: this Terraform release"; !strings.Contains(errStr, subStr) {
t.Errorf("error output should mention the 'nonexist' provider\nwant substr: %s\ngot:\n%s", subStr, errStr)
}
add test for internal plugins with -plugin-dir -plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 17:51:09 +01:00
}
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
// newMockProviderSource is a helper to succinctly construct a mock provider
// source that contains a set of packages matching the given provider versions
// that are available for installation (from temporary local files).
//
// The caller must call the returned close callback once the source is no
// longer needed, at which point it will clean up all of the temporary files
// and the packages in the source will no longer be available for installation.
//
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
// Provider addresses must be valid source strings, and passing only the
// provider name will be interpreted as a "default" provider under
// registry.terraform.io/hashicorp. If you need more control over the
// provider addresses, pass a full provider source string.
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
//
// This function also registers providers as belonging to the current platform,
// to ensure that they will be available to a provider installer operating in
// its default configuration.
//
// In case of any errors while constructing the source, this function will
// abort the current test using the given testing.T. Therefore a caller can
// assume that if this function returns then the result is valid and ready
// to use.
func newMockProviderSource(t *testing.T, availableProviderVersions map[string][]string) (source *getproviders.MockSource, close func()) {
t.Helper()
var packages []getproviders.PackageMeta
var closes []func()
close = func() {
for _, f := range closes {
f()
}
}
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
for source, versions := range availableProviderVersions {
addr := addrs.MustParseProviderSourceString(source)
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
for _, versionStr := range versions {
version, err := getproviders.ParseVersion(versionStr)
if err != nil {
close()
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
t.Fatalf("failed to parse %q as a version number for %q: %s", versionStr, addr.ForDisplay(), err)
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
}
getproviders: FakeInstallablePackageMeta filename Add an optional execFilename argument to the test helper function FakeInstallablePackageMeta, which allows the creation of invalid packages.
2020-07-07 20:41:45 +02:00
meta, close, err := getproviders.FakeInstallablePackageMeta(addr, version, getproviders.VersionList{getproviders.MustParseVersion("5.0")}, getproviders.CurrentPlatform, "")
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
if err != nil {
close()
command/init: Use full config for provider reqs Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.
2020-05-25 22:38:01 +02:00
t.Fatalf("failed to prepare fake package for %s %s: %s", addr.ForDisplay(), versionStr, err)
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
}
closes = append(closes, close)
packages = append(packages, meta)
}
}
internal/getproviders: decode and return any registry warnings (#25337) * internal/getproviders: decode and return any registry warnings The public registry may include a list of warnings in the "versions" response for any given provider. This PR adds support for warnings from the registry and an installer event to return those warnings to the user.
2020-06-25 16:49:48 +02:00
return getproviders.NewMockSource(packages, nil), close
command: Make the tests compile again They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.
2020-03-31 23:02:40 +02:00
}
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
// installFakeProviderPackages installs a fake package for the given provider
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
// names (interpreted as a "default" provider address) and versions into the
// local plugin cache for the given "meta".
//
// Any test using this must be using testChdir or some similar mechanism to
// make sure that it isn't writing directly into a test fixture or source
// directory within the codebase.
//
// If a requested package cannot be installed for some reason, this function
// will abort the test using the given testing.T. Therefore if this function
// returns the caller can assume that the requested providers have been
// installed.
func installFakeProviderPackages(t *testing.T, meta *Meta, providerVersions map[string][]string) {
t.Helper()
command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.
2020-04-01 19:46:38 +02:00
cacheDir := meta.providerLocalCacheDir()
installFakeProviderPackagesElsewhere(t, cacheDir, providerVersions)
}
// installFakeProviderPackagesElsewhere is a variant of installFakeProviderPackages
// that will install packages into the given provider cache directory, rather
// than forcing the use of the local cache of the current "Meta".
func installFakeProviderPackagesElsewhere(t *testing.T, cacheDir *providercache.Dir, providerVersions map[string][]string) {
t.Helper()
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
// It can be hard to spot the mistake of forgetting to run testChdir before
// modifying the working directory, so we'll use a simple heuristic here
// to try to detect that mistake and make a noisy error about it instead.
wd, err := os.Getwd()
if err == nil {
wd = filepath.Clean(wd)
// If the directory we're in is named "command" or if we're under a
// directory named "testdata" then we'll assume a mistake and generate
// an error. This will cause the test to fail but won't block it from
// running.
if filepath.Base(wd) == "command" || filepath.Base(wd) == "testdata" || strings.Contains(filepath.ToSlash(wd), "/testdata/") {
t.Errorf("installFakeProviderPackage may be used only by tests that switch to a temporary working directory, e.g. using testChdir")
}
}
for name, versions := range providerVersions {
addr := addrs.NewDefaultProvider(name)
for _, versionStr := range versions {
version, err := getproviders.ParseVersion(versionStr)
if err != nil {
t.Fatalf("failed to parse %q as a version number for %q: %s", versionStr, name, err)
}
getproviders: FakeInstallablePackageMeta filename Add an optional execFilename argument to the test helper function FakeInstallablePackageMeta, which allows the creation of invalid packages.
2020-07-07 20:41:45 +02:00
meta, close, err := getproviders.FakeInstallablePackageMeta(addr, version, getproviders.VersionList{getproviders.MustParseVersion("5.0")}, getproviders.CurrentPlatform, "")
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
// We're going to install all these fake packages before we return,
// so we don't need to preserve them afterwards.
defer close()
if err != nil {
t.Fatalf("failed to prepare fake package for %s %s: %s", name, versionStr, err)
}
command/init: Read, respect, and update provider dependency locks This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.
2020-10-03 01:41:56 +02:00
_, err = cacheDir.InstallPackage(context.Background(), meta, nil)
command/init: Fix TestInit_getUpgradePlugins This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.
2020-04-01 01:48:37 +02:00
if err != nil {
t.Fatalf("failed to install fake package for %s %s: %s", name, versionStr, err)
}
}
}
}
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
// expectedPackageInstallPath is a companion to installFakeProviderPackages
// that returns the path where the provider with the given name and version
// would be installed and, relatedly, where the installer will expect to
// find an already-installed version.
//
// Just as with installFakeProviderPackages, this function is a shortcut helper
// for "default-namespaced" providers as we commonly use in tests. If you need
// more control over the provider addresses, use functions of the underlying
// getproviders and providercache packages instead.
//
// The result always uses forward slashes, even on Windows, for consistency
// with how the getproviders and providercache packages build paths.
func expectedPackageInstallPath(name, version string, exe bool) string {
platform := getproviders.CurrentPlatform
command: new cache directory .terraform/providers for providers Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.
2020-10-14 00:03:56 +02:00
baseDir := ".terraform/providers"
command/init: Fix TestInit_providerSource This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-01 02:31:03 +02:00
if exe {
p := fmt.Sprintf("registry.terraform.io/hashicorp/%s/%s/%s/terraform-provider-%s_%s", name, version, platform, name, version)
if platform.OS == "windows" {
p += ".exe"
}
return filepath.ToSlash(filepath.Join(baseDir, p))
}
return filepath.ToSlash(filepath.Join(
baseDir, fmt.Sprintf("registry.terraform.io/hashicorp/%s/%s/%s", name, version, platform),
))
}