2019-01-09 03:39:14 +01:00
package initwd
2018-02-13 23:40:53 +01:00
import (
initwd: Error message for local paths escaping module packages
Our module installer has a somewhat-informal idea of a "module package",
which is some external thing we can go fetch in order to add one or more
modules to the current configuration. Our documentation doesn't talk much
about it because most users seem to have found the distinction between
external and local modules pretty intuitive without us throwing a lot of
funny terminology at them, but there are some situations where the
distinction between a module and a module package are material to the
end-user.
One such situation is when using an absolute rather than relative
filesystem path: we treat that as an external package in order to make the
resulting working directory theoretically "portable" (although users can
do various other things to defeat that), and so Terraform will copy the
directory into .terraform/modules in the same way as it would download and
extract a remote archive package or clone a git repository.
A consequence of this, though, is that any relative paths called from
inside a module loaded from an absolute path will fail if they try to
traverse upward into the parent directory, because at runtime we're
actually running from a copy of the directory that's been taking out of
its original context.
A similar sort of situation can occur in a truly remote module package if
the author accidentally writes a "../" source path that traverses up out
of the package root, and so this commit introduces a special error message
for both situations that tries to be a bit clearer about there being a
package boundary and use that to explain why installation failed.
We would ideally have made escaping local references like that illegal in
the first place, but sadly we did not and so when we rebuilt the module
installer for Terraform v0.12 we ended up keeping the previous behavior of
just trying it and letting it succeed if there happened to somehow be a
matching directory at the given path, in order to remain compatible with
situations that had worked by coincidence rather than intention. For that
same reason, I've implemented this as a replacement error message we will
return only if local module installation was going to fail anyway, and
thus it only modifies the error message for some existing error situations
rather than introducing new error situations.
This also includes some light updates to the documentation to say a little
more about how Terraform treats absolute paths, though aiming not to get
too much into the weeds about module packages since it's something that
most users can get away with never knowing.
2021-05-22 00:28:20 +02:00
"bytes"
2019-01-09 03:39:14 +01:00
"flag"
"fmt"
"io/ioutil"
2018-02-14 01:36:36 +01:00
"os"
2018-02-13 23:40:53 +01:00
"path/filepath"
2018-02-15 18:23:07 +01:00
"strings"
2018-02-13 23:40:53 +01:00
"testing"
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
"github.com/davecgh/go-spew/spew"
2019-01-09 03:39:14 +01:00
"github.com/go-test/deep"
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
"github.com/google/go-cmp/cmp"
2018-02-13 23:40:53 +01:00
version "github.com/hashicorp/go-version"
2021-05-17 21:17:09 +02:00
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/configs/configload"
2020-10-07 18:48:25 +02:00
"github.com/hashicorp/terraform/internal/copy"
2021-05-17 18:45:36 +02:00
"github.com/hashicorp/terraform/internal/registry"
2021-05-17 19:11:06 +02:00
"github.com/hashicorp/terraform/internal/tfdiags"
2020-10-18 16:01:48 +02:00
_ "github.com/hashicorp/terraform/internal/logging"
2018-02-13 23:40:53 +01:00
)
2019-01-09 03:39:14 +01:00
func TestMain ( m * testing . M ) {
flag . Parse ( )
os . Exit ( m . Run ( ) )
}
func TestModuleInstaller ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/local-modules" )
2019-01-09 03:39:14 +01:00
dir , done := tempChdir ( t , fixtureDir )
2018-02-14 01:36:36 +01:00
defer done ( )
2018-02-13 23:40:53 +01:00
hooks := & testInstallHooks { }
2019-01-09 03:39:14 +01:00
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.
This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.
In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.
Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.
The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
excludes Terraform versions prior to v0.12 then the configuration is
probably _already_ upgraded and so it's just a normal syntax error,
even if the early loader didn't detect it.
Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
_ , diags := inst . InstallModules ( "." , false , hooks )
2018-02-13 23:40:53 +01:00
assertNoDiagnostics ( t , diags )
wantCalls := [ ] testInstallHookCall {
{
Name : "Install" ,
ModuleAddr : "child_a" ,
PackageAddr : "" ,
2018-02-14 01:36:36 +01:00
LocalPath : "child_a" ,
2018-02-13 23:40:53 +01:00
} ,
{
Name : "Install" ,
ModuleAddr : "child_a.child_b" ,
PackageAddr : "" ,
2018-02-14 01:36:36 +01:00
LocalPath : "child_a/child_b" ,
2018-02-13 23:40:53 +01:00
} ,
}
2018-02-14 01:36:36 +01:00
if assertResultDeepEqual ( t , hooks . Calls , wantCalls ) {
return
}
2019-01-09 03:39:14 +01:00
loader , err := configload . NewLoader ( & configload . Config {
ModulesDir : modulesDir ,
} )
if err != nil {
t . Fatal ( err )
}
2018-02-14 01:36:36 +01:00
// Make sure the configuration is loadable now.
// (This ensures that correct information is recorded in the manifest.)
2018-02-15 18:23:07 +01:00
config , loadDiags := loader . LoadConfig ( "." )
2019-01-09 03:39:14 +01:00
assertNoDiagnostics ( t , tfdiags . Diagnostics { } . Append ( loadDiags ) )
2018-02-15 18:23:07 +01:00
wantTraces := map [ string ] string {
"" : "in root module" ,
"child_a" : "in child_a module" ,
"child_a.child_b" : "in child_b module" ,
}
gotTraces := map [ string ] string { }
config . DeepEach ( func ( c * configs . Config ) {
path := strings . Join ( c . Path , "." )
if c . Module . Variables [ "v" ] == nil {
gotTraces [ path ] = "<missing>"
return
}
varDesc := c . Module . Variables [ "v" ] . Description
gotTraces [ path ] = varDesc
} )
assertResultDeepEqual ( t , gotTraces , wantTraces )
2018-02-14 01:36:36 +01:00
}
2019-03-11 23:25:21 +01:00
func TestModuleInstaller_error ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/local-module-error" )
2019-03-11 23:25:21 +01:00
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
_ , diags := inst . InstallModules ( "." , false , hooks )
if ! diags . HasErrors ( ) {
t . Fatal ( "expected error" )
} else {
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
assertDiagnosticSummary ( t , diags , "Invalid module source address" )
2019-03-11 23:25:21 +01:00
}
}
initwd: Error message for local paths escaping module packages
Our module installer has a somewhat-informal idea of a "module package",
which is some external thing we can go fetch in order to add one or more
modules to the current configuration. Our documentation doesn't talk much
about it because most users seem to have found the distinction between
external and local modules pretty intuitive without us throwing a lot of
funny terminology at them, but there are some situations where the
distinction between a module and a module package are material to the
end-user.
One such situation is when using an absolute rather than relative
filesystem path: we treat that as an external package in order to make the
resulting working directory theoretically "portable" (although users can
do various other things to defeat that), and so Terraform will copy the
directory into .terraform/modules in the same way as it would download and
extract a remote archive package or clone a git repository.
A consequence of this, though, is that any relative paths called from
inside a module loaded from an absolute path will fail if they try to
traverse upward into the parent directory, because at runtime we're
actually running from a copy of the directory that's been taking out of
its original context.
A similar sort of situation can occur in a truly remote module package if
the author accidentally writes a "../" source path that traverses up out
of the package root, and so this commit introduces a special error message
for both situations that tries to be a bit clearer about there being a
package boundary and use that to explain why installation failed.
We would ideally have made escaping local references like that illegal in
the first place, but sadly we did not and so when we rebuilt the module
installer for Terraform v0.12 we ended up keeping the previous behavior of
just trying it and letting it succeed if there happened to somehow be a
matching directory at the given path, in order to remain compatible with
situations that had worked by coincidence rather than intention. For that
same reason, I've implemented this as a replacement error message we will
return only if local module installation was going to fail anyway, and
thus it only modifies the error message for some existing error situations
rather than introducing new error situations.
This also includes some light updates to the documentation to say a little
more about how Terraform treats absolute paths, though aiming not to get
too much into the weeds about module packages since it's something that
most users can get away with never knowing.
2021-05-22 00:28:20 +02:00
func TestModuleInstaller_packageEscapeError ( t * testing . T ) {
fixtureDir := filepath . Clean ( "testdata/load-module-package-escape" )
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
// For this particular test we need an absolute path in the root module
// that must actually resolve to our temporary directory in "dir", so
// we need to do a little rewriting. We replace the arbitrary placeholder
// %%BASE%% with the temporary directory path.
{
rootFilename := filepath . Join ( dir , "package-escape.tf" )
template , err := ioutil . ReadFile ( rootFilename )
if err != nil {
t . Fatal ( err )
}
final := bytes . ReplaceAll ( template , [ ] byte ( "%%BASE%%" ) , [ ] byte ( filepath . ToSlash ( dir ) ) )
err = ioutil . WriteFile ( rootFilename , final , 0644 )
if err != nil {
t . Fatal ( err )
}
}
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
_ , diags := inst . InstallModules ( "." , false , hooks )
if ! diags . HasErrors ( ) {
t . Fatal ( "expected error" )
} else {
assertDiagnosticSummary ( t , diags , "Local module path escapes module package" )
}
}
func TestModuleInstaller_explicitPackageBoundary ( t * testing . T ) {
fixtureDir := filepath . Clean ( "testdata/load-module-package-prefix" )
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
// For this particular test we need an absolute path in the root module
// that must actually resolve to our temporary directory in "dir", so
// we need to do a little rewriting. We replace the arbitrary placeholder
// %%BASE%% with the temporary directory path.
{
rootFilename := filepath . Join ( dir , "package-prefix.tf" )
template , err := ioutil . ReadFile ( rootFilename )
if err != nil {
t . Fatal ( err )
}
final := bytes . ReplaceAll ( template , [ ] byte ( "%%BASE%%" ) , [ ] byte ( filepath . ToSlash ( dir ) ) )
err = ioutil . WriteFile ( rootFilename , final , 0644 )
if err != nil {
t . Fatal ( err )
}
}
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
_ , diags := inst . InstallModules ( "." , false , hooks )
if diags . HasErrors ( ) {
t . Fatalf ( "unexpected errors\n%s" , diags . Err ( ) . Error ( ) )
}
}
2019-05-17 22:19:31 +02:00
func TestModuleInstaller_invalid_version_constraint_error ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/invalid-version-constraint" )
2019-05-17 22:19:31 +02:00
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
2019-05-31 16:57:42 +02:00
_ , diags := inst . InstallModules ( "." , false , hooks )
if ! diags . HasErrors ( ) {
t . Fatal ( "expected error" )
} else {
assertDiagnosticSummary ( t , diags , "Invalid version constraint" )
}
}
func TestModuleInstaller_invalidVersionConstraintGetter ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/invalid-version-constraint" )
2019-05-31 16:57:42 +02:00
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
_ , diags := inst . InstallModules ( "." , false , hooks )
if ! diags . HasErrors ( ) {
t . Fatal ( "expected error" )
} else {
assertDiagnosticSummary ( t , diags , "Invalid version constraint" )
}
}
func TestModuleInstaller_invalidVersionConstraintLocal ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/invalid-version-constraint-local" )
2019-05-31 16:57:42 +02:00
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
2019-05-17 22:19:31 +02:00
_ , diags := inst . InstallModules ( "." , false , hooks )
if ! diags . HasErrors ( ) {
t . Fatal ( "expected error" )
} else {
assertDiagnosticSummary ( t , diags , "Invalid version constraint" )
}
}
2019-04-24 14:19:27 +02:00
func TestModuleInstaller_symlink ( t * testing . T ) {
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/local-module-symlink" )
2019-04-24 14:19:27 +02:00
dir , done := tempChdir ( t , fixtureDir )
defer done ( )
hooks := & testInstallHooks { }
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , nil )
_ , diags := inst . InstallModules ( "." , false , hooks )
assertNoDiagnostics ( t , diags )
wantCalls := [ ] testInstallHookCall {
{
Name : "Install" ,
ModuleAddr : "child_a" ,
PackageAddr : "" ,
LocalPath : "child_a" ,
} ,
{
Name : "Install" ,
ModuleAddr : "child_a.child_b" ,
PackageAddr : "" ,
LocalPath : "child_a/child_b" ,
} ,
}
if assertResultDeepEqual ( t , hooks . Calls , wantCalls ) {
return
}
loader , err := configload . NewLoader ( & configload . Config {
ModulesDir : modulesDir ,
} )
if err != nil {
t . Fatal ( err )
}
// Make sure the configuration is loadable now.
// (This ensures that correct information is recorded in the manifest.)
config , loadDiags := loader . LoadConfig ( "." )
assertNoDiagnostics ( t , tfdiags . Diagnostics { } . Append ( loadDiags ) )
wantTraces := map [ string ] string {
"" : "in root module" ,
"child_a" : "in child_a module" ,
"child_a.child_b" : "in child_b module" ,
}
gotTraces := map [ string ] string { }
config . DeepEach ( func ( c * configs . Config ) {
path := strings . Join ( c . Path , "." )
if c . Module . Variables [ "v" ] == nil {
gotTraces [ path ] = "<missing>"
return
}
varDesc := c . Module . Variables [ "v" ] . Description
gotTraces [ path ] = varDesc
} )
assertResultDeepEqual ( t , gotTraces , wantTraces )
}
2018-02-14 01:36:36 +01:00
func TestLoaderInstallModules_registry ( t * testing . T ) {
if os . Getenv ( "TF_ACC" ) == "" {
t . Skip ( "this test accesses registry.terraform.io and github.com; set TF_ACC=1 to run it" )
}
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/registry-modules" )
2019-09-25 22:27:17 +02:00
tmpDir , done := tempChdir ( t , fixtureDir )
2019-09-26 17:30:52 +02:00
// the module installer runs filepath.EvalSymlinks() on the destination
// directory before copying files, and the resultant directory is what is
// returned by the install hooks. Without this, tests could fail on machines
// where the default temp dir was a symlink.
2019-09-25 22:27:17 +02:00
dir , err := filepath . EvalSymlinks ( tmpDir )
if err != nil {
t . Error ( err )
}
2018-02-14 01:36:36 +01:00
defer done ( )
hooks := & testInstallHooks { }
2019-01-09 03:39:14 +01:00
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , registry . NewClient ( nil , nil ) )
command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.
This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.
In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.
Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.
The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
excludes Terraform versions prior to v0.12 then the configuration is
probably _already_ upgraded and so it's just a normal syntax error,
even if the early loader didn't detect it.
Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
_ , diags := inst . InstallModules ( dir , false , hooks )
2018-02-14 01:36:36 +01:00
assertNoDiagnostics ( t , diags )
v := version . Must ( version . NewVersion ( "0.0.1" ) )
wantCalls := [ ] testInstallHookCall {
// the configuration builder visits each level of calls in lexicographical
// order by name, so the following list is kept in the same order.
// acctest_child_a accesses //modules/child_a directly
{
Name : "Download" ,
ModuleAddr : "acctest_child_a" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "registry.terraform.io/hashicorp/module-installer-acctest/aws" , // intentionally excludes the subdir because we're downloading the whole package here
2018-02-14 01:36:36 +01:00
Version : v ,
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_child_a" ,
Version : v ,
2021-06-02 20:38:53 +02:00
// NOTE: This local path and the other paths derived from it below
// can vary depending on how the registry is implemented. At the
// time of writing this test, registry.terraform.io returns
// git repository source addresses and so this path refers to the
// root of the git clone, but historically the registry referred
// to GitHub-provided tar archives which meant that there was an
// extra level of subdirectory here for the typical directory
// nesting in tar archives, which would've been reflected as
// an extra segment on this path. If this test fails due to an
// additional path segment in future, then a change to the upstream
// registry might be the root cause.
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_a/modules/child_a" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_child_a.child_b
// (no download because it's a relative path inside acctest_child_a)
{
Name : "Install" ,
ModuleAddr : "acctest_child_a.child_b" ,
2021-06-02 20:38:53 +02:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_a/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_child_b accesses //modules/child_b directly
{
Name : "Download" ,
ModuleAddr : "acctest_child_b" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "registry.terraform.io/hashicorp/module-installer-acctest/aws" , // intentionally excludes the subdir because we're downloading the whole package here
2018-02-14 01:36:36 +01:00
Version : v ,
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_child_b" ,
Version : v ,
2021-06-02 20:38:53 +02:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_b/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root
{
Name : "Download" ,
ModuleAddr : "acctest_root" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "registry.terraform.io/hashicorp/module-installer-acctest/aws" ,
2018-02-14 01:36:36 +01:00
Version : v ,
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_root" ,
Version : v ,
2021-06-02 20:38:53 +02:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root.child_a
// (no download because it's a relative path inside acctest_root)
{
Name : "Install" ,
ModuleAddr : "acctest_root.child_a" ,
2021-06-02 20:38:53 +02:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root/modules/child_a" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root.child_a.child_b
// (no download because it's a relative path inside acctest_root, via acctest_root.child_a)
{
Name : "Install" ,
ModuleAddr : "acctest_root.child_a.child_b" ,
2021-06-02 20:38:53 +02:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
}
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
if diff := cmp . Diff ( wantCalls , hooks . Calls ) ; diff != "" {
t . Fatalf ( "wrong installer calls\n%s" , diff )
2018-02-14 01:36:36 +01:00
}
2020-01-07 21:03:23 +01:00
//check that the registry reponses were cached
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
if _ , ok := inst . moduleVersions [ "registry.terraform.io/hashicorp/module-installer-acctest/aws" ] ; ! ok {
t . Errorf ( "module versions cache was not populated\ngot: %s\nwant: key hashicorp/module-installer-acctest/aws" , spew . Sdump ( inst . moduleVersions ) )
2020-01-07 21:03:23 +01:00
}
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
if _ , ok := inst . moduleVersionsUrl [ moduleVersion { module : "registry.terraform.io/hashicorp/module-installer-acctest/aws" , version : "0.0.1" } ] ; ! ok {
t . Errorf ( "module download url cache was not populated\ngot: %s" , spew . Sdump ( inst . moduleVersionsUrl ) )
2020-01-07 21:03:23 +01:00
}
2019-01-09 03:39:14 +01:00
loader , err := configload . NewLoader ( & configload . Config {
ModulesDir : modulesDir ,
} )
if err != nil {
t . Fatal ( err )
}
2018-02-14 01:36:36 +01:00
// Make sure the configuration is loadable now.
// (This ensures that correct information is recorded in the manifest.)
2018-02-15 18:23:07 +01:00
config , loadDiags := loader . LoadConfig ( "." )
2019-01-09 03:39:14 +01:00
assertNoDiagnostics ( t , tfdiags . Diagnostics { } . Append ( loadDiags ) )
2018-02-15 18:23:07 +01:00
wantTraces := map [ string ] string {
"" : "in local caller for registry-modules" ,
"acctest_root" : "in root module" ,
"acctest_root.child_a" : "in child_a module" ,
"acctest_root.child_a.child_b" : "in child_b module" ,
"acctest_child_a" : "in child_a module" ,
"acctest_child_a.child_b" : "in child_b module" ,
"acctest_child_b" : "in child_b module" ,
}
gotTraces := map [ string ] string { }
config . DeepEach ( func ( c * configs . Config ) {
path := strings . Join ( c . Path , "." )
if c . Module . Variables [ "v" ] == nil {
gotTraces [ path ] = "<missing>"
return
}
varDesc := c . Module . Variables [ "v" ] . Description
gotTraces [ path ] = varDesc
} )
assertResultDeepEqual ( t , gotTraces , wantTraces )
2018-02-14 01:36:36 +01:00
}
func TestLoaderInstallModules_goGetter ( t * testing . T ) {
if os . Getenv ( "TF_ACC" ) == "" {
t . Skip ( "this test accesses github.com; set TF_ACC=1 to run it" )
}
2019-06-30 09:38:36 +02:00
fixtureDir := filepath . Clean ( "testdata/go-getter-modules" )
2019-09-25 22:27:17 +02:00
tmpDir , done := tempChdir ( t , fixtureDir )
2019-09-26 17:30:52 +02:00
// the module installer runs filepath.EvalSymlinks() on the destination
// directory before copying files, and the resultant directory is what is
// returned by the install hooks. Without this, tests could fail on machines
// where the default temp dir was a symlink.
2019-09-25 22:27:17 +02:00
dir , err := filepath . EvalSymlinks ( tmpDir )
if err != nil {
t . Error ( err )
}
2018-02-14 01:36:36 +01:00
defer done ( )
hooks := & testInstallHooks { }
2019-01-09 03:39:14 +01:00
modulesDir := filepath . Join ( dir , ".terraform/modules" )
inst := NewModuleInstaller ( modulesDir , registry . NewClient ( nil , nil ) )
command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.
This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.
In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.
Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.
The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
excludes Terraform versions prior to v0.12 then the configuration is
probably _already_ upgraded and so it's just a normal syntax error,
even if the early loader didn't detect it.
Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
_ , diags := inst . InstallModules ( dir , false , hooks )
2018-02-14 01:36:36 +01:00
assertNoDiagnostics ( t , diags )
wantCalls := [ ] testInstallHookCall {
// the configuration builder visits each level of calls in lexicographical
// order by name, so the following list is kept in the same order.
// acctest_child_a accesses //modules/child_a directly
{
Name : "Download" ,
ModuleAddr : "acctest_child_a" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "git::https://github.com/hashicorp/terraform-aws-module-installer-acctest.git?ref=v0.0.1" , // intentionally excludes the subdir because we're downloading the whole repo here
2018-02-14 01:36:36 +01:00
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_child_a" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_a/modules/child_a" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_child_a.child_b
// (no download because it's a relative path inside acctest_child_a)
{
Name : "Install" ,
ModuleAddr : "acctest_child_a.child_b" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_a/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_child_b accesses //modules/child_b directly
{
Name : "Download" ,
ModuleAddr : "acctest_child_b" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "git::https://github.com/hashicorp/terraform-aws-module-installer-acctest.git?ref=v0.0.1" , // intentionally excludes the subdir because we're downloading the whole package here
2018-02-14 01:36:36 +01:00
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_child_b" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_child_b/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root
{
Name : "Download" ,
ModuleAddr : "acctest_root" ,
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
PackageAddr : "git::https://github.com/hashicorp/terraform-aws-module-installer-acctest.git?ref=v0.0.1" ,
2018-02-14 01:36:36 +01:00
} ,
{
Name : "Install" ,
ModuleAddr : "acctest_root" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root.child_a
// (no download because it's a relative path inside acctest_root)
{
Name : "Install" ,
ModuleAddr : "acctest_root.child_a" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root/modules/child_a" ) ,
2018-02-14 01:36:36 +01:00
} ,
// acctest_root.child_a.child_b
// (no download because it's a relative path inside acctest_root, via acctest_root.child_a)
{
Name : "Install" ,
ModuleAddr : "acctest_root.child_a.child_b" ,
2019-03-11 23:25:21 +01:00
LocalPath : filepath . Join ( dir , ".terraform/modules/acctest_root/modules/child_b" ) ,
2018-02-14 01:36:36 +01:00
} ,
}
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
if diff := cmp . Diff ( wantCalls , hooks . Calls ) ; diff != "" {
t . Fatalf ( "wrong installer calls\n%s" , diff )
2018-02-14 01:36:36 +01:00
}
2019-01-09 03:39:14 +01:00
loader , err := configload . NewLoader ( & configload . Config {
ModulesDir : modulesDir ,
} )
if err != nil {
t . Fatal ( err )
}
2018-02-14 01:36:36 +01:00
// Make sure the configuration is loadable now.
// (This ensures that correct information is recorded in the manifest.)
2018-02-15 18:23:07 +01:00
config , loadDiags := loader . LoadConfig ( "." )
2019-01-09 03:39:14 +01:00
assertNoDiagnostics ( t , tfdiags . Diagnostics { } . Append ( loadDiags ) )
2018-02-15 18:23:07 +01:00
wantTraces := map [ string ] string {
"" : "in local caller for go-getter-modules" ,
"acctest_root" : "in root module" ,
"acctest_root.child_a" : "in child_a module" ,
"acctest_root.child_a.child_b" : "in child_b module" ,
"acctest_child_a" : "in child_a module" ,
"acctest_child_a.child_b" : "in child_b module" ,
"acctest_child_b" : "in child_b module" ,
}
gotTraces := map [ string ] string { }
config . DeepEach ( func ( c * configs . Config ) {
path := strings . Join ( c . Path , "." )
if c . Module . Variables [ "v" ] == nil {
gotTraces [ path ] = "<missing>"
return
}
varDesc := c . Module . Variables [ "v" ] . Description
gotTraces [ path ] = varDesc
} )
assertResultDeepEqual ( t , gotTraces , wantTraces )
2018-02-13 23:40:53 +01:00
}
type testInstallHooks struct {
Calls [ ] testInstallHookCall
}
type testInstallHookCall struct {
Name string
ModuleAddr string
PackageAddr string
Version * version . Version
LocalPath string
}
func ( h * testInstallHooks ) Download ( moduleAddr , packageAddr string , version * version . Version ) {
h . Calls = append ( h . Calls , testInstallHookCall {
Name : "Download" ,
ModuleAddr : moduleAddr ,
PackageAddr : packageAddr ,
Version : version ,
} )
}
func ( h * testInstallHooks ) Install ( moduleAddr string , version * version . Version , localPath string ) {
h . Calls = append ( h . Calls , testInstallHookCall {
Name : "Install" ,
ModuleAddr : moduleAddr ,
Version : version ,
LocalPath : localPath ,
} )
}
2019-01-09 03:39:14 +01:00
// tempChdir copies the contents of the given directory to a temporary
// directory and changes the test process's current working directory to
// point to that directory. Also returned is a function that should be
// called at the end of the test (e.g. via "defer") to restore the previous
// working directory.
//
// Tests using this helper cannot safely be run in parallel with other tests.
func tempChdir ( t * testing . T , sourceDir string ) ( string , func ( ) ) {
t . Helper ( )
tmpDir , err := ioutil . TempDir ( "" , "terraform-configload" )
if err != nil {
t . Fatalf ( "failed to create temporary directory: %s" , err )
return "" , nil
}
2020-10-07 18:48:25 +02:00
if err := copy . CopyDir ( tmpDir , sourceDir ) ; err != nil {
2019-01-09 03:39:14 +01:00
t . Fatalf ( "failed to copy fixture to temporary directory: %s" , err )
return "" , nil
}
oldDir , err := os . Getwd ( )
if err != nil {
t . Fatalf ( "failed to determine current working directory: %s" , err )
return "" , nil
}
err = os . Chdir ( tmpDir )
if err != nil {
t . Fatalf ( "failed to switch to temp dir %s: %s" , tmpDir , err )
return "" , nil
}
// Most of the tests need this, so we'll make it just in case.
os . MkdirAll ( filepath . Join ( tmpDir , ".terraform/modules" ) , os . ModePerm )
t . Logf ( "tempChdir switched to %s after copying from %s" , tmpDir , sourceDir )
return tmpDir , func ( ) {
err := os . Chdir ( oldDir )
if err != nil {
panic ( fmt . Errorf ( "failed to restore previous working directory %s: %s" , oldDir , err ) )
}
if os . Getenv ( "TF_CONFIGLOAD_TEST_KEEP_TMP" ) == "" {
os . RemoveAll ( tmpDir )
}
}
}
func assertNoDiagnostics ( t * testing . T , diags tfdiags . Diagnostics ) bool {
t . Helper ( )
return assertDiagnosticCount ( t , diags , 0 )
}
func assertDiagnosticCount ( t * testing . T , diags tfdiags . Diagnostics , want int ) bool {
t . Helper ( )
if len ( diags ) != 0 {
t . Errorf ( "wrong number of diagnostics %d; want %d" , len ( diags ) , want )
for _ , diag := range diags {
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
t . Logf ( "- %#v" , diag )
2019-01-09 03:39:14 +01:00
}
return true
}
return false
}
func assertDiagnosticSummary ( t * testing . T , diags tfdiags . Diagnostics , want string ) bool {
t . Helper ( )
for _ , diag := range diags {
if diag . Description ( ) . Summary == want {
return false
}
}
t . Errorf ( "missing diagnostic summary %q" , want )
for _ , diag := range diags {
Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
t . Logf ( "- %#v" , diag )
2019-01-09 03:39:14 +01:00
}
return true
}
func assertResultDeepEqual ( t * testing . T , got , want interface { } ) bool {
t . Helper ( )
if diff := deep . Equal ( got , want ) ; diff != nil {
for _ , problem := range diff {
t . Errorf ( "%s" , problem )
}
return true
}
return false
}
