Module detection currently requires calling the registry to determine
the subdirectory. Since we're not directly accessing the subdirectory
through FolderStorage, and now handling it within terraform so modules can
reference sibling paths, we need to call out to the registry every
time we load a configuration to verify the subdirectory for the module,
which is returned during the Detect.
Record the subdirectories for each module in the top-level of the
FolderStorage path for retrieval during Tree.Load. This lets us bypass
Detection altogether, modules can be loaded without redetecting.
In order to remain backward compatible with some modules, we need to
handle subdirs during Load. This means duplicating part of the go-getter
code path for subDir handling so we can resolve any subDirs and globs
internally, while keeping the entire remote directory structure within
the file storage.
Test that we can get a subdirectory from a tarball (or any other
"packed" source that we support).
The 'tar-subdir-to-parent' test highlights a regression where the
subdirectory module references a module in its parent directory. This
breaks the intended use ofr the subdirectory and the implementation in
go-getter. We need to fix this in terraform, and possible plan warnings
and deprecations for this type of source.
The getter.FileDetector was intended to be the final detector, only
converting a path to a file URL and returning a true in all cases. We
want to check for a local module before checking the registry so no
local modules that happen to match a registry module are broken.
Wrap the getter.FileDetector to check the module source's existence
before delegating the search to the registry.
updating the key will cause the FolderStorage hash to change forcing
modules to be re-fetched. This is required because any configurations
using the subDir notation will have the configuration in the wrong
directory.
Terraform was redundantly handling `//dir` notation which should be
handled by go-getter. Rather than allowing go-getter to unpack a subdir
as expected, the subdir was stripped off and accessed through the module
configuration.
This scheme will no longer works now that go-getter supports `*`
subdirectories
(e.g. `//*` would be analogous to `tar --strip-components=1`).
Even though this allows Terraform to use go-getter's native unpacking,
detection is still done separately because Detect requires a `pwd` which
is dependent on the configuration directory and not known to the
global FolderStorage.
Add a getter.Detector for detecting registry modules and looking up
the download location of the latest version. This is essentially a
temporary API until constraint solving is supported by the registry, as
then we'll have to supply the full set of known contraints to the
registry at once for resolution and we will fetch specific versions of
modules.
Fixes#12788
We would panic when referencing an output from an undefined module. The
panic above this is correct but in this case Load will not catch
interpolated variables that _reference_ an unloaded/undefined module.
Test included.
It can be tedious fixing a new module with many errors when Terraform
only outputs the first random error it encounters.
Accumulate all errors from validation, and format them for the user.
Fixes#11038
This is a **short term fix**.
Terraform core doesn't currently handle root modules named "root" well
because the prefix `[]string{"root"}` has special meaning and Terraform
core [currently] can't disambiguate between the root module and a module
named "root" in the root module.
This PR introduces a short term fix by simply disallowing root modules
named "root". This shouldn't break any BC because since 0.8.0 this
didn't work at all in many broken ways (including crashes).
Longer term, this should be fixed by removing the special prefix at all
and having empty paths be root. I started down this path but the core
changes necessary are far too scary for a patch release. We can aim for
0.9.
Fixes#4789
This improves the validation that valid provider aliases are used.
Previously, we required that provider aliases be defined in every module
they're used. This isn't correct because the alias may be used in a
parent module and inherited.
This removes that validation and creates the validation that a provider
alias must be defined in the used module or _any parent_. This allows
inheritance to work properly.
We've always had this type of validation for aliases because we believe
its a good UX tradeoff: typo-ing an alias is really painful, so we
require declaration of alias usage. It may add a small burden to
declare, but since relatively few aliases are used, it improves the
scenario where a user fat-fingers an alias name.
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.
This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
Fixes the following error when cross compiling:
```
--> freebsd/amd64 error: exit status 2
Stderr: # github.com/hashicorp/terraform/config/module
config/module/inode.go:18: cannot use st.Ino (type uint32) as type uint64 in return argument
```
When copying a config module, make sure the full path for src and dst
files don't match, and also check the inode in case we resolved a
different path to the same file.
Make a note about the unsafe usage of reusing a tempDir path.
HgGetter tests failed on windows/amd64 using Mercurial version 3.2.4:
--- FAIL: TestHgGetter (0.11s)
get_hg_test.go:35: err: C:\Program Files\Mercurial\hg.exe exited with 255: abort: file:// URLs can only refer to localhost
--- FAIL: TestHgGetter_branch (0.11s)
get_hg_test.go:62: err: C:\Program Files\Mercurial\hg.exe exited with 255: abort: file:// URLs can only refer to localhost
FAIL
FAIL github.com/hashicorp/terraform/config/module 5.615s
This commit fixes the failures by adjusting the file:// URL to a form that
Mercurial expects.
Only adjust the URL Scheme when parsing drive letter file paths on
Windows, don't add a file scheme prefix.
FileDetector is responsible for adding the file scheme prefix.
Absolute file paths were not correctly detected by module.Detect
when using url.Parse to parse the source URL. Wrap the detection
with urlParse to properly handle file path detections on Windows.
Fixes command test failures on Windows.
When parsing URLs on Windows, assume it is a drive letter path
if the second element is a ':' character. Format the drive letter
path as a "file:///"-path prior to parsing the URL.
Fixes test failures of the following form in command on Windows:
=== RUN TestApply_plan
--- FAIL: TestApply_plan (0.00s)
apply_test.go:379: bad: 1
module download not supported for scheme 'c'
Using url.Parse to parse an absolute file path on Windows yields
a URL type where the Path element is prefixed by a slash.
For example, parsing "file:///C:/Users/user" gives a URL type
with Path:"/C:/Users/user".
According to golang.org/issue/6027, the parsing is correct as is.
The leading slash on the Path must be eliminated before any file
operations.
This commit introduces a urlParse function which wraps the url.Parse
functionality and removes the leading slash in Path for absolute file
paths on Windows.
Fixes config/module test failures on Windows.
Specify laddr on the form host:port in the call to net.Listen as
documented for net.Dial, see godoc.org/net#Dial
Fixes the following test failures on Windows:
> go test -run=TestHttpGetter
--- FAIL: TestHttpGetter_header (0.00s)
get_http_test.go:31: err: Get http://[::]:52101/header?terraform-get=1: dial tcp [::]:52101: ConnectEx tcp: The requested address is not valid in its context.
--- FAIL: TestHttpGetter_meta (0.00s)
get_http_test.go:55: err: Get http://[::]:52103/meta?terraform-get=1: dial tcp [::]:52103: ConnectEx tcp: The requested address is not valid in its context.
--- FAIL: TestHttpGetter_metaSubdir (0.00s)
get_http_test.go:79: err: Get http://[::]:52105/meta-subdir?terraform-get=1: dial tcp [::]:52105: ConnectEx tcp: The requested address is not valid in its context.
FAIL
exit status 1
FAIL github.com/hashicorp/terraform/config/module 0.054s
"/foo" is not an absolute path on Windows. Adjust the FileDetector
tests to take that into account when verifying the results.
Fixes FileDetector test failures on Windows.
builtin/providers/aws/tags_test.go:56: unrecognized printf verb 'i'
builtin/providers/aws/tags_test.go:59: unrecognized printf verb 'i'
config/config_test.go:101: possible formatting directive in Fatal call
config/config_test.go:157: possible formatting directive in Fatal call
config/module/get_file_test.go:91: missing argument for Fatalf(%s): format reads arg 1, have only 0 args
helper/schema/schema.go:341: arg v.Type for printf verb %s of wrong type: schema.ValueType
helper/schema/schema.go:656: missing argument for Errorf(%s): format reads arg 2, have only 1 args
helper/schema/schema.go:912: arg schema.Type for printf verb %s of wrong type: schema.ValueType
terraform/context.go:178: arg v.Type() for printf verb %s of wrong type: github.com/hashicorp/terraform/config.VariableType
terraform/context.go:486: arg c.Operation for printf verb %s of wrong type: terraform.walkOperation
terraform/diff_test.go💯 arg actual for printf verb %s of wrong type: terraform.DiffChangeType
terraform/diff_test.go:235: arg actual for printf verb %s of wrong type: terraform.DiffChangeType
James Bardin