ResiLien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
terraform/configs/module_merge_test.go

309 lines
8.1 KiB
Go
Raw Normal View History

configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
package configs
import (
"testing"
configs: extend module.ProviderRequirements to include the addrs.Provider instead of just version constraints. (#23843) Renamed file.ProviderRequirements to file.RequiredProviders to match the name of the block in the configuration. file.RequiredProviders contains the contents of the file(s); module.ProviderRequirements contains the parsed and merged provider requirements. Extended decodeRequiredProvidersBlock to parse the new provider source syntax (version only, it will ignore any other attributes). Added some tests; swapped deep.Equal with cmp.Equal in the terraform/module_dependencies_test.go because deep was not catching incorrect constraints.
2020-01-13 17:31:47 +01:00
version "github.com/hashicorp/go-version"
vendor: switch to HCL 2.0 in the HCL repository Previously we were using the experimental HCL 2 repository, but now we'll shift over to the v2 import path within the main HCL repository as part of actually releasing HCL 2.0 as stable. This is a mechanical search/replace to the new import paths. It also switches to the v2.0.0 release of HCL, which includes some new code that Terraform didn't previously have but should not change any behavior that matters for Terraform's purposes. For the moment the experimental HCL2 repository is still an indirect dependency via terraform-config-inspect, so it remains in our go.sum and vendor directories for the moment. Because terraform-config-inspect uses a much smaller subset of the HCL2 functionality, this does still manage to prune the vendor directory a little. A subsequent release of terraform-config-inspect should allow us to completely remove that old repository in a future commit.
2019-09-10 00:58:44 +02:00
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
configs: extend module.ProviderRequirements to include the addrs.Provider instead of just version constraints. (#23843) Renamed file.ProviderRequirements to file.RequiredProviders to match the name of the block in the configuration. file.RequiredProviders contains the contents of the file(s); module.ProviderRequirements contains the parsed and merged provider requirements. Extended decodeRequiredProvidersBlock to parse the new provider source syntax (version only, it will ignore any other attributes). Added some tests; swapped deep.Equal with cmp.Equal in the terraform/module_dependencies_test.go because deep was not catching incorrect constraints.
2020-01-13 17:31:47 +01:00
"github.com/hashicorp/terraform/addrs"
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
"github.com/zclconf/go-cty/cty"
)
func TestModuleOverrideVariable(t *testing.T) {
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
mod, diags := testModuleFromDir("testdata/valid-modules/override-variable")
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
assertNoDiagnostics(t, diags)
if mod == nil {
t.Fatalf("module is nil")
}
got := mod.Variables
want := map[string]*Variable{
"fully_overridden": {
Name: "fully_overridden",
Description: "b_override description",
DescriptionSet: true,
Default: cty.StringVal("b_override"),
configs: allow full type constraints for variables Previously we just ported over the simple "string", "list", and "map" type hint keywords from the old loader, which exist primarily as hints to the CLI for whether to treat -var=... arguments and environment variables as literal strings or as HCL expressions. However, we've been requested before to allow more specific constraints here because it's generally better UX for a type error to be detected within an expression in a calling "module" block rather than at some point deep inside a third-party module. To allow for more specific constraints, here we use the type constraint expression syntax defined as an extension within HCL, which uses the variable and function call syntaxes to represent types rather than values, like this: - string - number - bool - list(string) - list(any) - list(map(string)) - object({id=string,name=string}) In native HCL syntax this looks like: variable "foo" { type = map(string) } In JSON, this looks like: { "variable": { "foo": { "type": "map(string)" } } } The selection of literal processing or HCL parsing of CLI-set values is now explicit in the model and separate from the type, though it's still derived from the type constraint and thus not directly controllable in configuration. Since this syntax is more complex than the keywords that replaced it, for now the simpler keywords are still supported and "list" and "map" are interpreted as list(any) and map(any) respectively, mimicking how they were interpreted by Terraform 0.11 and earlier. For the time being our documentation should continue to recommend these shorthand versions until we gain more experience with the more-specific type constraints; most users should just make use of the additional primitive type constraints this enables: bool and number. As a result of these more-complete type constraints, we can now type-check the default value at config load time, which has the nice side-effect of allowing us to produce a tailored error message if an override file produces an invalid situation; previously the result was rather confusing because the error message referred to the original definition of the variable and not the overridden parts.
2018-03-07 02:37:51 +01:00
Type: cty.String,
ParsingMode: VariableParseLiteral,
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
DeclRange: hcl.Range{
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
Filename: "testdata/valid-modules/override-variable/primary.tf",
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
Start: hcl.Pos{
Line: 1,
Column: 1,
Byte: 0,
},
End: hcl.Pos{
Line: 1,
Column: 28,
Byte: 27,
},
},
},
"partially_overridden": {
Name: "partially_overridden",
Description: "base description",
DescriptionSet: true,
Default: cty.StringVal("b_override partial"),
configs: allow full type constraints for variables Previously we just ported over the simple "string", "list", and "map" type hint keywords from the old loader, which exist primarily as hints to the CLI for whether to treat -var=... arguments and environment variables as literal strings or as HCL expressions. However, we've been requested before to allow more specific constraints here because it's generally better UX for a type error to be detected within an expression in a calling "module" block rather than at some point deep inside a third-party module. To allow for more specific constraints, here we use the type constraint expression syntax defined as an extension within HCL, which uses the variable and function call syntaxes to represent types rather than values, like this: - string - number - bool - list(string) - list(any) - list(map(string)) - object({id=string,name=string}) In native HCL syntax this looks like: variable "foo" { type = map(string) } In JSON, this looks like: { "variable": { "foo": { "type": "map(string)" } } } The selection of literal processing or HCL parsing of CLI-set values is now explicit in the model and separate from the type, though it's still derived from the type constraint and thus not directly controllable in configuration. Since this syntax is more complex than the keywords that replaced it, for now the simpler keywords are still supported and "list" and "map" are interpreted as list(any) and map(any) respectively, mimicking how they were interpreted by Terraform 0.11 and earlier. For the time being our documentation should continue to recommend these shorthand versions until we gain more experience with the more-specific type constraints; most users should just make use of the additional primitive type constraints this enables: bool and number. As a result of these more-complete type constraints, we can now type-check the default value at config load time, which has the nice side-effect of allowing us to produce a tailored error message if an override file produces an invalid situation; previously the result was rather confusing because the error message referred to the original definition of the variable and not the overridden parts.
2018-03-07 02:37:51 +01:00
Type: cty.String,
ParsingMode: VariableParseLiteral,
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
DeclRange: hcl.Range{
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
Filename: "testdata/valid-modules/override-variable/primary.tf",
configs: allow overrides files to omit args that primary files can't Some of the fields in our config structs are either mandatory in primary files or there is a default value that we apply if absent. Unfortunately override files impose the additional constraint that we be allowed to omit required fields (which have presumably already been set in the primary files) and that we are able to distinguish between a default value and omitting a value entirely. Since most of our fields were already acceptable for override files, here we just add some new fields to deal with the few cases where special handling is required and a helper function to disable the "Required" flag on attributes in a given schema.
2018-02-07 03:05:14 +01:00
Start: hcl.Pos{
Line: 7,
Column: 1,
Byte: 103,
},
End: hcl.Pos{
Line: 7,
Column: 32,
Byte: 134,
},
},
},
}
assertResultDeepEqual(t, got, want)
}
configs: Implementation of mergeBody mergeBody is a hcl.Body implementation that deals with our override file merging behavior for the portions of the configuration that are not processed until full eval time. Mimicking the behavior of our old config merge implementation from the "config" package, the rules here are: - Attributes in the override body hide attributes of the same name in the base body. - Any block in the override body hides all blocks with the same type name that appear in the base body. This is tested by a new test for the overriding of module arguments, which asserts the correct behavior of the merged body as part of its work.
2018-02-07 03:30:12 +01:00
func TestModuleOverrideModule(t *testing.T) {
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
mod, diags := testModuleFromDir("testdata/valid-modules/override-module")
configs: Implementation of mergeBody mergeBody is a hcl.Body implementation that deals with our override file merging behavior for the portions of the configuration that are not processed until full eval time. Mimicking the behavior of our old config merge implementation from the "config" package, the rules here are: - Attributes in the override body hide attributes of the same name in the base body. - Any block in the override body hides all blocks with the same type name that appear in the base body. This is tested by a new test for the overriding of module arguments, which asserts the correct behavior of the merged body as part of its work.
2018-02-07 03:30:12 +01:00
assertNoDiagnostics(t, diags)
if mod == nil {
t.Fatalf("module is nil")
}
if _, exists := mod.ModuleCalls["example"]; !exists {
t.Fatalf("no module 'example'")
}
if len(mod.ModuleCalls) != 1 {
t.Fatalf("wrong number of module calls in result %d; want 1", len(mod.ModuleCalls))
}
got := mod.ModuleCalls["example"]
want := &ModuleCall{
Name: "example",
SourceAddr: "./example2-a_override",
SourceAddrRange: hcl.Range{
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
Filename: "testdata/valid-modules/override-module/a_override.tf",
configs: Implementation of mergeBody mergeBody is a hcl.Body implementation that deals with our override file merging behavior for the portions of the configuration that are not processed until full eval time. Mimicking the behavior of our old config merge implementation from the "config" package, the rules here are: - Attributes in the override body hide attributes of the same name in the base body. - Any block in the override body hides all blocks with the same type name that appear in the base body. This is tested by a new test for the overriding of module arguments, which asserts the correct behavior of the merged body as part of its work.
2018-02-07 03:30:12 +01:00
Start: hcl.Pos{
Line: 3,
Column: 12,
Byte: 31,
},
End: hcl.Pos{
Line: 3,
Column: 35,
Byte: 54,
},
},
SourceSet: true,
DeclRange: hcl.Range{
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
Filename: "testdata/valid-modules/override-module/primary.tf",
configs: Implementation of mergeBody mergeBody is a hcl.Body implementation that deals with our override file merging behavior for the portions of the configuration that are not processed until full eval time. Mimicking the behavior of our old config merge implementation from the "config" package, the rules here are: - Attributes in the override body hide attributes of the same name in the base body. - Any block in the override body hides all blocks with the same type name that appear in the base body. This is tested by a new test for the overriding of module arguments, which asserts the correct behavior of the merged body as part of its work.
2018-02-07 03:30:12 +01:00
Start: hcl.Pos{
Line: 2,
Column: 1,
Byte: 1,
},
End: hcl.Pos{
Line: 2,
Column: 17,
Byte: 17,
},
},
}
// We're going to extract and nil out our hcl.Body here because DeepEqual
// is not a useful way to assert on that.
gotConfig := got.Config
got.Config = nil
assertResultDeepEqual(t, got, want)
type content struct {
Kept *string `hcl:"kept"`
Foo *string `hcl:"foo"`
New *string `hcl:"new"`
Newer *string `hcl:"newer"`
}
var gotArgs content
diags = gohcl.DecodeBody(gotConfig, nil, &gotArgs)
assertNoDiagnostics(t, diags)
wantArgs := content{
Kept: stringPtr("primary kept"),
Foo: stringPtr("a_override foo"),
New: stringPtr("b_override new"),
Newer: stringPtr("b_override newer"),
}
assertResultDeepEqual(t, gotArgs, wantArgs)
}
configs: Handle "dynamic" blocks as special during override merging Previously we were treating "dynamic" blocks in configuration the same as any other block type when merging config bodies, so that dynamic blocks in the override would override any dynamic blocks present in the base, without considering the dynamic block type. It's more useful and intuitive for us to treat dynamic blocks as if they are instances of their given block type for the purpose of overriding. That means a foo block can be overridden by a dynamic "foo" block and vice-versa, and dynamic blocks of different types do not interact at all during overriding. This requires us to recognize dynamic blocks and treat them specially during decoding of a merged body. We leave them unexpanded here because this package is not responsible for dynamic block expansion (that happens in the sibling "lang" package) but we do decode them enough to recognize their labels so we can treat them as if they were blocks of the labelled type.
2019-04-16 00:25:00 +02:00
func TestModuleOverrideDynamic(t *testing.T) {
schema := &hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{Type: "foo"},
{Type: "dynamic", LabelNames: []string{"type"}},
},
}
t.Run("base is dynamic", func(t *testing.T) {
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
mod, diags := testModuleFromDir("testdata/valid-modules/override-dynamic-block-base")
configs: Handle "dynamic" blocks as special during override merging Previously we were treating "dynamic" blocks in configuration the same as any other block type when merging config bodies, so that dynamic blocks in the override would override any dynamic blocks present in the base, without considering the dynamic block type. It's more useful and intuitive for us to treat dynamic blocks as if they are instances of their given block type for the purpose of overriding. That means a foo block can be overridden by a dynamic "foo" block and vice-versa, and dynamic blocks of different types do not interact at all during overriding. This requires us to recognize dynamic blocks and treat them specially during decoding of a merged body. We leave them unexpanded here because this package is not responsible for dynamic block expansion (that happens in the sibling "lang" package) but we do decode them enough to recognize their labels so we can treat them as if they were blocks of the labelled type.
2019-04-16 00:25:00 +02:00
assertNoDiagnostics(t, diags)
if mod == nil {
t.Fatalf("module is nil")
}
if _, exists := mod.ManagedResources["test.foo"]; !exists {
t.Fatalf("no module 'example'")
}
if len(mod.ManagedResources) != 1 {
t.Fatalf("wrong number of managed resources in result %d; want 1", len(mod.ManagedResources))
}
body := mod.ManagedResources["test.foo"].Config
content, diags := body.Content(schema)
assertNoDiagnostics(t, diags)
if len(content.Blocks) != 1 {
t.Fatalf("wrong number of blocks in result %d; want 1", len(content.Blocks))
}
if got, want := content.Blocks[0].Type, "foo"; got != want {
t.Fatalf("wrong block type %q; want %q", got, want)
}
})
t.Run("override is dynamic", func(t *testing.T) {
Standardise directory name for test data
2019-06-30 09:38:36 +02:00
mod, diags := testModuleFromDir("testdata/valid-modules/override-dynamic-block-override")
configs: Handle "dynamic" blocks as special during override merging Previously we were treating "dynamic" blocks in configuration the same as any other block type when merging config bodies, so that dynamic blocks in the override would override any dynamic blocks present in the base, without considering the dynamic block type. It's more useful and intuitive for us to treat dynamic blocks as if they are instances of their given block type for the purpose of overriding. That means a foo block can be overridden by a dynamic "foo" block and vice-versa, and dynamic blocks of different types do not interact at all during overriding. This requires us to recognize dynamic blocks and treat them specially during decoding of a merged body. We leave them unexpanded here because this package is not responsible for dynamic block expansion (that happens in the sibling "lang" package) but we do decode them enough to recognize their labels so we can treat them as if they were blocks of the labelled type.
2019-04-16 00:25:00 +02:00
assertNoDiagnostics(t, diags)
if mod == nil {
t.Fatalf("module is nil")
}
if _, exists := mod.ManagedResources["test.foo"]; !exists {
t.Fatalf("no module 'example'")
}
if len(mod.ManagedResources) != 1 {
t.Fatalf("wrong number of managed resources in result %d; want 1", len(mod.ManagedResources))
}
body := mod.ManagedResources["test.foo"].Config
content, diags := body.Content(schema)
assertNoDiagnostics(t, diags)
if len(content.Blocks) != 1 {
t.Fatalf("wrong number of blocks in result %d; want 1", len(content.Blocks))
}
if got, want := content.Blocks[0].Type, "dynamic"; got != want {
t.Fatalf("wrong block type %q; want %q", got, want)
}
if got, want := content.Blocks[0].Labels[0], "foo"; got != want {
t.Fatalf("wrong dynamic block label %q; want %q", got, want)
}
})
}
configs: extend module.ProviderRequirements to include the addrs.Provider instead of just version constraints. (#23843) Renamed file.ProviderRequirements to file.RequiredProviders to match the name of the block in the configuration. file.RequiredProviders contains the contents of the file(s); module.ProviderRequirements contains the parsed and merged provider requirements. Extended decodeRequiredProvidersBlock to parse the new provider source syntax (version only, it will ignore any other attributes). Added some tests; swapped deep.Equal with cmp.Equal in the terraform/module_dependencies_test.go because deep was not catching incorrect constraints.
2020-01-13 17:31:47 +01:00
configs: attach provider fqn to Resource (#24382) * configs: attach provider fqn to Resource
2020-03-16 19:36:16 +01:00
func TestModuleOverrideResourceFQNs(t *testing.T) {
mod, diags := testModuleFromDir("testdata/valid-modules/override-resource-provider")
assertNoDiagnostics(t, diags)
got := mod.ManagedResources["test_instance.explicit"]
wantProvider := addrs.NewProvider(addrs.DefaultRegistryHost, "bar", "test")
wantProviderCfg := &ProviderConfigRef{
Name: "bar-test",
NameRange: hcl.Range{
Filename: "testdata/valid-modules/override-resource-provider/a_override.tf",
Start: hcl.Pos{Line: 2, Column: 14, Byte: 51},
End: hcl.Pos{Line: 2, Column: 22, Byte: 59},
},
}
if !got.Provider.Equals(wantProvider) {
t.Fatalf("wrong provider %s, want %s", got.Provider, wantProvider)
}
assertResultDeepEqual(t, got.ProviderConfigRef, wantProviderCfg)
// now verify that a resource with no provider config falls back to default
got = mod.ManagedResources["test_instance.default"]
wantProvider = addrs.NewLegacyProvider("test")
if !got.Provider.Equals(wantProvider) {
t.Fatalf("wrong provider %s, want %s", got.Provider, wantProvider)
}
if got.ProviderConfigRef != nil {
t.Fatalf("wrong result: found provider config ref %s, expected nil", got.ProviderConfigRef)
}
}
configs: extend module.ProviderRequirements to include the addrs.Provider instead of just version constraints. (#23843) Renamed file.ProviderRequirements to file.RequiredProviders to match the name of the block in the configuration. file.RequiredProviders contains the contents of the file(s); module.ProviderRequirements contains the parsed and merged provider requirements. Extended decodeRequiredProvidersBlock to parse the new provider source syntax (version only, it will ignore any other attributes). Added some tests; swapped deep.Equal with cmp.Equal in the terraform/module_dependencies_test.go because deep was not catching incorrect constraints.
2020-01-13 17:31:47 +01:00
func TestMergeProviderVersionConstraints(t *testing.T) {
v1, _ := version.NewConstraint("1.0.0")
vc1 := VersionConstraint{
Required: v1,
}
v2, _ := version.NewConstraint("2.0.0")
vc2 := VersionConstraint{
Required: v2,
}
tests := map[string]struct {
Input map[string]ProviderRequirements
Override []*RequiredProvider
Want map[string]ProviderRequirements
}{
"basic merge": {
map[string]ProviderRequirements{
"random": ProviderRequirements{
Type: addrs.Provider{Type: "random"},
VersionConstraints: []VersionConstraint{},
},
},
[]*RequiredProvider{
&RequiredProvider{
Name: "null",
Requirement: VersionConstraint{},
},
},
map[string]ProviderRequirements{
"random": ProviderRequirements{
Type: addrs.Provider{Type: "random"},
VersionConstraints: []VersionConstraint{},
},
"null": ProviderRequirements{
Type: addrs.NewLegacyProvider("null"),
VersionConstraints: []VersionConstraint{
VersionConstraint{
Required: version.Constraints(nil),
DeclRange: hcl.Range{},
},
},
},
},
},
"override version constraint": {
map[string]ProviderRequirements{
"random": ProviderRequirements{
Type: addrs.Provider{Type: "random"},
VersionConstraints: []VersionConstraint{vc1},
},
},
[]*RequiredProvider{
&RequiredProvider{
Name: "random",
Requirement: vc2,
},
},
map[string]ProviderRequirements{
"random": ProviderRequirements{
Type: addrs.NewLegacyProvider("random"),
VersionConstraints: []VersionConstraint{vc2},
},
},
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
mergeProviderVersionConstraints(test.Input, test.Override)
assertResultDeepEqual(t, test.Input, test.Want)
})
}
}