DecoderSpec may be called many times, and deeply recursive calls are
expensive. Since we cannot synchronize the Blocks themselves due to them
being copied in parts of the code, we use a separate cache to store the
generated Specs.
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.
In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.
Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
This new-ish package ended up under "helper" during the 0.12 cycle for
want of some other place to put it, but in retrospect that was an odd
choice because the "helper/" tree is otherwise a bunch of legacy code from
when the SDK lived in this repository.
Here we move it over into the "internal" directory just to distance it
from the guidance of not using "helper/" packages in new projects;
didyoumean is a package we actively use as part of error message hints.
Previous deprecations only included direct assignment of template-only
expressions to arguments. That is, this was not deprecated:
locals {
foo = ["${var.foo}"]
}
This commit uses hclsyntax.VisitAll to detect and show deprecations for
all template-only expressions, no matter how deep they are in a given
expression.
The providers schema command is using the Config.ProviderTypes method,
which had not been kept up to date with the changes to provider
requirements detection made in Config.ProviderRequirements. This
resulted in any currently-unused providers being omitted from the
output.
This commit changes the ProviderTypes method to use the same underlying
logic as ProviderRequirements, which ensures that `required_providers`
blocks are taken into account.
Includes an integration test case to verify that this fixes the provider
schemas command bug.
A few users have recently been confused about the purpose of the
required_providers objects, adding provider configuration parameters in
addition to version and source. This previously did not cause an error
so would result in a confusingly distant failure.
This commit adds a single diagnostic for any required_providers object
which includes attributes other than version or source.
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
We've not been using HIL in the main codepaths since Terraform 0.12, but
some references to it (and some supporting functionality in Terraform)
stuck around due to interactions with types we'd kept around to support
legacy shims.
However, removing the configs.RawConfig field from
terraform.ResourceConfig disconnects that subtree of dependencies from
everything else, allowing us to remove it. This is safe because the only
remaining uses of terraform.ResourceConfig are shims from values that
were already evaluated using the HCL 2 API, and thus they never need
the "just in time" HIL evaluation that ResourceConfig.interpolateForce
used to do.
We also had some HIL references in configs/hcl2shim that were previously
in support of the "terraform 0.12upgrade" command, but the implementation
of that command is now removed.
There was one remaining reference to HIL in a now-unused function in the
helper/schema package, which I removed entirely here.
This then allows us to remove the HIL dependency entirely, and also to
clean up some remaining old remants of the legacy "config" package that
we'd recently moved into the "configs" package pending further pruning.
An invalid type name in a resource (or data source) could cause a panic
when determining the implied provider for the resource. This commit adds
verification that the type name is valid. It does not add a diagnostic,
since the invalid type name would have already been caught by the
parser.
Fixes#25560
The main motivation here is to produce a helpful error if a user
incorrectly uses the terraform-provider- prefix (which we see on provider
VCS repositories and plugin executables) as part of the source address.
However, this also more broadly blocks "terraform-" as a prefix in
anticipation of whatever instinct causes the phenomenon where e.g.
Python's PyPI has thousands of packages whose names start with "python-",
even though everything on PyPI is for Python by definition. This is
definitely not _necessary_, but it's better to be restrictive at first
and weaken later as needed.
If a resource's "provider" reference is invalid and cannot be parsed, we
should not store the reference as part of a `ProviderConfigRef`. Doing
so creates an invalid data structure, which prevents us from using
`MustParseProviderPart` with the name in later steps.
The invalid test files added in this commit will cause a panic without
the code change.
When parsing provider requirements we should check the type of the
source and version attributes rather than assuming that they are
strings. Otherwise an invalid attribute value will cause a panic.
In a recent PR, we changed the provider requirements code to permit
per-module requirements gathering, to enhance the provider command
output. This had an incorrect implementation of recursive requirements
gathering for the normal case, which resulted in only depth-1 modules
being inspected.
This commit fixes the broken recursion and adds a grandchild module to
the unit tests as test coverage. This also demanded fixing the
testNestedModuleConfigFromDir helper function to cope with nested
modules in test configs.
* Refactor provider validation into separate func & recurse
Refactors the validate provider functions into a separate function
that can recursively search above a module to check and see if
any parents of the module contain count/for_each configs to be
considered
The new provider installer code is using a new version constraint parser
because it produces better error messages than the one we were using
before. However, it has some cases where it returns errors that the old
parser (which was entirely regex-match-based) didn't catch.
In the long run we should consistently use the new parser everywhere, but
until then we'll avoid panicking then the two disagree, by returning
diagnostic messages instead of using MustParseVersionConstraints.
For now, we only hit these error cases if the user enters something that
the old parser allows but the new parser does not.
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.
This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.
For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.
This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
addProviderRequirements() was incorrectly using the map keys from the module
provider configs when looking up the provider FQN. The map keys include
alias, so this resulted in a panic. Update addProviderRequirements() to
use the provider's name (only) when looking up the FQN.
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.
Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.
Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString
ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.
This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.
* terraform: fix provider local names in tests
* configs: validate that all provider names are normalized
The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.
* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
Validate providers in expanding modules. Expanding modules cannot have provider configurations with non-empty configs, which includes having a version configured. If an empty or alias-only block is passed, the provider must be passed through the providers argument on the module call
This commit implements most of the intended functionality of the upgrade
command for rewriting configurations.
For a given module, it makes a list of all providers in use. Then it
attempts to detect the source address for providers without an explicit
source.
Once this step is complete, the tool rewrites the relevant configuration
files. This results in a single "required_providers" block for the
module, with a source for each provider.
Any providers for which the source cannot be detected (for example,
unofficial providers) will need a source to be defined by the user. The
tool writes an explanatory comment to the configuration to help with
this.
Previously, resources without explicit provider configuration (i.e. a
`provider =` attribute) would be assigned a default provider based upon
the resource type. For example, a resource `foo_bar` would be assigned
provider `hashicorp/foo`.
This behaviour did not work well with community or partner providers,
with sources configured in `terraform.required_providers` blocks. With
the following configuration:
terraform {
required_providers {
foo = {
source = "acme/foo"
}
}
}
resource foo_bar "a" { }
the resource would be configured with the `hashicorp/foo` provider.
This commit fixes this implied provider behaviour. First we look for a
provider with local name matching the resource type in the module's
required providers map. If one is found, this provider is assigned to
the resource. Otherwise, we still fall back to a default provider.
We now permit at most one `required_providers` block per module (except
for overrides). This prevents users (and Terraform) from struggling to
understand how to merge multiple `required_providers` configurations,
with `version` and `source` attributes split across multiple blocks.
Because only one `required_providers` block is permitted, there is no
need to concatenate version constraints and resolve them. This allows us
to simplify the structs used to represent provider requirements,
aligning more closely with other structs in this package.
This commit also fixes a semantic use-before-initialize bug, where
resources defined before a `required_providers` block would be unable to
use its source attribute. We achieve this by processing the module's
`required_providers` configuration (and overrides) before resources.
Overrides for `required_providers` work as before, replacing the entire
block per provider.
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.
Additional tests to exercise providers in modules and providers from
state have been included.
This PR adds iteration through any provider configuration blocks in the
config in addProviderRequirements().
A stale comment (of mine!) would leave one expecting the
module.ProviderRequirements to include any requirements from provider
configs. The comment was inaccurate and has been updated.
James Bardin