This probably isn't the best UI we could do here, but it's a placeholder
for now just to avoid making it seem like we're ignoring the lock file
and checking for new versions anyway.
This changes the approach used by the provider installer to remember
between runs which selections it has previously made, using the lock file
format implemented in internal/depsfile.
This means that version constraints in the configuration are considered
only for providers we've not seen before or when -upgrade mode is active.
* remove unused code
I've removed the provider-specific code under registry, and unused nil
backend, and replaced a call to helper from backend/oss (the other
callers of that func are provisioners scheduled to be deprecated).
I also removed the Dockerfile, as our build process uses a different
file.
Finally I removed the examples directory, which had outdated examples
and links. There are better, actively maintained examples available.
* command: remove various unused bits
* test wasn't running
* backend: remove unused err
We no longer need to support 0.12-and-earlier-style provider addresses
because users should've upgraded their existing configurations and states
on Terraform 0.13 already.
For now this is only checked in the "init" command, because various test
shims are still relying on the idea of legacy providers the core layer.
However, rejecting these during init is sufficient grounds to avoid
supporting legacy provider addresses in the new dependency lock file
format, and thus sets the stage for a more severe removal of legacy
provider support in a later commit.
The tmp directory path is longer on Macs than other systems and was wrapping
across lines when printed, breaking the string match in the test.
Fix suggested by @apparentlymart is to add two spaces before the leading
'-' when printing to prevent the diagnostic renderer wrapping the line.
As we add and remove support for different target platforms over time,
there will be transition periods where the available platforms for
each provider will be different than the available platforms for Terraform
CLI itself.
In recognition of that possibility, here we add a more specialized error
message for that situation which tries to explain the problem a little
more clearly than the generic error message that came before it.
In an ideal world we'd be able to detect situations where a newer or older
version has support in a similar vein to what we do with provider protocol
incompatibilities, but we don't currently have logic to fetch the data
necessary to implement that, so this is instead a presentation-only change
aimed at allowing some possible near-future changes to the supported
target platforms.
In earlier commits we started to make the installation codepath
context-aware so that it could be canceled in the event of a SIGINT, but
we didn't complete wiring that through the API of the getproviders
package.
Here we make the getproviders.Source interface methods, along with some
other functions that can make network requests, take a context.Context
argument and act appropriately if that context is cancelled.
The main providercache.Installer.EnsureProviderVersions method now also
has some context-awareness so that it can abort its work early if its
context reports any sort of error. That avoids waiting for the process
to wind through all of the remaining iterations of the various loops,
logging each request failure separately, and instead returns just
a single aggregate "canceled" error.
We can then set things up in the "terraform init" and
"terraform providers mirror" commands so that the context will be
cancelled if we get an interrupt signal, allowing provider installation
to abort early while still atomically completing any local-side effects
that may have started.
When init attempts to install a legacy provider required by state and
fails, but another provider with the same type is successfully
installed, this almost definitely means that the user is migrating an
in-house provider. The solution here is to use the `terraform state
replace-provider` subcommand.
This commit makes that next step clearer, by detecting this specific
case, and displaying a list of commands to fix the existing state
provider references.
If a provider changes namespace in the registry, we can detect this when
running the 0.13upgrade command. As long as there is a version matching
the user's constraints, we now use the provider's new source address.
Otherwise, warn the user that the provider has moved and a version
upgrade is necessary to move to it.
When applying a backend config override file, we must not check for the
presence of all required fields, as the override can be a partial
configuration. It is only valid to check for required fields after all
overrides have been merged, which init already does.
When loading a backend config override file, init was doing two things
wrong:
- First, if the file failed to parse, we accidentally didn't return,
which caused a panic due to the parsed body being nil;
- Secondly, we were overzealous with the validation of the file,
allowing only attributes. While most backend configs are attributes
only, the enhanced remote backend body also contains a `workspaces`
block, which we need to support here.
This commit fixes the first bug with an early return and adds test cases
for missing file and intentionally-blank filename (to clear the config).
We also add a schema validation for the backend block, based on the
backend schema itself. This requires constructing an HCL body schema so
that we can call `Content` and check for diagnostic errors.
The result is more useful errors when an invalid backend config override
file is used, while also supporting the enhanced remote backend config
fully.
Does not include tests specific to the remote backend, because the
mocking involved to allow the backend to fully initialize is too
involved to be worth it.
The workspace name can be overridden by setting a TF_WORKSPACE
environment variable. If this is done, we should still validate the
resulting workspace name; otherwise, we could end up with an invalid and
unselectable workspace.
This change updates the Meta.Workspace function to return an error, and
handles that error wherever necessary.
If the user specifies a host that isn't a provider registry in a provider
source address then we'll print out some specialized error messages for
different variants of that situation.
In particular, this includes a special case for when the error is on the
hostname "github.com", in anticipation of folks incorrectly attempting to
use GitHub repository URLs (or Go-style module paths that happen to be
on GitHub) to specify providers, so we can give a more specific hint about
that.
This is just a different presentation of an existing error case that we
are already covering in the installer tests, so there are no new tests
here. We could in principle have a test covering the exact text of these
error messages, but we don't have much precedent for command package tests
covering that level of cosmetic detail.
For Terraform v0.12 we introduced a special loading mode where we would
use the 0.11-syntax-compatible "earlyconfig" package as a heuristic to
identify situations where it was likely that the user was trying to use
0.11-only syntax that the upgrade tool might help with.
However, as the language has moved on that is no longer a suitable
heuristic in Terraform 0.13 and later: other new additions to the
language can cause the main loader to disagree with earlyconfig, which
would lead us to give poor advice about how to respond.
Instead, we'll now return the same generic "there are errors" message in
all syntax error cases. We have an extra message for errors in this
case (as compared to other commands) because "terraform init" is usually
the first command a new user interacts with and so this message gives some
extra explanation about what "terraform init" will do with the
configuration once it's valid.
This also includes a reset control character in the output of the message
as part of our ongoing mission to stop Terraform printing out whole
paragraphs of colored text, which can often be hard to read for various
reasons.
After installing providers, we validate the presence of an executable
file, and generate a selected versions lockfile. If this process fails,
notify the user. One possible cause for this is an invalid provider
package with a missing or misnamed executable file.
* command/init: return an error with invalid -backend-config files
The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.
Fixes#24845
* emphasize backend configuration file in docs
* internal/getproviders: decode and return any registry warnings
The public registry may include a list of warnings in the "versions"
response for any given provider. This PR adds support for warnings from
the registry and an installer event to return those warnings to the
user.
When using `-flag=value` with Powershell, unquoted values are broken
into separate arguments. This means that the following command:
terraform init -backend-config=./backend.conf
is interpreted by Terraform as:
terraform init -backend-config= ./backend.conf
This results in an empty backend-config setting (which is semantically
valid!) followed by a custom configuration path (pointing at a file).
Due to a bug where we could exit without printing diagnostics, this
would result in a silent failure that was very difficult to diagnose.
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.
This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.
For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
Fetching a default namespace provider from the public registry can
result in 404 Not Found error. This might be caused by a previously-
default provider moving to a new namespace, which means that the
configuration needs to be upgraded to use an explicit provider source.
This commit adds a more detailed diagnostic for this situation,
suggesting that the intended provider might be in a new namespace. The
recommended course of action is to run the 0.13upgrade command to
generate the correct required_providers configuration.
Relying on the early config for provider requirements was necessary in
Terraform 0.12, to allow the 0.12upgrade command to run after init
installs providers.
However in 0.13, the same restrictions do not apply, and the detection
of provider requirements has changed. As a result, the early config
loader gives incorrect provider requirements in some circumstances,
such as those in the new test in this commit.
Therefore we are changing the init command to use the requirements found
by the full configuration loader. This also means that we can remove the
internal initwd CheckCoreVersionRequirements function.
provider is not found.
Previously a user would see the following error even if terraform was
only searching the local filesystem:
"provider registry registry.terraform.io does not have a provider named
...."
This PR adds a registry-specific error type and modifies the MultiSource
installer to check for registry errors. It will return the
registry-specific error message if there is one, but if not the error
message will list all locations searched.
Previously the diagnostics from the config loaders (earlyconfig and
regular) were only appended to the overall diags if an error was found.
This adds all diagnostics from the regular config loader so that any
generated warnings will be displayed, even if there are no errors.
I did not add the `earlyconfig` warnings since they will be displayed if
there is an error and are likely to be duplicated by the config loader.
* internal/registry source: return error if requested provider version protocols are not supported
* getproviders: move responsibility for protocol compatibility checks into the registry client
The original implementation had the providercache checking the provider
metadata for protocol compatibility, but this is only relevant for the
registry source so it made more sense to move the logic into
getproviders.
This also addresses an issue where we were pulling the metadata for
every provider version until we found one that was supported. I've
extended the registry client to unmarshal the protocols in
`ProviderVersions` so we can filter through that list, instead of
pulling each version's metadata.
Providers installed from the registry are accompanied by a list of
checksums (the "SHA256SUMS" file), which is cryptographically signed to
allow package authentication. The process of verifying this has multiple
steps:
- First we must verify that the SHA256 hash of the package archive
matches the expected hash. This could be done for local installations
too, in the future.
- Next we ensure that the expected hash returned as part of the registry
API response matches an entry in the checksum list.
- Finally we verify the cryptographic signature of the checksum list,
using the public keys provided by the registry.
Each of these steps is implemented as a separate PackageAuthentication
type. The local archive installation mechanism uses only the archive
checksum authenticator, and the HTTP installation uses all three in the
order given.
The package authentication system now also returns a result value, which
is used by command/init to display the result of the authentication
process.
There are three tiers of signature, each of which is presented
differently to the user:
- Signatures from the embedded HashiCorp public key indicate that the
provider is officially supported by HashiCorp;
- If the signing key is not from HashiCorp, it may have an associated
trust signature, which indicates that the provider is from one of
HashiCorp's trusted partners;
- Otherwise, if the signature is valid, this is a community provider.
When a provider dependency is implicit rather than explicit, or otherwise
when version constraints are lacking, we produce a warning recommending
the addition of explicit version constraints in the configuration.
This restores the warning functionality from previous Terraform versions,
adapting it slightly to account for the new provider FQN syntax and to
recommend using a required_providers block rather than version constraints
in "provider" blocks, because the latter is no longer recommended in the
documentation.
The provider fully-qualified name string used in configuration is very
long, and since most providers are hosted in the public registry, most
of that length is redundant. This commit adds and uses a `ForDisplay`
method, which simplifies the presentation of provider FQNs.
If the hostname is the default hostname, we now display only the
namespace and type. This is only used in UI, but should still be
unambiguous, as it matches the FQN string parsing behaviour.
Built-in providers are special providers that are distributed as part of
Terraform CLI itself, rather than being installed separately. They always
live in the terraform.io/builtin/... namespace so it's easier to see that
they are special, and currently there is only one built-in provider named
"terraform".
Previous commits established the addressing scheme for built-in providers.
This commit makes the installer aware of them to the extent that it knows
not to try to install them the usual way and it's able to report an error
if the user requests a built-in provider that doesn't exist or tries to
impose a particular version constraint for a built-in provider.
For the moment the tests for this are the ones in the "command" package
because that's where the existing testing infrastructure for this
functionality lives. A later commit should add some more focused unit
tests here in the internal/providercache package, too.
This is a slightly different approach than we used to take for this
option: rather than disabling the installer and causing all future
commands to look elsewhere for plugins, we'll now leave the installer
enabled by constrain it to only look at the given directories.
This is overall simpler because it doesn't require any special tracking
of the plugin directories for subsequent commands. Instead, the selections
file generated by the installer will record the versions it selected from
the specified directories, and we'll link them in to the local cache just
as we would normally so that other commands don't need to do anything
special to select the right plugins in either case.
Back when we first introduced provider versioning in Terraform 0.10, we
did the provider version resolution in terraform.NewContext because we
weren't sure yet how exactly our versioning model was going to play out
(whether different versions could be selected per provider configuration,
for example) and because we were building around the limitations of our
existing filesystem-based plugin discovery model.
However, the new installer codepath is new able to do all of the
selections up front during installation, so we don't need such a heavy
inversion of control abstraction to get this done: the command package can
select the exact provider versions and pass their factories directly
to terraform.NewContext as a simple static map.
The result of this commit is that CLI commands other than "init" are now
able to consume the local cache directory and selections produced by the
installation process in "terraform init", passing all of the selected
providers down to the terraform.NewContext function for use in
implementing the main operations.
This commit is just enough to get the providers passing into the
terraform.Context. There's still plenty more to do here, including to
repair all of the tests this change has additionally broken.
There's still a lot of work to do here around both the UX and the
follow-up steps that need to happen after installation completes, but this
is enough to faciliate some initial end-to-end testing of the new-style
install process.
missingPlugins was hard-coded to work only with provider plugins, so I
renamed it to clarify the usage.
Also renamed a test provider from greater_than to greater-than as the
underscore is an invalid provider name character and this will become a
hard error in the near future.
Our initial Terraform 0.13.0 release will continue to support only the
hard-coded official HashiCorp signing key, with support for other keys to
follow in a later release once the trust infrastructure is in place to
support that.
This change is intended to (marginally) improve the UX for a possible
future situation where a HashiCorp-distributed provider makes a released
signed with a new key and a prior version of Terraform ends up trying to
install it due to incorrect version constraints. With this new text we
hope to give the user a better prompt for onward troubleshooting, but
in a sort of hedging way because we have not yet finalized the details of
how new keys might roll out in practice.
Hopefully a user seeing this message would consult the release notes for
Terraform itself and for the provider in question and find some
as-yet-undetermined information about how to proceed.
If the decentralized trust model design comes together before the v0.13.0
release then we may make further amendments here to prepare for that, but
that work should not block the v0.13.0 release if other work concludes
first.
* command: use backend config from state when backend=false is used.
When a user runs `terraform init --backend=false`, terraform should
inspect the state for a previously-configured backend, and use that
backend, ignoring any backend config in the current configuration. If no
backend is configured or there is no state, return a local backend.
Fixes#16593
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders
This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
Martin Atkins