* tools/terraform-bundle: refactor to use new provider installer and
provider directory layouts
terraform-bundle now supports a "source" attribute for providers,
uses the new provider installer, and the archive it creates preserves
the new (required) directory hierarchy for providers, under a "plugins"
directory.
This is a breaking change in many ways: source is required for any
non-HashiCorp provider, locally-installed providers must be given a
source (can be arbitrary, see docs) and placed in the expected directory
hierarchy, and the unzipped archive is no longer flat; there is a new
"plugins" directory created with providers in the new directory layout.
This PR also extends the existing test to check the contents of the zip
file.
TODO: Re-enable e2e tests (currently suppressed with a t.Skip)
This commit includes an update to our travis configuration, so the terraform-bundle e2e tests run. It also turns off the e2e tests, which will fail until we have a terraform 0.13.* release under releases.hashicorp.com. We decided it was better to merge this now instead of waiting when we started seeing issues opened from users who built terraform-bundle from 0.13 and found it didn't work with 0.12 - better that they get an immediate error message from the binary directing them to build from the appropriate release.
A side effect of the various changes to the provider installer included losing the initialization required error message which would occur if a user removed or modified the .terraform directory.
Previously, plugin factories were created after the configuration was loaded, in terraform.NewContext. Terraform would compare the required providers (from config and state) to the available providers and return the aforementioned error if a provider was missing.
Provider factories are now loaded at the beginning of any terraform command, before terraform even loads the configuration, and therefore before terraform has a list of required providers.
This commit replaces the current error when a providers' schema cannot be found in the provider factories with the init error, and adds a command test (to plan tests, for no real reason other than that's what I thought of first).
This more closely replicates the 0.12-and-earlier behavior, where having
at least one version of a provider installed locally would totally disable
any attempt to look for newer versions remotely.
This is just for the implicit default behavior. Assumption is that later
we'll have an explicit configuration mechanism that will allow the user
to specify exactly where to look for what, and thus avoid tricky
heuristics like this.
Providers installed from the registry are accompanied by a list of
checksums (the "SHA256SUMS" file), which is cryptographically signed to
allow package authentication. The process of verifying this has multiple
steps:
- First we must verify that the SHA256 hash of the package archive
matches the expected hash. This could be done for local installations
too, in the future.
- Next we ensure that the expected hash returned as part of the registry
API response matches an entry in the checksum list.
- Finally we verify the cryptographic signature of the checksum list,
using the public keys provided by the registry.
Each of these steps is implemented as a separate PackageAuthentication
type. The local archive installation mechanism uses only the archive
checksum authenticator, and the HTTP installation uses all three in the
order given.
The package authentication system now also returns a result value, which
is used by command/init to display the result of the authentication
process.
There are three tiers of signature, each of which is presented
differently to the user:
- Signatures from the embedded HashiCorp public key indicate that the
provider is officially supported by HashiCorp;
- If the signing key is not from HashiCorp, it may have an associated
trust signature, which indicates that the provider is from one of
HashiCorp's trusted partners;
- Otherwise, if the signature is valid, this is a community provider.
There is no codepath that can use this any longer, since we need to
evaluate the modules as whole objects.
This means we're going to have to live for now with invalid module
output references returning "object" errors rather that "module".
In order to efficiently build the module objects for evaluation, we need
to collect the outputs from a set of module instances. The ModuleOutputs
method will return a copy of the state outputs, while not requiring the
unnecessary copying of each entire module.
We need all module instance outputs to build the objects for evaluation,
but there is no need to copy all the resource instances along with that.
This allows us to only return the output states, with enough information
to connect them with their module instances.
The evaluationStateData needs the change to the GetModule method to work
with the new evaluator. This is using a deep copy of module instances,
which we will clean up after some changes to the states package.
Since modules need to be evaluated as whole objects, yet the outputs are
all handled individually, we need a method to collect and return all
output changes for a module from the plan, including all known
module instances.
Module references, like resource references, need to always return the
and object containing all instances in order to handle modules as single
values, and to postpone index evaluation to when the expression as whole
is evaluated.
In order to be able to use module values, and handle operations like
possibly invalid module indexes in conditional statements, whole modules
must always be returned during evaluation.
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.
Additional tests to exercise providers in modules and providers from
state have been included.
Kristin Laemmert