Commit Graph

25778 Commits

Author SHA1 Message Date
Martin Atkins c6cbbcb79a website: Initial documentation for provider_installation in the CLI config
This is an initial draft of documentation for this new feature of the
CLI configuration. This is mainly intended as a placeholder for now,
because there are other documentation updates pending for the new provider
namespacing and installation scheme and we'll likely want to revise these
docs to better complement the broader documentation once it's written.
2020-04-23 10:52:01 -07:00
Martin Atkins 6b2050f42a main: Properly handle provider installation method exclusions
Previously we were incorrectly using the Include configuration for both
the include and exclude list, making the include portion totally
ineffective.
2020-04-23 10:52:01 -07:00
Martin Atkins 3167067029 command/e2etest: provider installation with explicit install methods
This exercises the ability to customize the installation methods used by
the provider plugin installer, in this case forcing the use of a custom
local directory with a result essentially the same as what happens when
you pass -plugin-dir to "terraform init".
2020-04-23 10:52:01 -07:00
Martin Atkins c7fe6b9160 command/cliconfig: handle provider_installation block in JSON syntax
The CLI config can be written in both native HCL and HCL JSON syntaxes, so
the provider_installation block must be expressible using JSON too. Our
previous checks to approximate HCL 2-level strictness were too strict for
HCL JSON where things are more ambiguous even in HCL 2, so this includes
some additional relaxations if we detect that we're decoding an AST
produced from a JSON file.

This is still subject to the quirky ways HCL 1 handles JSON though, so
the JSON value must be structured in a way that doesn't trigger HCL's
heuristics that try to guess what is a block and what is an attribute.
(This is the issue that HCL 2 fixes by always decoding using a schema;
there's more context on this in:
  https://log.martinatkins.me/2019/04/25/hcl-json/ )
2020-04-23 10:52:01 -07:00
Martin Atkins f5012c12da command/cliconfig: Installation methods, not installation sources
Unfortunately in the user model the noun "source" is already used for the
argument in the required_providers block to specify which provider to use,
so it's confusing to use the same noun to also refer to the method used to
obtain that provider.

In the hope of mitigating that confusion, here we use the noun "method",
as in "installation method", to talk about the decision between getting
a provider directly from its origin registry or getting it from some
mirror. This is distinct from the provider's "source", which is the
location where a provider _originates_ (prior to mirroring).

This noun is also not super awesome, but better than overloading an
existing term in the same feature.
2020-04-23 10:52:01 -07:00
Martin Atkins e872ec4461 command/cliconfig: Remove remnant extraArg checks in provider_installation
In the first pass of implementing this it was strict about what arguments
are allowed inside source blocks, but that was counter to our usual design
principles for CLI config where we tend to ignore unrecognized things to
allow for some limited kinds of future expansion without breaking
compatibility with older versions of Terraform that will be sharing the
same CLI configuration files with newer versions.

However, I'd removed the tracking of that prior to the initial commit. I
missed some leftover parts when doing that removal, so this cleans up the
rest of it.
2020-04-23 10:52:01 -07:00
Martin Atkins 8b75d1498f command/cliconfig: Use existing HTTP mirror source rather than new stub
An earlier commit added a redundant stub for a new network mirror source
that was already previously stubbed as HTTPMirrorSource.

This commit removes the unnecessary extra stub and changes the CLI config
handling to use it instead. Along the way this also switches to using a
full base URL rather than just a hostname for the mirror, because using
the usual "Terraform-native service discovery" protocol here doesn't isn't
as useful as in the places we normally use it (the mirror mechanism is
already serving as an indirection over the registry protocol) and using
a direct base URL will make it easier to deploy an HTTP mirror under
a path prefix on an existing static file server.
2020-04-23 10:52:01 -07:00
Martin Atkins b8856c677c cliconfig: Ignore config dir if TF_CLI_CONFIG_FILE envvar is set
When we originally introduced this environment variable it was intended to
solve for the use-case where a particular invocation of Terraform needs
a different CLI configuration than usual, such as if Terraform is being
run as part of an automated test suite or other sort of automated
situation with different needs than normal use.

However, we accidentally had it only override the original singleton CLI
config file, while leaving the CLI configuration directory still enabled.
Now we'll take the CLI configuration out of the equation too, so that only
the single specified configuration file and any other environment-sourced
settings will be included.
2020-04-23 10:52:01 -07:00
Martin Atkins 94b87e056b fixup main.go comment about providersource 2020-04-23 10:52:01 -07:00
Martin Atkins 5af1e6234a main: Honor explicit provider_installation CLI config when present
If the CLI configuration contains a provider_installation block then we'll
use the source configuration it describes instead of the implied one we'd
build otherwise.
2020-04-21 16:28:59 -07:00
Martin Atkins c5bd783eba internal/getproviders: Stub NetworkMirrorSource
This is a placeholder for later implementation of a mirror source that
talks to a particular remote HTTP server and expects it to implement the
provider mirror protocol.
2020-04-21 16:27:48 -07:00
Martin Atkins 2c535d829d command/cliconfig: Decode provider_installation blocks
This new CLI config block type allows explicitly specifying where
Terraform should look to find provider plugins for installation. This is
not used anywhere as of this commit, but in a future commit we'll change
package main to treat the presence of a block of this type as a request
to disable the default set of provider sources and use these explicitly-
specified ones instead.
2020-04-21 15:48:07 -07:00
Kristin Laemmert 8108face36
terraform: return `initialization required` error when provider schemas not found (#24715)
A side effect of the various changes to the provider installer included losing the initialization required error message which would occur if a user removed or modified the .terraform directory.

Previously, plugin factories were created after the configuration was loaded, in terraform.NewContext. Terraform would compare the required providers (from config and state) to the available providers and return the aforementioned error if a provider was missing.

Provider factories are now loaded at the beginning of any terraform command, before terraform even loads the configuration, and therefore before terraform has a list of required providers.

This commit replaces the current error when a providers' schema cannot be found in the provider factories with the init error, and adds a command test (to plan tests, for no real reason other than that's what I thought of first).
2020-04-21 16:29:27 -04:00
Petros Kolyvas eb76f41031
Merge pull request #24723 from hashicorp/cgriggs01-provider-links
[Website] Adding provider docs
2020-04-21 12:59:27 -04:00
Chris Griggs b6d602c162 [Website] Adding provider docs 2020-04-21 09:11:14 -07:00
Petros Kolyvas a31cd8059a
Merge pull request #24702 from bmcustodio/bmcustodio-change-username
Change my username.
2020-04-21 10:13:21 -04:00
Alisdair McDiarmid 7871c40a2c
Update CHANGELOG.md 2020-04-20 12:13:36 -04:00
Alisdair McDiarmid e32e7e2c4b
Merge pull request #24617 from hashicorp/alisdair/provider-installer-signature-verification
internal: Verify provider signatures on install
2020-04-20 12:11:40 -04:00
Bruno Miguel Custódio 1bb292bcdb
Change my username.
Signed-off-by: Bruno Miguel Custódio <brunomcustodio@gmail.com>
2020-04-18 08:34:22 +01:00
Martin Atkins 92d6a30bb4 main: skip direct provider installation for providers available locally
This more closely replicates the 0.12-and-earlier behavior, where having
at least one version of a provider installed locally would totally disable
any attempt to look for newer versions remotely.

This is just for the implicit default behavior. Assumption is that later
we'll have an explicit configuration mechanism that will allow the user
to specify exactly where to look for what, and thus avoid tricky
heuristics like this.
2020-04-17 13:55:11 -07:00
Alisdair McDiarmid a5b3d497cc internal: Verify provider signatures on install
Providers installed from the registry are accompanied by a list of
checksums (the "SHA256SUMS" file), which is cryptographically signed to
allow package authentication. The process of verifying this has multiple
steps:

- First we must verify that the SHA256 hash of the package archive
  matches the expected hash. This could be done for local installations
  too, in the future.
- Next we ensure that the expected hash returned as part of the registry
  API response matches an entry in the checksum list.
- Finally we verify the cryptographic signature of the checksum list,
  using the public keys provided by the registry.

Each of these steps is implemented as a separate PackageAuthentication
type. The local archive installation mechanism uses only the archive
checksum authenticator, and the HTTP installation uses all three in the
order given.

The package authentication system now also returns a result value, which
is used by command/init to display the result of the authentication
process.

There are three tiers of signature, each of which is presented
differently to the user:

- Signatures from the embedded HashiCorp public key indicate that the
  provider is officially supported by HashiCorp;
- If the signing key is not from HashiCorp, it may have an associated
  trust signature, which indicates that the provider is from one of
  HashiCorp's trusted partners;
- Otherwise, if the signature is valid, this is a community provider.
2020-04-17 13:57:19 -04:00
Kristin Laemmert f09ae6f862
provider source tests: added test suite to exercise hyphenated providers (#24685) 2020-04-16 15:54:33 -04:00
Paul Tyng 1c9735bab5
Merge pull request #24686 from hashicorp/paultyng-patch-1
Add page explaining provider trust tiers
2020-04-16 10:53:47 -04:00
Paul Tyng 3856dacc1e
Update website/docs/registry/providers/tiers.html.md 2020-04-16 10:03:52 -04:00
Paul Tyng 663a57b263
Update website/docs/registry/providers/tiers.html.md
Co-Authored-By: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-04-16 10:02:48 -04:00
Paul Tyng 0f25f796e1
Add page explaining provider trust tiers 2020-04-16 09:34:35 -04:00
Ryan Uber dcbd835ac3
Merge pull request #24655 from hashicorp/b-remote-policy-check-race
backend/remote: display cost estimate and policy check whenever available
2020-04-15 12:38:15 -07:00
Ryan Uber 105fcb3cac backend/remote: move reading policy check logs earlier 2020-04-15 12:31:44 -07:00
Alisdair McDiarmid 24ced6bbca
Update CHANGELOG.md 2020-04-15 14:28:34 -04:00
Noah Mercado d4d8812afa
Feature: Sum Function (#24666)
The sum function takes a list or set of numbers and returns the sum of those
numbers.
2020-04-15 14:27:06 -04:00
Chris Griggs 8d71337596
Merge pull request #24653 from hashicorp/cgriggs01-infoblox-links
[Website] provider links
2020-04-14 09:36:29 -07:00
Alisdair McDiarmid a2b21db556
Update CHANGELOG.md 2020-04-14 09:24:54 -04:00
Alisdair McDiarmid 8e7ace44c3
Merge pull request #24650 from hashicorp/alisdair/upgrade-hcl
vendor: go get github.com/hashicorp/hcl/v2@v2.4.0
2020-04-14 09:21:44 -04:00
Daniel Dreier c14d6f4241
Fix typo in CONTRIBUTING.md 2020-04-13 18:01:12 -07:00
Pam Selle ab101f99df
Merge pull request #24635 from hashicorp/pselle/docs-mod-count
Docs for count/for_each on modules
2020-04-13 18:39:59 -04:00
Ryan Uber 93906e1ad8 backend/remote: display cost estimate and policy check whenever available 2020-04-13 13:04:53 -07:00
Chris Griggs 8da255073e [Website] provider links 2020-04-13 12:29:15 -07:00
Pam Selle 146644a848 update addressing section 2020-04-13 12:13:18 -04:00
Pam Selle 608c17d518 Update providers block notes 2020-04-13 12:08:08 -04:00
Fred 76f583ac9f
website/docs: fix broken hcl json spec link (#24630) 2020-04-13 11:15:13 -04:00
Alisdair McDiarmid 9128ce611a vendor: go get github.com/hashicorp/hcl/v2@v2.4.0 2020-04-13 10:48:29 -04:00
Alisdair McDiarmid 65f9de04af
Update CHANGELOG.md 2020-04-13 09:44:47 -04:00
Alisdair McDiarmid 4cc18ce061
Merge pull request #24621 from hashicorp/alisdair/upgrade-go-cty
vendor: go get github.com/zclconf/go-cty@v1.4.0
2020-04-13 09:42:42 -04:00
Mikhail Mazurskiy c9ce1f0c1f
Fix typos (#24531) 2020-04-13 08:34:40 -04:00
Pam Selle 65c4ba9c85 First docs for count/for_each on modules 2020-04-10 15:14:06 -04:00
Kristin Laemmert 4079247270
Update CHANGELOG.md 2020-04-10 15:11:46 -04:00
Kristin Laemmert 269d511481 command/providers: refactor with new provider types and functions
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.

Additional tests to exercise providers in modules and providers from
state have been included.
2020-04-10 15:08:10 -04:00
Kristin Laemmert 0a5fb40fdf configs: include provider configs in ProviderRequirements()
This PR adds iteration through any provider configuration blocks in the
config in addProviderRequirements().

A stale comment (of mine!) would leave one expecting the
module.ProviderRequirements to include any requirements from provider
configs. The comment was inaccurate and has been updated.
2020-04-10 15:08:10 -04:00
James Bardin 5ddb1a5808
Merge pull request #24623 from hashicorp/jbardin/module-output-references
Correctly connect module output references during plan
2020-04-10 14:46:25 -04:00
James Bardin 46b85820ba
Update addrs/module_call.go
Co-Authored-By: Pam Selle <pam@hashicorp.com>
2020-04-10 14:45:36 -04:00