Commit Graph

19850 Commits

Author SHA1 Message Date
James Bardin a529b64cc8 better init error output
Provide log-form message when a provider isn't found, along with the
desired constraints.
2017-06-09 14:03:59 -07:00
James Bardin fdbfc17fae missing constraints passed erroniously
ConstrainVersions was documented as returning nil, but it was instead
returning an empty set. Use the Count() method to check for nil or
empty. Add test to verify failed constraints will show up as missing.
2017-06-09 14:03:59 -07:00
James Bardin 283ae0c7c4 fix comment and add some logging in discovery 2017-06-09 14:03:59 -07:00
James Bardin ea3d9c86ba sort constraints for consistent output 2017-06-09 14:03:59 -07:00
James Bardin 190d5fae76 vendor x/net/html 2017-06-09 14:03:59 -07:00
Martin Atkins 55d5e8a689 website: initial v0.10 upgrade guide
Further iteration of this will undoubtedly be needed before the final
release, but this is a start.
2017-06-09 14:03:59 -07:00
Martin Atkins 98ef947ed1 website: correction of heading and usage in "providers" command docs 2017-06-09 14:03:59 -07:00
Martin Atkins fb405ff296 website: new documentation about provider version configuration 2017-06-09 14:03:59 -07:00
Martin Atkins 766f8e5d64 command init: remove confusing uses of "environment" in the usage
"environment" is a very overloaded term, so here we prefer to use the
term "working directory" to talk about a local directory where operations
are executed on a given Terraform configuration.
2017-06-09 14:03:59 -07:00
Martin Atkins eaf5a62197 website: don't recommend to use "terraform init" to clone examples
This form of "terraform init" is vestigial at this point and being phased
out in 0.10. Something similar may return in a later version for
installing modules from a more formal module library, but for now we are
advising to use git manually to simplify the UX for "terraform init".
2017-06-09 14:03:59 -07:00
Martin Atkins d9b8c3cac0 website: "Initialization" section in the getting started guide
It's now required to run "terraform init" to get the provider plugins
installed, so we need to mention that in the intro guide.
2017-06-09 14:03:59 -07:00
Martin Atkins 3c429b3628 command init: show log output for each provider plugin downloaded
Each provider plugin will take at least a few seconds to download, so
providing feedback about each one should make users feel less like
Terraform has hung.

Ideally we'd show ongoing progress during the download, but that's not
possible without re-working go-getter, so we'll accept this as an interim
solution for now.
2017-06-09 14:03:59 -07:00
Martin Atkins 1b673746fd core: don't allow core or providers to change between plan and apply
The information stored in a plan is tightly coupled to the Terraform core
and provider plugins that were used to create it, since we have no
mechanism to "upgrade" a plan to reflect schema changes and so mismatching
versions are likely to lead to the "diffs didn't match during apply"
error.

To allow us to catch this early and return an error message that _doesn't_
say it's a bug in Terraform, we'll remember the Terraform version and
plugin binaries that created a particular plan and then require that
those match when loading the plan in order to apply it.

The planFormatVersion is increased here so that plan files produced by
earlier Terraform versions _without_ this information won't be accepted
by this new version, and also that older versions won't try to process
plans created by newer versions.
2017-06-09 14:03:59 -07:00
Martin Atkins 4571a16b15 command: remove Meta.forceProviderSHA256s
This was added with the idea of using it to override the SHA256 hashes
to match those hypothetically stored in a plan, but we already have a
mechanism elsewhere for populating context fields from plan fields, so
this is not actually necessary.
2017-06-09 14:03:59 -07:00
Martin Atkins 9aae06db97 command: update mockGetProvider.GetProvider for new interface
The expected type was changed in the mainline code but the tests were not
updated to match.
2017-06-09 14:03:59 -07:00
Martin Atkins 4ba20f9c1c command init: show suggested constraints for unconstrained providers
When running "terraform init" with providers that are unconstrained, we
will now produce information to help the user update configuration to
constrain for the particular providers that were chosen, to prevent
inadvertently drifting onto a newer major release that might contain
breaking changes.

A ~> constraint is used here because pinning to a single specific version
is expected to create dependency hell when using child modules. By using
this constraint mode, which allows minor version upgrades, we avoid the
need for users to constantly adjust version constraints across many
modules, but make major version upgrades still be opt-in.

Any constraint at all in the configuration will prevent the display of
these suggestions, so users are free to use stronger or weaker constraints
if desired, ignoring the recommendation.
2017-06-09 14:03:59 -07:00
Martin Atkins f70318097a config: fix provider version constraint validation
Previously we were using the "semver" library to parse version
constraints, but we switched over to go-version and encapsulated it
inside our own plugin/discovery package to reduce dependency sprawl in
the code.

This particular situation was missed when updating references to the new
path, which meant that our validation code disagreed with the rest of
the code about what is considered a valid version constraint string.
By using the correct function, we ensure that we catch early any invalid
versions.
2017-06-09 14:03:59 -07:00
James Bardin dbbafbd43f clean up plugin fetching
We can filter the allowed versions and sort them before checking the
protocol version, that way we can just return the first one found
reducing network requests.
2017-06-09 14:03:59 -07:00
James Bardin e0f2235f66 update init command with new GetProvider signature
GetProvider needs the plugin protocol version to be passed in
2017-06-09 14:03:59 -07:00
James Bardin 48d37131e0 more tests for fetching providers
Extend the test reslease server to return the protocol version header
and a dummy zip file for the provider.

Test filtering the plugins by plugin protocol version and add a full
GetProvder test.
2017-06-09 14:03:59 -07:00
James Bardin 5f053a2e64 refactor GetProvider
Get provider needs to be provided with the plugin protocol version,
because it can't be imported here directly.

The plugin url types and methods were confusing; replace them with a few
functions to format the urls.
2017-06-09 14:03:59 -07:00
James Bardin 8ad67991a5 check protocol version on plugins
Verify that the plugin we're requesting has a compatible protocol
version.
2017-06-09 14:03:59 -07:00
James Bardin fdeb3d929c Add Versions.Sort
Sort versions from newest to oldest.
2017-06-09 14:03:59 -07:00
Martin Atkins 032f71f1ff command: produce provider lock file during "terraform init"
Once we've installed the necessary plugins, we'll do one more walk of
the available plugins and record the SHA256 hashes of all of the plugins
we select in the provider lock file.

The file we write here gets read when we're building ContextOpts to
initialize the main terraform context, so any command that works with
the context will then fail if any of the provider binaries change.
2017-06-09 14:03:59 -07:00
Martin Atkins 04bcece59c plugin/discovery: handle the -Xn suffix used by auto-installed plugins
This is used to mark the plugin protocol version. Currently we actually
just ignore this entirely, since only one protocol version exists anyway.
Later we will need to add checks here to ensure that we only pay attention
to plugins of the right version.
2017-06-09 14:03:59 -07:00
Martin Atkins 6ba6508ec9 command: pass the locked plugin hashes into ContextOpts
By reading our lock file and passing this into the context, we ensure that
only the plugins referenced in the lock file can be used. As of this
commit there is no way to create that lock file, but that will follow soon
as part of "terraform init".

We also provide a way to force a particular set of SHA256s. The main use
for this is to allow us to persist a set of plugins in the plan and
check the same plugins are used during apply, but it may also be useful
for automated tests.
2017-06-09 14:03:59 -07:00
Martin Atkins aa1c644499 core: allow setting required plugin hashes on Context
When set, this information gets passed on to the provider resolver as
part of the requirements information, causing us to reject any plugins
that do not match during initialization.
2017-06-09 14:03:59 -07:00
Martin Atkins 720670fae7 command: helper to manage the provider plugins lock file
This is just a JSON file with the SHA256 digests of the plugin
executables.
2017-06-09 14:03:59 -07:00
Martin Atkins 7d0a98af46 command: provider resolver to also check SHA256 constraints when set
In addition to looking for matching versions, the caller can also
optionally require a specific executable by its SHA256 digest.
2017-06-09 14:03:59 -07:00
Martin Atkins e3401947a6 plugin/discovery: PluginRequirements can specify SHA256 digests
As well as constraining plugins by version number, we also want to be
able to pin plugins to use specific executables so that we can detect
drift in available plugins between commands.

This commit allows such requirements to be specified, but doesn't yet
specify any such requirements, nor validate them.
2017-06-09 14:03:59 -07:00
Martin Atkins 9a398a7793 command: require resource to be in config before import
Previously we encouraged users to import a resource and _then_ write the
configuration block for it. This ordering creates lots of risk, since
for various reasons users can end up subsequently running Terraform
without any configuration in place, which then causes Terraform to want
to destroy the resource that was imported.

Now we invert this and require a minimal configuration block be written
first. This helps ensure that the user ends up with a correlated resource
config and state, protecting against any inconsistency caused by typos.

This addresses #11835.
2017-06-09 14:03:59 -07:00
Martin Atkins 7d8719150c command: validate import resource address early
Previously we deferred validation of the resource address on the import
command until we were in the core guts, which caused the error responses
to be rather unhelpful.

By validating these things early we can give better feedback to the user.
2017-06-09 14:03:59 -07:00
Martin Atkins f6305fcc27 command: remove commented-out, misplaced tests
For some reason there was a block of commented-out tests for the refresh
command in the test file for the import command. Here we remove them to
reduce the noise in this file.
2017-06-09 14:03:59 -07:00
Martin Atkins 190626e2a8 core: improve consistency of ParseResourceAddress errors
Previously one of the errors had a built-in context message and the other
did not, making it hard for callers to present a user-friendly message
in both cases.

Now we generate an error message of the same form in both cases, with one
case providing additional information. Ideally the main case would be
able to give more specific guidance too, but that's hard to achieve with
the current regexp-based parsing implementation.
2017-06-09 14:03:59 -07:00
Martin Atkins b82ef2e30e core: ResourceAddress.MatchesConfig method
This is a useful building block for filtering configuration based on a
resource address. It is similar in principle to state filtering, but for
specific resource configuration blocks.
2017-06-09 14:03:59 -07:00
Martin Atkins f695e8b330 provider/test: allow test_resource to be imported
When working on the core import code, it's useful to have a zero-cost
local resource to work with for quick iteration.
2017-06-09 14:03:59 -07:00
Martin Atkins edf3cd7159 core: ResourceAddress.WholeModuleAddress method
This allows growing the scope of a resource address to include all of the
resources in the same module as the targeted resource. This is useful to
give context in error messages.
2017-06-09 14:03:59 -07:00
Martin Atkins 05a5eb0047 core: ResourceAddress.HasResourceSpec method
The resource address documentation defines a resource address as being in
two parts: the module path and the resource spec. The resource spec can
be omitted, which represents addressing _all_ resources in a module.

In some cases (such as import) it doesn't make sense to address an entire
module, so this helper makes it easy for validation code to check for
this to reject insufficiently-specific resource addresses.
2017-06-09 14:03:59 -07:00
James Bardin 044ad5ef59 rename some Constraints methods per code review 2017-06-09 14:03:59 -07:00
James Bardin 211f5b5d6e add test for newestVersion 2017-06-09 14:03:59 -07:00
James Bardin 46190590cb change []*Version to []Version
Versions are used as values, so don't keep them as pointers here
2017-06-09 14:03:59 -07:00
James Bardin a547e7c2f0 add releases version listing test 2017-06-09 14:03:59 -07:00
James Bardin 2994c6ac5d add init getPlugin test
add a mock plugin getter, and test that we can fetch requested version
of the plugins.
2017-06-09 14:03:59 -07:00
James Bardin 4c32cd432a make getProvider pluggable 2017-06-09 14:03:59 -07:00
James Bardin 66ebff90cd move some more plugin search path logic to command
Make less to change when we remove the old search path
2017-06-09 14:03:59 -07:00
James Bardin 2749946f5c basic plugin getter
Add discovery.GetProviders to fetch plugins from the relases site.

This is an early version, with no tests, that only (probably) fetches
plugins from the default location. The URLs are still subject to change,
and since there are no plugin releases, it doesn't work at all yet.
2017-06-09 14:03:59 -07:00
James Bardin fa49c69793 expose Version.NewerThan
NewerThan is useful outside of the package
2017-06-09 14:03:59 -07:00
James Bardin 7d2d951f27 Rename VersionSet to Constraints
VersionSet is a wrapper around version.Contraints, so rename it it as
such.
2017-06-09 14:03:59 -07:00
James Bardin 718ede0636 have Meta.Backend use a Config rather than loading
Instead of providing the a path in BackendOpts, provide a loaded
*config.Config instead. This reduces the number of places where
configuration is loaded.
2017-06-09 14:03:59 -07:00
Martin Atkins d7d8ea9543 website: Initial docs for the new "providers" subcommand
This will be fleshed out later as part of more holistic documentation for
the new provider plugin separation, but this is some minimal documentation
for just this subcommand.
2017-06-09 14:03:59 -07:00