There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.
This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.
In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.
Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.
The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
excludes Terraform versions prior to v0.12 then the configuration is
probably _already_ upgraded and so it's just a normal syntax error,
even if the early loader didn't detect it.
Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
This is an adaptation of the installation code from configs/configload,
now using the "earlyconfig" package instead of the "configs" package.
Module installation is an initialization-only process, with all other
commands assuming an already-initialized directory. Having it here can
therefore simplify the API of configs/configload, which can now focus only
on the problem of loading modules that have already been installed.
The old installer code in configs/configload is still in place for now
because the caller in "terraform init" isn't yet updated to use this.
"terraform init" is quite a complex beast in relation to other commands
since almost everything it does is unique to it and thus not factored out
into other packages.
To get some of that sprawl out of the "command" package, this new internal
package will give us somewhere to put this init functionality that is
also useful for test code that needs to mimic the initialization behavior
against fixture directories.
This is an alternative to the full config loader in the "configs" package
that is good for "early" use-cases like "terraform init", where we want
to find out what our dependencies are without getting tripped up on any
other errors that might be present.
This reverts commit 3677522a28.
Later changes negate the need for this, and removing these again
prevents us from having to strip them back out when helper/schema
doesn't want them.
This document was previously copied to the "Extending Terraform" section (in the
terraform-website repo), and the old URL was redirected so that the copy in
/guides can no longer be reached on the website. But the old copy of the file
remained, and now it runs the risk of confusing contributors, since the copy in
terraform-website/.../docs/extend is the more up-to-date version.
* command/show: added test scaffold for json output
More test cases will be added once the basic shape of the tests is
validated.
- command/json* packages now sort resources by address, matching
behavior elsewhere
- using cmp in tests instead of reflect.DeepEqual for the diffs
- updating expected output in tests to match sorting
Provider tests often rely on checking values contained within sets, by
directly accessing their flatmapped representation. In order to provider
the test harness with the expected set hashes, the sets must be
generated by the schema.Resource itself.
During the test we now build a fixed map of the providers, which should
only contain schema.Provider instances, and pass them into each
TestStep. The individual schema.Resource instances can then be pulled
from the providers, and used to recreate the state from the cty.Value
returned by the core operations.
Stricter type handling in the new shims may add empty containers into
the state where they were previously elided. Since the detection of
missing and empty containers in the legacy state was never reliable,
allow TestCheckNoResourceAttr to succeed if the key is a container count
index, and the value is "0"
The AWS Go SDK automatically provides a default request retryer with exponential backoff that is invoked via setting `MaxRetries` or leaving it `nil` will default to 3. The terraform-aws-provider `config.Client()` sets `MaxRetries` to 0 unless explicitly configured above 0. Previously, we were not overriding this behavior by setting the configuration and therefore not invoking the default request retryer.
The default retryer already handles HTTP error codes above 500, including S3's InternalError response, so the extraneous handling can be removed. This will also start automatically retrying many additional cases, such as temporary networking issues or other retryable AWS service responses.
Changes:
* s3/backend: Add `max_retries` argument
* s3/backend: Enhance S3 NoSuchBucket error to include additional information
Missing containers were often erroneously kept in the state, but since
the addition of the new provider shims, they can often be correctly
eliminated. There are however many tests that check for a "0" count in
the flatmap state when there shouldn't be a key at all. This addition
looks for a container count key and "0" pair, and allows for the key to
be missing.
There may be some tests negatively effected by this which were
legitimately checking for empty containers, but those were also not
reliably detected, and there should be much fewer tests involved.
Zero values and empty containers can be lost during the shimming
process, and during the provider's Apply step.
If we have known zero value containers and primitives in the source,
which appear as null values in the destination, we copy over the zero
value. Sets (and lists to an extent) are more difficult, since there
before and after indexes may not correlate. In that case we take the
entire container if it's wholly known, expecting the provider to have
correctly handled the value.
If a flatmap value has a count of 1 and no other attributes, it usually
indicates the equivalent configuration of an empty (or default value)
set block. Treat this as containing a single zero value object and
insert that into the set.
Martin Atkins