This brings in the new HCL extension functions "try", "can", and
"convert", along with the underlying HCL and cty infrastructure that allow
them to work.
* deps: bump terraform-config-inspect library
* configs: parse `version` in new required_providers block
With the latest version of `terraform-config-inspect`, the
required_providers attribute can now be a string or an object with
attributes "source" and "version". This change allows parsing the
version constraint from the new object while ignoring any given source attribute.
This also includes an upgrade to cty v1.1.1 because HCL calls for it.
The changes in these two libraries are mainly to codepaths that don't
directly affect Terraform, but including this upgrade will cause some
small improvements to Terraform's error messages for type conversion
problems.
In order to make this work reasonably we can't avoid using some funny
heuristics, which are somewhat reasonable to apply within the context of
Terraform itself but would not be good to add to the general "logutils".
Specifically, this is adding the additional heuristic that lines starting
with spaces are continuation lines and so should inherit the log level
of the most recent non-continuation line.
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
This new version includes Solaris support, the lack of which previously
caused us to disable readline-using features ("terraform console") on
Solaris builds.
We'll use this to call the Windows ReplaceFile API for safe file
replacement when updating credentials.tf.json in "terraform login" and
"terraform logout".
The dependencies here are dated and are causing conflicts with the
ACME provider, namely the version of the top-level autorest package.
This explicitly updates the Azure SDK and autorest packages, with the
separately versioned sub-packages being added automatically.
Notable changes (from Terraform AWS Provider CHANGELOG):
```
NOTES:
* backend/s3: Region validation now automatically supports the new `me-south-1` Middle East (Bahrain) region. For AWS operations to work in the new region, the region must be explicitly enabled as outlined in the [previous new region announcement blog post](https://aws.amazon.com/blogs/aws/now-open-aws-asia-pacific-hong-kong-region/). When the region is not enabled, the Terraform S3 Backend will return errors during credential validation (e.g. `error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid`).
* backend/s3: After this update, the AWS Go SDK will prefer credentials found via the `AWS_PROFILE` environment variable when both the `AWS_PROFILE` environment variable and the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables are statically defined. Previously the SDK would ignore the `AWS_PROFILE` environment variable, if static environment credentials were also specified. This is listed as a bug fix in the AWS Go SDK release notes.
ENHANCEMENTS:
* backend/s3: Add support for assuming role via web identity token via the `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN` environment variables
* backend/s3: Support automatic region validation for `me-south-1`
BUG FIXES:
* backend/s3: Load credentials via the `AWS_PROFILE` environment variable (if available) when `AWS_PROFILE` is defined along with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
```
Updated via:
```
go get github.com/aws/aws-sdk-go@v1.21.7
go mod tidy
go mod vendor
```
Verification with this update:
```hcl
terraform {
backend "s3" {
bucket = "me-south-1-testing"
key = "test"
region = "me-south-1"
}
}
output "test" {
value = timestamp()
}
```
Outputs:
```
$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
test = 2019-07-30T12:49:19Z
```
If the new region is not properly enabled for the account, errors like the below will be received:
```
$ terraform init
Initializing the backend...
Error: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
```
To use this region before this update:
```hcl
terraform {
# ... potentially other configuration ...
backend "s3" {
# ... other configuration ...
region = "me-south-1"
skip_region_validation = true
}
}
```
This includes a fix to make sure that an expression with a static string
index, like foo["bar"], will be parsed as a traversal rather than as a
dynamic index expression.
This module contains a YAML parser and encoder tailored to cty, though we
are mostly interested in it for its YAMLEncode and YAMLDecode cty
functions, which we can make available in Terraform.
This includes a small fix to ensure the parser doesn't produce an invalid
body for block parsing syntax errors, and instead produces an incomplete
result that calling applications like Terraform can still analyze.
The problem here was affecting our version-constraint-sniffing code, which
intentionally tried to find a core version constraint even if there's a
syntax error so that it can report that a new version of Terraform is a
likely cause of the syntax error. It was working in most cases, unless
it was the "terraform" block itself that contained the error, because then
we'd try to analyze a broken hcl.Block with a nil body.
This includes a new test for "terraform init" that exercises this
recovery codepath.
This corrects a bug in the HCL 2 scanner where a $ or % symbol would cause
incorrect tokenization if appearing immediately before a " .
This also includes some updates to Go extension libraries that the HCL
update brings in. Some of these changes update to support Unicode 11, but
only when compiling with Go 1.13, so we won't see the effect of these
changes until we start building Terraform with Go 1.13.
This contains an adjustment to how the dynamic blocks extension expands
a dynamic block whose for_each expression is unknown: it now produces an
block whose leaf attributes are all unknown, which is what Terraform had
previously been expecting but it wasn't actually true in practice.
cty now guarantees that sets of primitive values will iterate in a
reasonable order. Previously it was the caller's responsibility to deal
with that, but we invariably neglected to do so, causing inconsistent
ordering. Since cty prioritizes consistent behavior over performance, it
now imposes its own sort on set elements as part of iterating over them so
that calling applications don't have to worry so much about it.
This change also causes cty to consistently push unknown and null values
in sets to the end of iteration, where before that was undefined. This
means that our diff output will now consistently list additions before
removals when showing sets, rather than the ordering being undefined as
before.
The ordering of known, non-null, non-primitive values is still not
contractually fixed but remains consistent for a particular version of
cty.
By upgrading this to a version that supports Go Modules, we can avoid
the redundant dependencies labix.org/v2/mgo and launchpad.net/gocheck.
These dependencies come via hashicorp/consul, which we have vendored for
the Consul backend.
There are no changes to the go-msgpack code here; we're just adopting a
newer version that has a go.mod present so that its dependencies can be
determined more accurately.
This gives us an extra hook in the dynblock variables analysis that should
allow us to also make it subject also to the lang/blocktoattr fixup, to
ensure we'll find all the references in spite of these various
pre-processing wrappers.
This includes a fix to prevent unintentional infinite recursion when
trying to unify multiple object types to a single type for conversion to
list(any).
Sadly I wasn't able to reproduce the problem as reported (in #20728), so
therefore I wasn't able to write a Terraform test for it, but I have
confirmed that the cty behavior here was incorrect anyway (recursively
calling the same function we're already in with the same arguments is
clearly not productive) and so this change will allow whatever situation
that was to terminate with a type conversion error, rather than causing a
stack overflow.
It's likely that there is another bug lurking under this, since the
problematic code here was supposed to be unreachable, but avoiding the
crash is the priority for now. If the problem re-surfaces then it should
at least be an error message with some additional context about what the
goal of the caller was.
This also includes an unrelated fix for the gocty package, which doesn't
affect Terraform because it makes very little use of that package.
This includes improved functionality for HCL's "dynamic block extension",
which will allow us (in a subsequent commit) to properly detect
dependencies inside nested "dynamic" blocks, where currently they get
missed.
For this commit though, we just upgrade HCL to a version that includes it
and make a small change to our "lang" package to align with an upstream
renaming.
This includes two upstream fixes:
- Handle explicit JSON "null" consistently during decode of JSON syntax.
- Properly detect the end of a "heredoc" when formatting to avoid messing
up indentation of other lines following the heredoc.
This includes a fix for the parsing of object for expressions in newline-
sensitive contexts like block bodies.
It also includes a change to the JSON syntax decoder that cause it to
consider an explicit null to be equivalent to a property not being set at
all when interpreting a property value as a nested block. (It was
previously doing tha only when interpreting the property value as an
attribute value.)
Previously, configupgrade would panic if it encountered a HEREDOC. For
the time being, we will simply print out the HEREDOC as-is.
Unfortunately, we discovered that terraform 0.11's version of HCL
allowed for HEREDOCs with the termination delimiter inline (instead of
on a newline, which is technically correct). Since 0.12configupgrade
needs to be bug-compatible with terraform 0.11, we must roll back to the
same version of HCL used in terraform 0.11.
This fixes a bug in the TestConformance function that was generating false
positives when given two object types with the same number of attributes
but not identical attribute names.
The cty change here fixes a panic situation when cty.Path.Apply is given
a null value, making it now correctly return an error.
However, the HCL2 change includes an alternative to cty.Path.Apply that
uses HCL-level rules rather than cty-level rules, so the result behaves
like an HCL expression would. Most uses of cty.Path.Apply ought to use
hcl.ApplyPath instead, to ensure that the behavior is consistent with what
users expect in the main language.
An earlier commit incorrectly updated some versions in go.mod without also
updating the vendor tree, so this also rolls those back to where they used
to be so that we can roll them forward carefully and make sure the tests
actually pass. (If we just accept these new versions as specified the
tests do not pass, so some work is required to fix those regressions.)
This fixes some consistency problems with how number strings were parsed
in the msgpack decoder vs other situations.
This commit also includes an upgrade of HCL2 to use this new cty function,
though there's no change in behavior here since the new function is
functionally equivalent to what it replaced.
There are no code changes in this update. This just catches up with the
latest master commit in the upstream repo, which has only seen
documentation updates since the prior commit.
Martin Atkins