Commit Graph

24373 Commits

Author SHA1 Message Date
findkim 43b22d4e14 Fix plugin protocol test 2019-01-09 15:20:33 -06:00
cgriggs01 a9069cab7b edit fmt 2019-01-09 11:40:34 -08:00
James Bardin 35365e8ccf
Merge pull request #19950 from hashicorp/jbardin/resource-tests
Container checks in TestCheckFuncs
2019-01-09 14:26:08 -05:00
cgriggs01 885a366f2e add <tr> 2019-01-09 11:21:02 -08:00
cgriggs01 830e23b217 add new community providers 2019-01-09 10:39:32 -08:00
James Bardin 7973872524 allow TestCheckNoResourceAttr for empty containers
Stricter type handling in the new shims may add empty containers into
the state where they were previously elided. Since the detection of
missing and empty containers in the legacy state was never reliable,
allow TestCheckNoResourceAttr to succeed if the key is a container count
index, and the value is "0"
2019-01-09 13:09:02 -05:00
Brian Flad ed37d07632
backend/s3: Configure AWS Client MaxRetries and provide enhanced S3 NoSuchBucket error message
The AWS Go SDK automatically provides a default request retryer with exponential backoff that is invoked via setting `MaxRetries` or leaving it `nil` will default to 3. The terraform-aws-provider `config.Client()` sets `MaxRetries` to 0 unless explicitly configured above 0. Previously, we were not overriding this behavior by setting the configuration and therefore not invoking the default request retryer.

The default retryer already handles HTTP error codes above 500, including S3's InternalError response, so the extraneous handling can be removed. This will also start automatically retrying many additional cases, such as temporary networking issues or other retryable AWS service responses.

Changes:
* s3/backend: Add `max_retries` argument
* s3/backend: Enhance S3 NoSuchBucket error to include additional information
2019-01-09 13:01:37 -05:00
James Bardin c63040c737 have TestCheckResourceAttr accept missing counts
Missing containers were often erroneously kept in the state, but since
the addition of the new provider shims, they can often be correctly
eliminated. There are however many tests that check for a "0" count in
the flatmap state when there shouldn't be a key at all. This addition
looks for a container count key and "0" pair, and allows for the key to
be missing.

There may be some tests negatively effected by this which were
legitimately checking for empty containers, but those were also not
reliably detected, and there should be much fewer tests involved.
2019-01-09 13:01:17 -05:00
Kristin Laemmert cdf7cc2449
command/json*: updating documentation and adding tests (#19944)
A few minor fixes and cleanups as a result of said tests. Hooray for
eventual consistency!
2019-01-09 08:59:11 -08:00
Sander van Harmelen e997373f44
Update CHANGELOG.md 2019-01-09 09:10:47 +01:00
Sander van Harmelen bb1edadcf0
Merge pull request #19941 from hashicorp/svh/f-early-logging
backend/remote: log early to indicate execution started
2019-01-09 09:09:55 +01:00
James Bardin f7913bb168
Merge pull request #19943 from hashicorp/jbardin/empty-containers
Better handling for empty containers and zero values in provider shims
2019-01-08 16:54:12 -05:00
James Bardin 7455bf2a55 provider tests for empty values
Add tests to make limited use of empty container values and empty
strings.
2019-01-08 16:26:22 -05:00
James Bardin b55ec74c27 add copyMissingValues for normalizing shimmed Vals
Zero values and empty containers can be lost during the shimming
process, and during the provider's Apply step.

If we have known zero value containers and primitives in the source,
which appear as null values in the destination, we copy over the zero
value. Sets (and lists to an extent) are more difficult, since there
before and after indexes may not correlate. In that case we take the
entire container if it's wholly known, expecting the provider to have
correctly handled the value.
2019-01-08 16:26:22 -05:00
James Bardin 8300d65539 don't strip sets with count 1 when normalizing
normalizeFlatmapContainers should retain sets with a count of 1, and
convert sets with a count of 0 if they were 1 before the Apply step.
2019-01-08 16:26:21 -05:00
James Bardin f3c80b4765 add zero values to sets with a flatmap count of 1
If a flatmap value has a count of 1 and no other attributes, it usually
indicates the equivalent configuration of an empty (or default value)
set block. Treat this as containing a single zero value object and
insert that into the set.
2019-01-08 16:26:21 -05:00
James Bardin 3677522a28 insert empty objects into config from empty blocks
When creating a legacy config from a cty.Value, empty nested blocks
should corespond to empty objects in the config.
2019-01-08 15:16:46 -05:00
James Bardin 6f54bfaa7c send config during apply 2019-01-08 15:16:46 -05:00
Sander van Harmelen 3ce68d61a3
Update CHANGELOG.md 2019-01-08 18:26:53 +01:00
Sander van Harmelen ab2b17b98e
Merge pull request #19940 from hashicorp/svh/b-state-locking
command/state: add proper locking
2019-01-08 18:25:58 +01:00
Sander van Harmelen bba03384d5 backend/remote: log early to indicate execution started 2019-01-08 17:06:24 +01:00
Sander van Harmelen fb57e9b26f command/state: add proper locking 2019-01-08 14:57:52 +01:00
Martin Atkins cdad78d69b helper/resource: Allow multiple providers in a single TestCase
Due to incorrect use of a loop iterator variable inside a closure, all of
the given providers were ending up with the same factory function.
Now we copy the factory function to a local within the loop first so that
each iteration has its own variable.

This is the second round of similar bugs in this function, so we'll also
add a test case for it to reduce the risk of future regressions given that
most real callers don't exercise this with multiple providers in practice.
2019-01-07 16:58:36 -08:00
Martin Atkins d0e6a4c69a lang: Add "formatdate" function
We missed this one on a previous pass of bringing in most of the cty
stdlib functions.

This will resolve #17625 by allowing conversion from Terraform's
conventional RFC 3339 timestamps into various other formats.
2019-01-07 09:10:14 -08:00
Martin Atkins b190d3b4f2 helper/resource: Shim back to old state must preserve schema version
We use a shim to convert from the new state model back to the old because
the provider test API is still using the old API throughout. However, the
shim was not preserving the schema version recorded in the new-style state
and so a round-trip through this shim would cause the schema versions to
all revert to zero.

This can cause trouble with the destroy phase of provider tests because
(for API legacy reasons) we round-trip from old state back to new again
before the destroy phase and thus causing the providers to try to upgrade
from state version zero even though the data was already latest, which
can cause errors because state upgrades are generally not idempotent.
2019-01-05 10:00:30 -08:00
Martin Atkins 06acc3f6c8 helper/schema: Skip validation of unknown values
With the introduction of explicit "null" in 0.12 it's possible for a value
that is unknown during plan to become a known null during apply, so we
need to slightly weaken our validation rules to accommodate that, in
particular skipping the validation of conflicting attributes if the result
could potentially be valid after the unknown values become known.

This change is in the codepath that is common to both 0.12 and 0.11
callers, but that's safe because 0.11 re-runs validation during the apply
step and so will still catch problems here, albeit in the apply step
rather than in the plan step, thus matching the 0.12 behavior. This new
behavior is a superset of the old in the sense that everything that was
valid before is still valid.

The implementation here also causes us to skip all other validation for
an attribute whose value is unknown. Most of the downstream validation
functions handle this directly anyway, but again this doesn't add any new
failure cases, and should clean up some of the rough edges we've seen with
unknown values in 0.11 once people upgrade to 0.12-compatible providers.
Any issues we now short-circuit during planning will still be caught
during apply.

While working on this I found that the existing "Not a list" test was not
actually testing the correct behavior, so this also includes a tweak to
that to ensure that it really is checking the "should be a list" path
rather than the "cannot be set" codepath it was inadvertently testing
before.
2019-01-04 14:46:47 -08:00
appilon 148f6be244
Merge pull request #19917 from appilon/restore-state-remove
restore (via copypaste) terraform.State.Remove
2019-01-04 14:46:43 -05:00
Alex Pilon 660a854668
restore (via copypaste) terraform.State.Remove 2019-01-03 22:06:30 -05:00
Kristin Laemmert 3e9c51c726
command/jsonstate: marshal state into a machine readable blob o'json (#19911)
* command/jsonstate: marshal state into a machine readable blob o'json
* consistent json tags are nice
2019-01-03 12:08:03 -08:00
Martin Atkins c753df6a93 lang/funcs: templatefile function
This function is similar to the template_file data source offered by the
template provider, but having it built in to the language makes it more
convenient to use, allowing templates to be rendered from files anywhere
an inline template would normally be allowed:

    user_data = templatefile("${path.module}/userdata.tmpl", {
      hostname = format("petserver%02d", count.index)
    })

Unlike the template_file data source, this function allows values of any
type in its variables map, passing them through verbatim to the template.
Its tighter integration with Terraform also allows it to return better
error messages with source location information from the template itself.

The template_file data source was originally created to work around the
fact that HIL didn't have any support for map values at the time, and
even once map support was added it wasn't very usable. With HCL2
expressions, there's little reason left to use a data source to render
a template; the only remaining reason left to use template_file is to
render a template that is constructed dynamically during the Terraform
run, which is a very rare need.
2018-12-21 08:06:14 -08:00
James Bardin 725ccea6d4
Merge pull request #19732 from hashicorp/jbardin/terraform-remote-state
there is always an implied workspace of "default"
2018-12-21 09:11:18 -05:00
Martin Atkins faf4889c81
Update CHANGELOG.md 2018-12-20 14:51:18 -08:00
Kristin Laemmert 2b6dc13f29
jsonplan: remove "proposed_unknown" structure in favor of "after_unknown" field in "change" (#19709)
* jsonplan: remove "proposed_unknown" structure in favor of
"after_unknown" field in "change"
2018-12-20 14:30:18 -08:00
Martin Atkins 364d3ffc4a provider/test: Test for nested dynamic blocks
This is a HCL feature rather than a Terraform feature really, but we want
to make sure it keeps working consistently in future versions of Terraform
so this is a Terraform-flavored test for the block expansion behavior.

In particular, it tests that a nested dynamic block can access the parent
iterator, so that we won't regress #19543 in future.
2018-12-20 14:28:37 -08:00
Martin Atkins a3f6e67b7d vendor: upgrade github.com/hashicorp/hcl2
This includes a fix to the dynamic block extension so that nested dynamic
blocks can iterate over the temporary variables created by their ancestors.
2018-12-20 14:28:37 -08:00
Nick Fagerlund 17873ae61b website: Update functions section page titles 2018-12-20 14:20:42 -08:00
Nick Fagerlund 9f85e6d5cc website: Update configuration language section page titles 2018-12-20 14:20:42 -08:00
Nick Fagerlund d2abdc23ac website: Structural edit of configuration language docs
This commit is a wide-ranging set of edits to the pages under
/docs/configuration. Among other things, it

- Separates style conventions out into their own page.
- Separates type constraints and conversion info into their own page.
- Conflates similar complex types a little more freely, since the distinction is
  only relevant when restricting inputs for a reusable module or resource.
- Clarifies several concepts that confused me during edits.
2018-12-20 14:20:42 -08:00
James Bardin 41002e1a24 there is always an implied workspace of "default"
Fetching a state requires a workspace name, which should default to
"default"
2018-12-20 17:17:11 -05:00
Martin Atkins e39c69750c core: Specialized errors for incorrect indexes in resource reference
In prior versions of Terraform we permitted inconsistent use of indexes
in resource references, but in as of 0.12 the index usage must correlate
properly with whether "count" is set on the resource.

Since users are likely to have existing configurations with incorrect
usage, here we introduce some specialized error messages for situations
where we can detect such issues statically. This seems to cover all of the
common patterns we've seen in practice.

Some usage patterns will fall back on a less-helpful dynamic error here,
but no configurations coming from 0.11 can end up that way because 0.11
did not permit forms such as aws_instance.no_count[count.index].bar that
this validation would not be able to "see".

Our configuration upgrade tool also contains a fix for this already, but
it takes a more conservative approach of adding the index [1] rather than
[count.index] because it can't be sure (without human help) if correlation
of indices is what was intended.
2018-12-20 13:55:42 -08:00
Chris Griggs b50ac168d9
Merge pull request #19730 from cgriggs01/cgriggs01-hedvig
[Website] Hedvig provider doc links
2018-12-20 13:54:19 -08:00
cgriggs01 da370cd689 fix spacing in website docs 2018-12-20 13:27:15 -08:00
James Bardin 5ab9c049e3 update CHANGELOG.md 2018-12-20 15:37:20 -05:00
James Bardin 5ff003594e
Merge pull request #19726 from hashicorp/jbardin/destroy-plan
Don't fail decoding null values from plan changes
2018-12-20 15:35:28 -05:00
James Bardin f3fe6184a0 test for destroy plan round trip 2018-12-20 15:11:08 -05:00
cgriggs01 9df6e2df42 add hedvig provider links 2018-12-20 10:54:20 -08:00
James Bardin 1b8617cef0 don't attempt to decode empty changes values
An empty DynamicValue can't be decoded but indicates no state, so just
return a NullVal.
2018-12-20 13:06:53 -05:00
James Bardin a915f3f13e don't convert empty DynamicValue to nil 2018-12-20 10:28:26 -05:00
Martin Atkins c7c73bcfc2
Update CHANGELOG.md 2018-12-19 18:21:13 -08:00
Martin Atkins d627987e70 vendor: upgrade github.com/hashicorp/hcl2
This includes a change to accept and ignore a UTF-8 BOM at the start of
any given native syntax configuration.

Although a BOM is redundant in UTF-8, we learned in #18618 that several
software products on Windows will produce a BOM whenever they save as
UTF-8, so accepting it avoids friction when using those tools to author
or generate Terraform configuration files.

This fixes #18618.
2018-12-19 16:18:29 -08:00