Commit Graph

24371 Commits

Author SHA1 Message Date
Martin Atkins e63a1dfb96 lang: EvalExpr only convert if wantType is not dynamic
This actually seems to be a bug in the underlying cty Convert function
since converting to cty.DynamicPseudoType should always just return the
input verbatim, but it seems like it's actually converting unknown values
of any type to be cty.DynamicVal, losing the type information.

We should eventually fix this in cty too, but having this extra check in
the Terraform layer is harmless and allows us to make progress without
context-switching.
2018-12-07 17:05:36 -08:00
Martin Atkins bf25e12792 vendor: upgrade github.com/hashicorp/hcl2
This includes a change to improve the precision of types returned from
splat expressions when a source value is unknown.
2018-12-07 17:05:36 -08:00
--global 32d0c6faf9 release: clean up after v0.12.0-alpha4 2018-12-08 00:50:51 +00:00
--global 2c36829d32
v0.12.0-alpha4 2018-12-07 20:36:48 +00:00
Martin Atkins 55469cd416 helper/resource: Get schemas from Terraform context
Previously the test harness was preloading schemas from the providers
before running any test steps.

Since terraform.NewContext already deals with loading provider schemas,
we can instead just use the schemas it loaded for our shimming needs,
avoiding the need to reimplement the schema lookup behavior and thus
the need to create a throwaway provider instance with which to do it.
2018-12-07 08:12:59 -08:00
Martin Atkins a4991c5780 helper/resource: Create a separate provider instance each call
Previously we were running the factory function only once when
constructing the provider resolver, which means that all contexts created
from that resolver share the same provider instance.

Instead now we will call the given factory function once for each
instantiation, ensuring that each caller ends up with a separate object
as would be the case in real-world use.
2018-12-07 08:12:59 -08:00
Sander van Harmelen 70689f5aa1
Merge pull request #19555 from hashicorp/svh/f-entitlements
backend/remote: use entitlements to select backends
2018-12-07 09:13:35 +01:00
Brian Flad 9a3b02cd6c
backend/s3: Support DynamoDB, IAM, and STS endpoint configurations
This change enables a few related use cases:
* AWS has partitions outside Commercial, GovCloud (US), and China, which are the only endpoints automatically handled by the AWS Go SDK. DynamoDB locking and credential verification can not currently be enabled in those regions.
* Allows usage of any DynamoDB-compatible API for state locking
* Allows usage of any IAM/STS-compatible API for credential verification
2018-12-07 03:10:51 -05:00
Martin Atkins e8f9fad0e3 states/statemgr: use -mod=vendor to run the state locking helper
This ensures that we test using the same source as we're using everywhere
else, and more tactically also ensures that when running in Travis-CI we
won't try to download all of the dependencies of Terraform during this
test.

In the long run we will look for a more global solution to this, rather
than adding this to all of our embedded "go" command calls directly, but
this is intended as a low-risk solution to get the build working again in
the mean time.
2018-12-06 16:50:06 -08:00
--global cd2dd4715d release: clean up after v0.12.0-alpha3 2018-12-06 19:03:41 +00:00
--global 7200ed2b61
v0.12.0-alpha3 2018-12-06 17:53:16 +00:00
James Bardin 9831b56f2e
Merge pull request #19560 from hashicorp/jbardin/go-plugin
go plugin update
2018-12-05 20:43:29 -05:00
James Bardin 98870fadb1
Merge pull request #19544 from hashicorp/jbardin/import-tests
Fix provider import tests
2018-12-05 20:30:46 -05:00
James Bardin 1d4c8403a2 enable Auto mTLS in go-plugin 2018-12-05 20:30:16 -05:00
James Bardin 1178c799b6 update vendor from go.mod 2018-12-05 20:30:16 -05:00
James Bardin bf71b89a78 update go-plugin to our provisional alpha branch 2018-12-05 20:29:03 -05:00
Martin Atkins 9336b54dc0 vendor: Fix dependency on github.com/kardianos/osext
In 98c8ac0862 I merged a change to the vendored code for this module but
didn't spot that it didn't also update the dependency metadata to match.
Here we just catch up the metadata to match the vendored version, with
no change to the vendored code itself.
2018-12-05 17:07:46 -08:00
Chris Griggs 40a54cbb7f
Merge pull request #19562 from cgriggs01/cgriggs01-skytap
[Website] Skytap provider links
2018-12-05 16:18:57 -08:00
cgriggs01 808c76fa9b Add skytap links 2018-12-05 15:38:47 -08:00
James Bardin ac63d2995f
Merge pull request #19559 from hashicorp/jbardin/resource-test-shim
don't add numeric indexes to resources with a count of 0
2018-12-05 17:34:30 -05:00
Brian Flad ef0f66a7d4
Update CHANGELOG for #19548 2018-12-05 16:42:17 -05:00
Brian Flad a3b327dbac
Merge pull request #19548 from hashicorp/b-helper-schema-NewComputed-propagation
helper/schema: Always propagate NewComputed for previously zero value primitive type attributes
2018-12-05 16:39:58 -05:00
Alex Gottschalk 029f54792f
remove approximate release date
"Later this summer" is long-gone.
2018-12-05 12:19:08 -08:00
James Bardin 7d296f752c don't add numeric indexes to resources with a count of 0 2018-12-05 13:41:53 -05:00
Martin Atkins 8112f589c1 configs/configupgrade: Pass through connection and provisioner blocks
This is a temporary implementation of these rules just so that these can
be passed through verbatim (rather than generating an error) while we
do testing of other features.

A subsequent commit will finish these with their own custom rulesets.
2018-12-05 10:25:01 -08:00
Martin Atkins 028b5ba34e configs/configupgrade: Upgrade depends_on in resources and outputs 2018-12-05 10:25:01 -08:00
Martin Atkins ef017345f1 configs/configupgrade: Upgrade the resource "lifecycle" nested block
The main tricky thing here is ignore_changes, which contains strings that
are better given as naked traversals in 0.12. We also handle here mapping
the old special case ["*"] value to the new "all" keyword.
2018-12-05 10:25:01 -08:00
Martin Atkins 4b52148262 configs/configupgrade: Upgrade provider addresses
Both resource blocks and module blocks contain references to providers
that are expressed as short-form provider addresses ("aws.foo" rather than
"provider.aws.foo").

These rules call for those to be unwrapped as naked identifiers during
upgrade, rather than appearing as quoted strings. This also introduces
some further rules for other simpler meta-arguments that are required
for the test fixtures for this feature.
2018-12-05 10:25:01 -08:00
Martin Atkins ea3b8b364c configs/configupgrade: Initial passthrough mapping for module blocks
Some further rules are required here to deal with the meta-arguments we
accept inside these blocks, but this is good enough to pass through most
module blocks using the standard attribute-expression-based mapping.
2018-12-05 10:25:01 -08:00
Martin Atkins 4b5d31d35d configs/configupgrade: Rules-based upgrade for "locals" block
Previously we were handling this one as a special case, effectively
duplicating most of the logic from upgradeBlockBody.

By doing some prior analysis of the block we can produce a "rules" that
just passes through all of the attributes as-is, allowing us to reuse
upgradeBlockBody. This is a little weird for the locals block since
everything in it is user-selected names, but this facility will also be
useful in a future commit for dealing with module blocks, which contain
a mixture of user-chosen and reserved argument names.
2018-12-05 10:25:01 -08:00
Sander van Harmelen 9062d887b8 backend/remote: use entitlements to select backends
Use the entitlements to a) determine if the organization exists, and b) as a means to select which backend to use (the local backend with remote state, or the remote backend).
2018-12-05 12:29:08 +01:00
Sander van Harmelen 03ac6ec774 go-mod: update the `go-tfe` dependency 2018-12-05 11:46:37 +01:00
Farid Neshat 44a45b7332 helper/schema: Fix setting a set in a list
The added test in this commit, without the fix, will make d.Set return
the following error:

`Invalid address to set: []string{"ports", "0", "set"}`

This was due to the fact that setSet in feild_writer_map tried to
convert a slice into a set by creating a temp set schema and calling
writeField on that with the address(`[]string{"ports", "0", "set"}"` in
this case). However the temp schema was only for the set and not the
whole schema as seen in the address so, it should have been `[]string{"set"}"`
so it would align with the schema.

This commits adds another variable there(tempAddr) which will only
contain the last entry of the address that would be the set key, which
would match the created schema

This commit potentially fixes the problem described in #16331
2018-12-05 10:09:54 +01:00
Brian Flad 1e81a3e7fa
helper/schema: Always propagate NewComputed for previously zero value primative type attributes
When the following conditions were met:
* Schema attribute with a primative type (e.g. Type: TypeString) and Computed: true
* Old state of attribute set to zero value for type (e.g. "")
* Old state ID of resource set to non-empty (e.g. existing resource)

Attempting to use CustomizeDiff with SetNewComputed() would result in the difference  previously being discarded. This update ensures that previous zero values or resource existence does not influence the propagation of the computed update.

Previously:

```
--- FAIL: TestSetNewComputed (0.00s)
    --- FAIL: TestSetNewComputed/NewComputed_should_always_propagate (0.00s)
        resource_diff_test.go:684: Expected (*terraform.InstanceDiff)(0xc00051cea0)({
             mu: (sync.Mutex) {
              state: (int32) 0,
              sema: (uint32) 0
             },
             Attributes: (map[string]*terraform.ResourceAttrDiff) (len=1) {
              (string) (len=3) "foo": (*terraform.ResourceAttrDiff)(0xc0003dcec0)({
               Old: (string) "",
               New: (string) "",
               NewComputed: (bool) true,
               NewRemoved: (bool) false,
               NewExtra: (interface {}) <nil>,
               RequiresNew: (bool) false,
               Sensitive: (bool) false,
               Type: (terraform.DiffAttrType) 0
              })
             },
             Destroy: (bool) false,
             DestroyDeposed: (bool) false,
             DestroyTainted: (bool) false,
             Meta: (map[string]interface {}) <nil>
            })
            , got (*terraform.InstanceDiff)(0xc00051ce80)({
             mu: (sync.Mutex) {
              state: (int32) 0,
              sema: (uint32) 0
             },
             Attributes: (map[string]*terraform.ResourceAttrDiff) {
             },
             Destroy: (bool) false,
             DestroyDeposed: (bool) false,
             DestroyTainted: (bool) false,
             Meta: (map[string]interface {}) <nil>
            })

--- FAIL: TestSchemaMap_Diff (0.01s)
    --- FAIL: TestSchemaMap_Diff/79-NewComputed_should_always_propagate_with_CustomizeDiff (0.00s)
        schema_test.go:3289: expected:
            *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"foo":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}

            got:
            <nil>

FAIL
FAIL  github.com/hashicorp/terraform/helper/schema  0.825s
```
2018-12-04 22:48:30 -05:00
James Bardin af9d046afb update CHANGELOG.md 2018-12-04 18:43:40 -05:00
James Bardin 484d67028a handle shim errors in provider tests
These should be rare, and even though it's likely a shim bug, the error
is probably easier for provider developers to deal with than the
panic
2018-12-04 16:33:16 -05:00
James Bardin 547d63bcde remove empty flatmap containers from test states
Don't compare attributes with zero-length flatmap continers in tests.
2018-12-04 16:33:16 -05:00
James Bardin 7f9d76cbf5 add implied providers during import
The CLI adds the provider references during import, but tests may not
have them specified.
2018-12-04 16:04:19 -05:00
James Bardin a65624e0c1 export ShimLegacyState to use it in resource tests
The helper/resource test harness needs to shim the legacy states as
well. Export this so we can get an error to check too.
2018-12-04 15:38:34 -05:00
Martin Atkins 76221a3a4a configs/configupgrade: Retain any .tf.json files unchanged
We don't change JSON files at all and instead just emit a warning about
them since JSON files are usually generated rather than hand-written and
so any updates need to happen in the generator program rather than in its
output.

However, we do still need to copy them verbatim into the output map so
that we can keep track of them through any subsequent steps.
2018-12-04 11:37:39 -08:00
Martin Atkins 038b295ef1 vendor: upgrade github.com/hashicorp/hcl2
This includes a fix to the HCL scanner to properly parse multi-line
comments.
2018-12-04 11:37:39 -08:00
Martin Atkins 8490fc36f7 configs/configupgrade: Fix up references to counted/uncounted resources
Prior to v0.12 Terraform was liberal about these and allowed them to
mismatch, but now it's important to get this right so that resources
and resource instances can be used directly as object values, and so
we'll fix up any sloppy existing references so things keep working as
expected.

This is particularly important for the pattern of using count to create
conditional resources, since previously the "true" case would create one
instance and Terraform would accept an unindexed reference to that.
2018-12-04 11:37:39 -08:00
Martin Atkins ceedeb69a9 configs/configupgrade: Normalize and upgrade reference expressions
The reference syntax is not significantly changed, but there are some
minor additional restrictions on identifiers in HCL2 and as a special case
we need to rewrite references to data.terraform_remote_state .

Along with those mandatory upgrades, we will also switch references to
using normal index syntax where it's safe to do so, as part of
de-emphasizing the old strange integer attribute syntax (like foo.0.bar).
2018-12-04 11:37:39 -08:00
Martin Atkins e83976c008 configs/configupgrade: Print trailing comments inside blocks
Previously we were erroneously moving these out of their original block
into the surrounding body. Now we'll make sure to collect up any remaining
ad-hoc comments inside a nested block body before closing it.
2018-12-04 11:37:39 -08:00
Martin Atkins de0eb9ea30 configs/configupgrade: Distinguish data resources in analysis
Early on it looked like we wouldn't need to distinguish these since we
were only analyzing for provider types, but we're now leaning directly
on the resource addresses later on and so we need to make sure we produce
valid ones when data resources are present.
2018-12-04 11:37:39 -08:00
Martin Atkins 6596d031d9 configs/configupgrade: Convert block-as-attr to dynamic blocks
Users discovered that they could exploit some missing validation in
Terraform v0.11 and prior to treat block types as if they were attributes
and assign dynamic expressions to them, with some significant caveats and
gotchas resulting from the fact that this was never intended to work.

However, since such patterns are in use in the wild we'll convert them
to a dynamic block during upgrade. With only static analysis we must
unfortunately generate a very conservative, ugly dynamic block with
every possible argument set. Users ought to then clean up the generated
configuration after confirming which arguments are actually required.
2018-12-04 11:37:39 -08:00
Martin Atkins f96d702d4f configs/configupgrade: Upgrading of simple nested blocks 2018-12-04 11:37:39 -08:00
Martin Atkins 39c3e7112f configs/configupgrade: Use break to cancel default function rendering
We're using break elsewhere in here so it was weird to have a small set
of situations that return instead, which could then cause confusion for
future maintenance if a reader doesn't notice that control doesn't always
leave the outer switch statement.
2018-12-04 11:37:39 -08:00
Martin Atkins 48f1245e6b configs/configupgrade: Replace lookup(...) with index syntax
If lookup is being used with only two arguments then it is equivalent to
index syntax and more readable that way, so we'll replace it.

Ideally we'd do similarly for element(...) here but sadly we cannot
because we can't prove in static analysis that the user is not relying
on the modulo wraparound behavior of that function.
2018-12-04 11:37:39 -08:00
Martin Atkins 4927a4105b configs/configupgrade: Replace calls to list() and map()
We now have native language features for declaring tuples and objects,
which are the idiomatic way to construct sequence and mapping values that
can then be converted to list, set, and map types as needed.
2018-12-04 11:37:39 -08:00