Commit Graph

23952 Commits

Author SHA1 Message Date
James Bardin 97bde5467c filter nulls when shimming a config
Nulls can't exist in HCL, and the legacy sdk will panic when
encountering them. The map shim already did this, just copy the same
pattern in the list case.
2019-04-02 14:27:31 -04:00
Brian Flad a8e3787afc
helper/schema: Prevent setSet() panic with typed nil
References:

* https://github.com/hashicorp/terraform/issues/14418
* v0.9.5 (original bug report): a59ee0b30e/helper/schema/field_writer_map.go (L311)
* v0.11.12 (Terraform AWS Provider discovery): 057286e522/helper/schema/field_writer_map.go (L343)

When creating flatten functions in Terraform Providers that return *schema.Set, its possible to return a typed `nil`, e.g.

```go
func flattenHeaders(h *cloudfront.Headers) *schema.Set {
	if h.Items != nil {
		return schema.NewSet(schema.HashString, flattenStringList(h.Items))
	}
	return nil
}
```

This previously could cause a panic, e.g.

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1881911]

goroutine 1325 [running]:
github.com/hashicorp/terraform/helper/schema.(*MapFieldWriter).setSet(0xc00054bf00, 0xc00073efa0, 0x5, 0x5, 0x5828140, 0x0, 0xc0002cea50, 0xc000e996a8, 0xc001026e40)
	/Users/bflad/go/pkg/mod/github.com/hashicorp/terraform@v0.11.12/helper/schema/field_writer_map.go:343 +0x211
```

Here we catch the typed `nil` and return an empty list flatmap result instead. Unit testing result prior to code update:

```
--- FAIL: TestMapFieldWriter (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
  panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1777cdc]

goroutine 913 [running]:
testing.tRunner.func1(0xc00045b800)
  /usr/local/Cellar/go/1.12.1/libexec/src/testing/testing.go:830 +0x392
panic(0x192cf20, 0x2267ca0)
  /usr/local/Cellar/go/1.12.1/libexec/src/runtime/panic.go:522 +0x1b5
github.com/hashicorp/terraform/helper/schema.(*MapFieldWriter).setSet(0xc0004648a0, 0xc0004408d0, 0x1, 0x1, 0x19e3de0, 0x0, 0xc00045c600, 0x30, 0x19e0080)
  /Users/bflad/src/github.com/hashicorp/terraform/helper/schema/field_writer_map.go:344 +0x68c
github.com/hashicorp/terraform/helper/schema.(*MapFieldWriter).set(0xc0004648a0, 0xc0004408d0, 0x1, 0x1, 0x19e3de0, 0x0, 0x1, 0x18)
  /Users/bflad/src/github.com/hashicorp/terraform/helper/schema/field_writer_map.go:107 +0x28b
github.com/hashicorp/terraform/helper/schema.(*MapFieldWriter).WriteField(0xc0004648a0, 0xc0004408d0, 0x1, 0x1, 0x19e3de0, 0x0, 0x0, 0x0)
  /Users/bflad/src/github.com/hashicorp/terraform/helper/schema/field_writer_map.go:89 +0x504
github.com/hashicorp/terraform/helper/schema.TestMapFieldWriter(0xc00045b800)
  /Users/bflad/src/github.com/hashicorp/terraform/helper/schema/field_writer_map_test.go:337 +0x2ddd
testing.tRunner(0xc00045b800, 0x1a44f90)
  /usr/local/Cellar/go/1.12.1/libexec/src/testing/testing.go:865 +0xc0
created by testing.(*T).Run
  /usr/local/Cellar/go/1.12.1/libexec/src/testing/testing.go:916 +0x35a
```
2019-04-01 20:10:32 -04:00
Chris Griggs 16778fea92
Merge pull request #20887 from cgriggs01/cgriggs01-fix-typo
[Website] Update provider name
2019-04-01 14:35:46 -07:00
Martin Atkins 6c5819f910 configs/configupgrade: Prefer block syntax for list-of-object attributes
In order to preserve pre-v0.12 idiom for list-of-object attributes, we'll
prefer to use block syntax for them except for the special situation where
the user explicitly assigns an empty list, where attribute syntax is
required in order to allow existing provider logic to differentiate from
an implicit lack of blocks.
2019-04-01 13:30:24 -07:00
cgriggs01 cfd271a040 update provider name 2019-04-01 08:41:45 -07:00
Kristin Laemmert 39ef97beff
website: adding examples of complex variables on the command line (#20871) 2019-04-01 07:51:17 -04:00
Bruno Tavares 6bad319187 go.mod: remove unneeded indirect for "mgo" and "gocheck"
We don't appear to actually need to constrain these, and removing these decls may help avoid the need for `bzr` to install all the dependencies.
2019-03-29 16:35:22 -07:00
Martin Atkins 1f9a8aa75e
Update CHANGELOG.md 2019-03-29 14:54:01 -07:00
Frederic 0f1f504c22 command/format: indicate in diff when adding an attribute forces replacement 2019-03-29 14:52:13 -07:00
James Bardin e893bae5da
Merge pull request #20874 from hashicorp/jbardin/shim-unknown-values
Better handling of unknowns in sdk shims
2019-03-29 15:03:29 -04:00
James Bardin 64c76be804 fixup lost collections containing unknowns
It turns out that collections containing only unknowns could be lost,
meaning there wasn't a direct correlation between the unknown and null
value which would have otherwise been restored.
2019-03-29 14:54:54 -04:00
James Bardin 6868ddd085 don't set NewRemoved values as empty strings
NewRemoved diff values were somtimes left in maps and lists.
2019-03-29 13:56:43 -04:00
James Bardin 86e30add98 fix unknowns added to maps by schemaMap
The legacy diff process inserts unknown values into an optional+computed
map. Fix these up in post-plan normalization process, by looking for
known strings that were changed to unknown.
2019-03-29 13:56:43 -04:00
James Bardin 009df443f7 restore lost unknowns during a planned update.
Because schema.ResourceDiff can't differentiate between unknown
values and new computed values, unknowns can be lost during an update.
If a planned value converted an unknown to a null, restore the unknown
so that it can be correctly replaced in the final plan.
2019-03-29 13:56:43 -04:00
James Bardin 8b9fa6d05f add test provider coverage around unknown vals 2019-03-29 13:56:42 -04:00
Alex Somesan ef681e527d Rephrase for clarity 2019-03-29 16:53:13 +01:00
Sander van Harmelen 137974c4a8
Update CHANGELOG.md 2019-03-29 08:34:49 +01:00
Sander van Harmelen b94ddab3a7
Merge pull request #20857 from hashicorp/svh/b-console-vars
backend/remote: correctly load remote variables
2019-03-29 08:34:01 +01:00
Kristin Laemmert da52170797
configupgrade: fix test (#20863) 2019-03-28 18:00:33 -04:00
Pam Selle 4c1bb8abd7
Update CHANGELOG.md 2019-03-28 15:06:41 -04:00
Kristin Laemmert 1baa1b907e
configs/configupgrade: detect invalid resource names and print a TODO (#20856)
* configs/configupgrade: detect invalid resource names and print a TODO
message

In terraform 0.11 and prior it was possible to start a resource name
with a number. This is no longer valid, as the resource name would would
be ambiguous with number values in HCL expressions.

Fixes #19919

* Update configs/configupgrade/test-fixtures/valid/invalid-resource-name/want/resource.tf

Co-Authored-By: mildwonkey <mildwonkey@users.noreply.github.com>
2019-03-28 13:48:35 -04:00
Martin Atkins 003317d7c8 lang: Detect references when a list/set attr is defined using blocks
For compatibility with documented patterns from existing providers we are
now allowing (via a pre-processing step) any attribute whose type is a
list-of-object or set-of-object type to optionally be assigned using one
or more blocks whose type is the attribute name.

The pre-processing functionality was implemented in previous commits but
we were not correctly detecting references within these blocks that are,
from the perspective of the primary schema, invalid. Now we'll use an
alternative implementation of variable detection that is able to apply the
same schema rewriting technique we used to implement the transform and
thus can find all of the references as if they were already in their
final locations.
2019-03-28 10:41:01 -07:00
Martin Atkins 8746e9e8ad lang/blocktoattr: ExpandedVariables function
Because we handle FixUpBlockAttrs after dynamic block expansion, when
resolving variables we unfortunately need to consider the possibility of
both dynamic block expansion _and_ the block attrs fixup.

To accommodate this we have a variant of dynblock.VariablesHCLDec that
instead walks using the configschema.Block representation of the schema
and applies the same opportunistic schema rewrite used by FixUpBlockAttrs
at each body encountered during the walk.
2019-03-28 10:41:01 -07:00
Martin Atkins b35bc255d2 vendor: go get github.com/hashicorp/hcl2@master
This gives us an extra hook in the dynblock variables analysis that should
allow us to also make it subject also to the lang/blocktoattr fixup, to
ensure we'll find all the references in spite of these various
pre-processing wrappers.
2019-03-28 10:41:01 -07:00
Martin Atkins 87786484ea lang/eval: Apply attr-as-nested-block fixup in EvalBlock
For any block content we evaluate dynamically via this API, we'll make a
special allowance for users to optionally write members of a list
attribute instead as a sequence of nested blocks, thus allowing some
existing provider features that were assuming this capability to continue
to support it after v0.12.

This should not be used for any new provider features, and should ideally
be eventually phased out so that there aren't two
similar-but-slightly-different syntaxes for saying the same thing.
2019-03-28 10:41:01 -07:00
Martin Atkins 6dcf8195b8 lang/blocktoattr: Selectively allow block syntax to be used for attributes
This preprocessing step allows users to use nested block syntax to specify
elements of an attribute that is defined as being a list or set of an
object type.

This restores part of the unintended flexibility permitted in Terraform
v0.11 so that we can work around a few tricky edges where provider
implementations were relying on Terraform's failure to validate this in
earlier versions.

For any body that is pre-processed using this new helper, we will
recognize when a configuration author uses nested block syntax with a
name that is specified in the schema as an attribute of a suitable type
and tweak the schema just in time before decoding to expect that usage
and then fix up the result on the way out to conform to the original
schema.

Achieving this requires an abstraction inversion because only Terraform's
high-level schema has enough information to decide how to rewrite the
incoming low-level schema. We must therefore here implement HCL's
lowest-level API interface in terms of the higher-level abstractions of
hcldec and Terraform's configschema.

Because of the abstraction inversion this fixup mechanism cannot be used
generally for arbitrary HCL bodies but we can use it carefully inside the
lang package where its own API can guarantee the necessary invariants for
this to work.
2019-03-28 10:41:01 -07:00
Pam Selle d1c1bc6cdb
Merge pull request #20853 from pselle/err-handling-registry-unresponsive
Add friendly error for when registry unresponsive
2019-03-28 13:09:36 -04:00
Sander van Harmelen 39a95e4222 backend/remote: correctly load remote variables
When using `terraform console` in combination with the remote backend, variables defined in Terraform Enterprise were load loaded correctly.
2019-03-28 17:23:48 +01:00
Kristin Laemmert 931851ca82
update terraform-config-inspect (#20852) 2019-03-28 07:54:01 -04:00
Sander van Harmelen 86164c0bb4
Merge pull request #20843 from hashicorp/svh/f-force-push
backend/local: preserve serial and lineage on failure
2019-03-28 10:19:11 +01:00
Pam Selle ff7245f27c Add status link to make message more helpful 2019-03-27 16:22:37 -04:00
Chris Griggs c69cc88c8b
Merge pull request #20848 from cgriggs01/cgriggs01-jdcloud
[Website] Add JDcloud provider
2019-03-27 13:00:02 -07:00
Alex Somesan adebc65d95 Document that ValidateFunc works on maps. 2019-03-27 20:10:46 +01:00
cgriggs01 fc47a78e60 add jdcloud provider links 2019-03-27 11:41:00 -07:00
Pam Selle d72456d188 Add friendly error for when registry unresponsive
If the registry is unresponsive, you will now get an error
specific to this, rather than a misleading "provider unavailable" type
error. Also adds debug logging for when errors like this may occur
2019-03-27 14:39:14 -04:00
Sander van Harmelen 21cca34716 Fixup the docs for WriteStateForMigration 2019-03-27 17:41:11 +01:00
Sander van Harmelen 57f6e01830 backend/local: preserve serial and lineage on failure
When failing to write the state, the local backend writes the state to a local file called `errrored.tfstate`. Previously it would do so by creating a new state file which would use a new serial and lineage. By exorting the existing state file and directly assigning the new state, the serial and lineage are preserved.
2019-03-27 16:15:16 +01:00
Martin Atkins f302747077 website: Additional explanation for cidrsubnet function
Some users are not accustomed to thinking of IP addresses in a bitwise
fashion, so the hope here is to give enough of an introduction to that way
of thinking for the reader to understand what the "newbits" and "netnum"
arguments represent.
2019-03-26 10:04:29 -07:00
James Bardin a63c1af8e0
Merge pull request #20826 from hashicorp/jbardin/validate-connection
merge connection blocks for validation
2019-03-26 12:00:36 -04:00
James Bardin 3c8b46fffe merge connection blocks for validation
The resource connection block was not being validated. Merge the two
bodies, with the provider as the override, before validation.
2019-03-26 11:59:23 -04:00
Kristin Laemmert d5bb09b13b
command/show: refresh is required to show remote state (#20818) 2019-03-25 16:28:35 -04:00
Alex McLarty 4b9a764eb1 website: Fix typo in null_resource.html.markdown (#20817)
Fix typo in null_resource.html.markdown
2019-03-25 11:16:14 -07:00
Nick Fagerlund d0b7771634
website: Simplify wording for split built-in function (#20803)
The definition of split was referring the built-in function join. However, join is just one of the ways a string might have been created, and this could cause confusion.
2019-03-25 10:59:58 -07:00
Nick Fagerlund b32cce9637 website: Fix contradictory description of remote state `outputs` object 2019-03-25 10:59:10 -07:00
Nick Fagerlund 499ccd42bf website: fix broken markdown in modules/composition 2019-03-25 10:58:45 -07:00
Martin Atkins 1f241f8f83 docs: Some updates to the architecture summary
This doc was originally written in the middle of Terraform v0.12
development while some refactoring was already in progress. Here we update
those parts to refer to where the relevant functionality finally landed,
and also update some things that we did not anticipate changing at the
time but ended up having to be changed during the v0.12 development
anyway.
2019-03-25 08:01:44 -07:00
Martin Atkins 18c88f5a41 docs: Import some existing docs to bootstrap the codebase docs section
For a while now we've been gathering some codebase-level docs that felt
out-of-place on the main Terraform website (since they are about the
implementation, not usage) but we had no existing suitable place to put
them.

In order to make this information more available (and, hopefully, more
likely to stay up-to-date as we change things), here we'll establish the
"docs" directory as a place to keep documentation aimed at those who are
working on code changes to the Terraform Core codebase.

User-oriented docs should never appear in this directory. The Terraform
website is always the better place for those. The set of docs here is
rudimentary to start and we'll see if it makes sense to expand and
reorganize it over time based on the experience with having these initial
docs available.
2019-03-25 08:01:44 -07:00
James Bardin 657dddceab
Merge pull request #20799 from hashicorp/jbardin/tests-force-new
add (forces new resource) to provider test diffs
2019-03-25 10:38:11 -04:00
Nick Fagerlund 4200b0b2c5 website: Use bullet lists in smaller provider lists, too
These are small enough that using multi-columns would just get in the way,
but might as well style them similarly.
2019-03-22 17:06:29 -07:00
Nick Fagerlund b14b967fb7 website: Fix some alphabetization in provider lists 2019-03-22 17:06:29 -07:00