Commit Graph

156 Commits

Author SHA1 Message Date
James Bardin b79adeae02 save resolved providers for resources to state
Use the ResourceState.Provider field to store the full name of the
provider used during apply. This field is only used when a resource is
removed from the config, and will allow that resource to be removed by
the exact same provider with which it was created.

Modify the locations which might accept the alue of the
ResourceState.Provider field to detect that the name is resolved.
2017-11-07 13:09:36 -05:00
James Bardin 57470f6bf5 remove ProviderConfig Path and Inherited fields
maving all inheritance into core
2017-11-06 17:11:07 -05:00
James Bardin 1756b3d09f ProviderConfig needs an Inherited flag
If a provider configuration is inherited from another module, any
interpolations in that config won't have variables declared locally. Let
the config only be validated in it's original location.
2017-10-27 09:08:15 -04:00
James Bardin 2e505083cd change ProviderConfig.Scope to Path
Though it's intended for "interpolation scope", Path is generally used
for this elsewhere.
2017-10-27 09:08:15 -04:00
James Bardin 29e5a355b9 add "version" and "provider" to module config
Add the Version and Providers fields to the module config.

Add ProviderConfig.Scope, which will be used to record the original
path of a ProviderConfig for interpolation.
2017-10-27 09:08:15 -04:00
Martin Atkins bbf9725134 config: Validate resource "count" for HCL2-specified resources
This early validation uses interpolation of a placeholder value to achieve
some "best effort" validation of the validity of the count attribute.
Since HCL2-specified resources can't be interpolated using the main
interpolator, here we branch and use the HCL2 API to do a
largely-equivalent (though slightly less accurate) check.

In the long run we don't really need this extra check at all, since the
validation walk does a more accurate version of the same thing. However,
we're preserving this for now in the interests of minimizing the amount
of change for the main codepath during our experiment.
2017-10-03 17:47:01 -07:00
Martin Atkins 0a342e8dc2 config: allow local value interpolations in count
There is some additional, early validation on the "count" meta-argument
that verifies that only suitable variable types are used, and adding local
values to this whitelist was missed in the initial implementation.
2017-09-01 17:54:05 -07:00
Martin Atkins f6797d6cb0 config: parsing of "locals" blocks in configuration 2017-08-21 15:15:25 -07:00
Martin Atkins f70318097a config: fix provider version constraint validation
Previously we were using the "semver" library to parse version
constraints, but we switched over to go-version and encapsulated it
inside our own plugin/discovery package to reduce dependency sprawl in
the code.

This particular situation was missed when updating references to the new
path, which meant that our validation code disagreed with the rest of
the code about what is considered a valid version constraint string.
By using the correct function, we ensure that we catch early any invalid
versions.
2017-06-09 14:03:59 -07:00
Martin Atkins 0b14c2cdb3 Resolve resource provider types in config package
Previously the logic for inferring a provider type from a resource name
was buried a utility function in the 'terraform' package. Instead here we
lift it up into the 'config' package where we can make broader use of it
and where it's easier to discover.
2017-06-09 14:03:59 -07:00
Martin Atkins 7e7d4c70df config: allow version constraints on providers, but validate them
We now accept syntactically-valid version constraints on provider blocks,
though we still don't actually do anything with them.
2017-06-09 14:03:59 -07:00
Martin Atkins 73fc9985b2 config: add "version" argument to provider blocks, disabled
In future we will support version constraints on providers, so we're
reserving this attribute name that is currently not used by any builtin
providers.

For now using this will produce an error, since the rest of Terraform
(outside of the config parser) doesn't currently have this notion and we
don't want people to start trying to use it until its behavior is fully
defined and implemented.

It may be used by third-party providers, so this is a breaking change
worth warning about in CHANGELOG but one whose impact should be small.
Any third-party providers using this name should migrate to using a new
attribute name instead moving forward.
2017-06-09 14:03:59 -07:00
Martin Atkins 410b60cb7f Stop requiring multi-vars (splats) to be in array brackets
Prior to Terraform 0.7, lists in Terraform were just a shallow abstraction
on top of strings with a magic delimiter between items. Wrapping a single
string in brackets in the configuration was Terraform's prompt that it
needed to split the string on that delimiter during interpolation.

In 0.7, when first-class lists were added, this convention was preserved
by flattening lists-of-lists by one level when they were encountered in
configuration. However, there was an oversight in that change where it
did not correctly handle the case where the inner list was unknown.

In #14135 we removed some code that was flattening partially-unknown lists
into fully-unknown (untyped) values. This inadvertently exposed the missed
case from the previous paragraph, causing issues for list-wrapped splat
expressions with unknown members. While this worked fine for resources,
due to some fixup done inside helper/schema, this did not work for other
interpolation contexts such as module blocks.

Various attempts to fix this up and restore the flattening behavior
selectively were unsuccessful, due to a proliferation of assumptions all
over the core code that would be too risky to change just to fix this bug.

This change, then, takes the different approach of removing the
requirement that splats be presented inside list brackets. This
requirement didn't make much sense anymore anyway, since no other
list-returning expression had this constraint and so the rest of Terraform
was already successfully dealing with both cases.

This leaves us with two different scenarios:

- For resource arguments, existing normalization code in helper/schema
  does its own flattening that preserves compatibility with the common
  practice of using bracketed splats. This change proves this with a test
  within the "test" provider that exercises the whole Terraform core and
  helper/schema stack that assigns bracketed splats to list and set
  attributes.

- For arguments in other blocks, such as in module callsites, the
  interpolator's own flattening behavior applies to known lists,
  preserving compatibility with configurations from before
  partially-computed splats were possible, but those wishing to use
  partially-computed splats are required to drop the surrounding brackets.
  This is less concerning because this scenario was introduced only in
  0.9.5, so the scope for breakage is limited to those who adopted this
  new feature quickly after upgrading.

As of this commit, the recommendation is to stop using brackets around
splats but the old form continues to be supported for backward
compatibility. In a future _major_ version of Terraform we will probably
phase out this legacy form to improve consistency, but for now both
forms are acceptable at the expense of some (pre-existing) weird behavior
when _actual_ lists-of-lists are used.

This addresses #14521 by officially adopting the suggested workaround of
dropping the brackets around the splat. However, it doesn't yet allow
passing of a partially-unknown list between modules: that still violates
assumptions in Terraform's core, so for the moment partially-unknown lists
work only within a _single_ interpolation expression, and cannot be
passed around between expressions. Until more holistic work is done to
improve Terraform's type handling, passing a partially-unknown splat
through to a module will result in a fully-unknown list emerging on
the other side, just as was the case before #14135; this change just
addresses the fact that this was failing with an error in 0.9.5.
2017-05-23 11:22:37 -07:00
Patrick Decat 392166faea Typo: now => no (#14493)
> This validation checks that there are now splat variables referencing ourself. This currently is not allowed.

=> 

> This validation checks that there are no splat variables referencing ourself. This currently is not allowed.
2017-05-15 15:39:55 +03:00
Justin Campbell c1c3127d8e config: Generate error copy from valid types map
Renders as:

```
Variable 'invalid_type' type must be one of [string, map, list] - 'not_a_type' is not a valid type
```
2017-04-20 13:30:22 -07:00
Justin Campbell cf775ded0e config: Add 'list' to variable type error message 2017-04-20 13:30:22 -07:00
Martin Atkins 76dca009e0 Allow escaped interpolation-like sequences in variable defaults
The variable validator assumes that any AST node it gets from an
interpolation walk is an indicator of an interpolation. Unfortunately,
back in f223be15 we changed the interpolation walker to emit a LiteralNode
as a way to signal that the result is a literal but not identical to the
input due to escapes.

The existence of this issue suggests a bit of a design smell in that the
interpolation walker interface at first glance appears to skip over all
literals, but it actually emits them in this one situation. In the long
run we should perhaps think about whether the abstraction is right here,
but this is a shallow, tactical change that fixes #13001.
2017-03-29 09:25:57 -07:00
Mitchell Hashimoto e2ca2c5911
config: allow TerraformVars in count 2017-03-13 16:38:54 -07:00
Mitchell Hashimoto 6b4c007894
config: validate backend configuration can't contain interpolations 2017-02-17 16:27:01 -08:00
Mitchell Hashimoto b6bfc4798d
config: Resource.Count should type check
Fixes #11800

Type check the value of count so we don't panic on the conversion.

I wondered "why didn't we do this before?" There is no excuse for NOT
doing it at all but the reasoning was beacuse prior to the list/map work
in 0.7, the value couldn't be anything other than a string since any
primitive can turn into a string.

Regardless, we should've always done this.
2017-02-10 10:41:41 -08:00
Mitchell Hashimoto cf46e1c3e0
terraform: don't validate computed values in validate
This disables the computed value check for `count` during the validation
pass. This enables partial support for #3888 or #1497: as long as the
value is non-computed during the plan, complex values will work in
counts.

**Notably, this allows data source values to be present in counts!**

The "count" value can be disabled during validation safely because we
can treat it as if any field that uses `count.index` is computed for
validation. We then validate a single instance (as if `count = 1`) just
to make sure all required fields are set.
2017-01-27 21:15:43 -08:00
Mitchell Hashimoto 7b342100d0
config: add "backend" loading to the Terraform section 2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 928fce71f7
config: parse "when" and "on_failure" on provisioners 2017-01-19 18:10:21 -08:00
Mitchell Hashimoto 0c30caec7d
config: smarter provider alias usage validation
Fixes #4789

This improves the validation that valid provider aliases are used.

Previously, we required that provider aliases be defined in every module
they're used. This isn't correct because the alias may be used in a
parent module and inherited.

This removes that validation and creates the validation that a provider
alias must be defined in the used module or _any parent_. This allows
inheritance to work properly.

We've always had this type of validation for aliases because we believe
its a good UX tradeoff: typo-ing an alias is really painful, so we
require declaration of alias usage. It may add a small burden to
declare, but since relatively few aliases are used, it improves the
scenario where a user fat-fingers an alias name.
2016-12-16 16:47:32 -08:00
Mitchell Hashimoto 3878b8b093
config: Merge respects Terraform blocks, provider aliases, and more
Fixes #10715

`config.Merge` was not updated to support a number of new features. This
updates the codepath to merge various fields, including the `terraform`
block which was the issue in #10715.

The `Merge` API is called when an `_override` file is present to _merge_
configurations. Normally configurations are _appended_. Only an override
file triggers a _merge_.

I started working on a generic library to do this automatically awhile
back but never finished it. This might motivate me to do so. In the
interest of getting a fix out though, we'll continue the manual
approach.
2016-12-13 21:48:59 -08:00
Mitchell Hashimoto b5f1738e17
config: allow IDs to start with numbers 2016-12-09 12:35:23 -05:00
Mitchell Hashimoto 8b9e2c17cc
config: fix NameRegexp validation to allow single-char names 2016-12-08 23:13:19 -05:00
Mitchell Hashimoto 6b458160b9
config: disallow names starting with ints
Fixes #10597

This disallows any names for variables, modules, etc. starting with
ints. This causes parse errors with the new HIL parser and actually
causes long term ambiguities if we allow this.

I've also updated the upgrade guide to note this as a backwards
compatibility and how people can fix this going forward.
2016-12-08 23:01:51 -05:00
Mitchell Hashimoto 3665fea2db
config: validate that data sources don't have provisioners 2016-11-23 08:46:13 -08:00
Martin Atkins 399542a168 core: allow outputs to have descriptions (#9722)
We allow variables to have descriptions specified, as additional context
for a module user as to what should be provided for a given variable.

We previously lacked a similar mechanism for outputs. Since they too are
part of a module's public interface, it makes sense to be able to add
descriptions for these for symmetry's sake.

This change makes a "description" attribute valid within an "output"
configuration block and stores it within the configuration data structure,
but doesn't yet do anything further with it. For now this is useful only
for third-party tools that might parse a module's config to generate
user documentation; later we could expose the descriptions as part of
the "apply" output, but that is left for a separate change.
2016-11-18 19:09:43 +02:00
Mitchell Hashimoto 25d19ef3d0 Merge pull request #10080 from hashicorp/f-tf-version
terraform: support version requirement in configuration
2016-11-14 11:53:30 -08:00
Mitchell Hashimoto df34fa88ce Merge pull request #10076 from hashicorp/f-depend-module
terraform: depends_on can reference entire modules
2016-11-14 11:53:12 -08:00
Mitchell Hashimoto 85d3439fa0
config: parse and validate terraform.required_version 2016-11-12 16:22:35 -08:00
Mitchell Hashimoto 576b61a21d
config: validate depends_on with module values 2016-11-12 08:21:27 -08:00
Mitchell Hashimoto 10426ba619
config: parse depends_on for outputs 2016-11-11 17:46:34 -08:00
Mitchell Hashimoto 694b16de5d
config: ignore_changes cannot have interpolations
This is the limitation of all lifecycle attributes currently. Right now,
interpolations are allowed through and the user ends up thinking it
should work. We should give an error.

In the future it should be possible to support some minimal set of
interpolations (static variables, data sources even perhaps) but for now
let's validate that this doesn't work.
2016-10-24 23:06:33 -07:00
Sander van Harmelen 47dd1ad153 Add wildcard (match all) support to ignore_changes (#8599) 2016-09-02 15:44:35 +02:00
Mitchell Hashimoto fbf06e2a59
config: vars must be unique 2016-08-25 14:51:49 -07:00
Mitchell Hashimoto 099293b690
config: outputs must be unique 2016-08-25 14:43:57 -07:00
Mitchell Hashimoto f4faf2274b
config: count can't be a SimpleVariable 2016-08-16 13:48:12 -07:00
James Nugent 01cd596c60 core: Fix detection of empty list/map defaults
This commit changes config parsing from weak decoding lists and maps
into []string and map[string]string respectively to decode into
[]interface{} and map[string]interface{} respectively. This is in order
to take advantage of the work integrated in #7082 to defeat the backward
compatibility features of the mapstructure library.

Test coverage of loading empty variables and validating their default
types against expectation.
2016-06-12 11:19:03 +02:00
James Nugent cb9ef298f3 core: Defeat backward compatibilty in mapstructure
The mapstructure library has a regrettable backward compatibility
concern whereby a WeakDecode of []interface{}{} into a target of
map[string]interface{} yields an empty map rather than an error. One
possibility is to switch to using Decode instead of WeakDecode, but this
loses the nice handling of type conversion, requiring a large volume of
code to be added to Terraform or HIL in order to retain that behaviour.

Instead we add a DecodeHook to our usage of the mapstructure library
which checks for decoding []interface{}{} or []string{} into a map and
returns an error instead.

This has the effect of defeating the code added to retain backwards
compatibility in mapstructure, giving us the correct (for our
circumstances) behaviour of Decode for empty structures and the type
conversion of WeakDecode.

The code is identical to that in the HIL library, and packaged into a
helper.
2016-06-08 18:38:41 +01:00
Martin Atkins 60c24e3319 command: Prevent data resources from being tainted
Since the data resource lifecycle contains no steps to deal with tainted
instances, we must make sure that they never get created.

Doing this out in the command layer is not the best, but this is currently
the only layer that has enough information to make this decision and so
this simple solution was preferred over a more disruptive refactoring,
under the assumption that this taint functionality eventually gets
reworked in terms of StateFilter anyway.
2016-05-14 08:26:37 -07:00
Martin Atkins 718cdda77b config: Parsing of data.TYPE.NAME.FIELD variables
This allows ${data.TYPE.NAME.FIELD} interpolation syntax at the
configuration level, though since there is no special handling of them
in the core package this currently just acts as an alias for
${TYPE.NAME.FIELD}.
2016-05-14 08:26:35 -07:00
Martin Atkins fc4fa10981 config: "ResourceMode" concept for resources
Previously resources were assumed to always support the full set of
create, read, update and delete operations, and Terraform's resource
management lifecycle.

Data sources introduce a new kind of resource that only supports the
"read" operation. To support this, a new "Mode" field is added to
the Resource concept within the config layer, which can be set to
ManagedResourceMode (to indicate the only mode previously possible) or
DataResourceMode (to indicate that only "read" is supported).

To support both managed and data resources in the tests, the
stringification of resources in config_string.go is adjusted slightly
to use the Id() method rather than the unusual type[name] serialization
from before, causing a simple mechanical adjustment to the loader tests'
expected result strings.
2016-05-14 08:26:35 -07:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
James Nugent e57a399d71 core: Use native HIL maps instead of flatmaps
This changes the representation of maps in the interpolator from the
dotted flatmap form of a string variable named "var.variablename.key"
per map element to use native HIL maps instead.

This involves porting some of the interpolation functions in order to
keep the tests green, and adding support for map outputs.

There is one backwards incompatibility: as a result of an implementation
detail of maps, one could access an indexed map variable using the
syntax "${var.variablename.key}".

This is no longer possible - instead HIL native syntax -
"${var.variablename["key"]}" must be used. This was previously
documented, (though not heavily used) so it must be noted as a backward
compatibility issue for Terraform 0.7.
2016-05-10 14:49:13 -04:00
James Nugent b62f6af158 core: Add support for marking outputs as sensitive (#6559)
* core: Add support for marking outputs as sensitive

This commit allows an output to be marked "sensitive", in which case the
value is redacted in the post-refresh and post-apply list of outputs.

For example, the configuration:

```
variable "input" {
    default = "Hello world"
}

output "notsensitive" {
    value = "${var.input}"
}

output "sensitive" {
    sensitive = true
    value = "${var.input}"
}
```

Would result in the output:

```
terraform apply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

  notsensitive = Hello world
  sensitive    = <sensitive>
```

The `terraform output` command continues to display the value as before.

Limitations: Note that sensitivity is not tracked internally, so if the
output is interpolated in another module into a resource, the value will
be displayed. The value is still present in the state.
2016-05-09 15:46:07 -04:00
James Nugent a0cc7115b3 deps: Update call sites of hil.Eval from update
hil.Eval() now returns (hil.EvaluationResult, error) instead of (value,
type, error). This commit updates the call sites, but retains all
previous behaviour. Tests are also updated.
2016-04-18 16:37:12 -07:00
James Nugent d7d39702c0 Type check variables between modules (#6185)
These tests demonstrates a problem where the types to a module input are 
not checked. For example, if a module - inner - defines a variable
"should_be_a_map" as a map, or with a default variable of map, we do not
fail if the user sets the variable value in the outer module to a string
value. This is also a problem in nested modules.

The implementation changes add a type checking step into the graph
evaluation process to ensure invalid types are not passed.
2016-04-15 12:07:54 -07:00