Commit Graph

145 Commits

Author SHA1 Message Date
Martin Atkins 67d95b97ce lang/funcs: templatefile requires valid variable names
Previously the templatefile function would permit any arbitrary string as
a variable name, but due to the HCL template syntax it would be impossible
to refer to one that isn't a valid HCL identifier without causing an
HCL syntax error.

The HCL syntax errors are correct, but don't really point to the root
cause of the problem. Instead, we'll pre-verify that the variable names
are valid before we even try to render the template, and given a
specialized error message that refers to the vars argument expression as
the problematic part, which will hopefully make the resolution path
clearer for a user encountering this situation.

The syntax error still remains for situations where all of the variable
names are correct but e.g. the user made a typo referring to one, which
makes sense because in that case the problem _is_ inside the template.
2020-02-25 10:19:46 -05:00
Martin Atkins ec9f950b3f lang/funcs: Test actual error messages from templatefile
This function has a number of different error cases with hopefully-helpful
error messages for each, so it's good to test we're getting the error
message we were actually expecting in each case.
2020-02-25 10:19:46 -05:00
James Bardin a765d69fb0
Merge pull request #24032 from hashicorp/jbardin/map-funcs
make the merge function more precise
2020-02-12 10:51:05 -05:00
James Goodhouse 25bfe7337b
lang: add setsubtract function (#23424)
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib

* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
2020-02-06 12:49:11 -05:00
James Bardin 529271e0be update merge docs to match behavior 2020-02-05 15:47:36 -05:00
James Bardin f5bf9aa55d make the merge function more precise
This PR implements 2 changes to the merge function.
 - Rather than always defining the merge return type as dynamic, return
 a precise type when all argument types match, or all possible object
 attributes are known.
 - Always return a value containing all keys when the keys are known.
 This allows the use of merge output in for_each, even when keys are yet
 to be determined.
2020-02-05 13:46:08 -05:00
Martin Atkins 02576988c1 lang: "try" and "can" functions
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.

For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:

    try(local.example.foo, "fallback-foo")

Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
2020-01-10 15:23:25 -08:00
George Christou 23fc68cc91
lang: Fix new `trim*` function mappings
Fixes #23413
2019-11-19 10:13:14 +00:00
George Christou 91100c003c lang/funcs: Add more `trim*` functions (#23016)
* lang/funcs: Add `trim*` functions
2019-11-18 08:31:44 -05:00
Parviz 6eaf1c23f9 lang/funcs: don't panic when transpose produces empty result 2019-11-08 09:40:39 -08:00
James Bardin 3839405d3d correctly evaluate self in for_each resources
The fallback type for GetResource from an EachMap is a cty.Object,
because resource schemas may contain dynamically typed attributes.
Check for an Object type in the evaluation of self, to use the proper
GetAttr method when extracting the value.
2019-10-29 12:44:42 -04:00
James Bardin 1ee851f256
Merge pull request #22846 from hashicorp/jbardin/evaluate-resource
Always evaluate resources in their entirety
2019-10-08 07:57:15 -04:00
James Bardin bfce78064b lang/eval: more evalContext fixups
self references do not need to be added to `managedResource`, and in
fact that could cause issues later if self is allowed in contexts other
than managed resources.

Coalesce 2 cases in the Referenceable switch, be take the
ContainingResource address of an instance beforehand.
2019-10-07 18:13:20 -04:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
Martin Atkins f84ab99b7d lang/funcs: cidrsubnets function
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.

Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
2019-09-20 15:58:01 -07:00
James Bardin d4031918d1 parse resource refs correctly
Now that we only evaluate whole resources, we can parse resource refs
correct as the resource, rather than an unknown instance.
2019-09-19 11:46:09 -04:00
James Bardin c49f976308 change lang eval to also only lookup resources
Continue only evaluating resource at a whole and push the indexing of
the resource down into the expression evaluation.

The exception here is that `self` must be an instance which must be
extracted from the resource. We now also add the entire resource to the
context, which was previously only partially populated with the self
referenced instance.
2019-09-19 11:46:09 -04:00
James Bardin 86bf674246 change GetResourceInstance to GetResource
In order to allow lazy evaluation of resource indexes, we can't index
resources immediately via GetResourceInstance. Change the evaluation to
always return whole Resources via GetResource, and index individual
instances during expression evaluation.

This will allow us to always check for invalid index errors rather than
returning an unknown value and ignoring it during apply.
2019-09-19 09:19:14 -04:00
Jeet Parekh bcc69c05bb lang/funcs: parseint function 2019-09-17 15:33:22 -07:00
Pam Selle f9ebae749c
Merge pull request #22707 from vsimon/spell
docs: Minor spelling and typo fixes
2019-09-06 11:46:01 -04:00
Vicken Simonian 853a0e0677 docs: Minor spelling and typo fixes 2019-09-05 10:08:34 -07:00
Brian Flad 19cf34114f
lang/funcs: Switch fileset() function glob implementation to github.com/bmatcuk/doublestar to support additional glob patterns
This allows the usage of the glob patterns `**` and `{alternative1,...}` to simplify Terraform configuration logic for more complex file matching.
2019-08-30 20:22:03 -04:00
Brian Flad 769f626a0e
lang/funcs: Remove homedir.Expand() and refactor path trimming with filepath.Rel() in fileset() function
Reference: https://github.com/hashicorp/terraform/pull/22621#pullrequestreview-282259385
2019-08-30 20:19:44 -04:00
Brian Flad 820b79d91d
lang/funcs: Clarification update to MakeFileSetFunc comment for filepath.ToSlash usage 2019-08-30 20:19:44 -04:00
Brian Flad af7f6ef441
lang/funcs: Update fileset() function to include path as separate first argument, automatically trim the path argument from results, and ensure results are always canonical with forward slash path separators
Reference: https://github.com/hashicorp/terraform/pull/22523#pullrequestreview-279694703

These changes center around better function usability and consistency with other functions. The function has not yet been released, so these breaking changes can be applied safely.
2019-08-30 20:19:44 -04:00
Brian Flad d48d9ed766
lang/funcs: Add fileset function
Reference: https://github.com/hashicorp/terraform/issues/16697

Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862

To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
2019-08-20 04:50:01 -04:00
Pam Selle e38d588cfc
Merge pull request #22314 from hashicorp/pselle/double-dynamic
Fix panic on double dynamic blocks
2019-08-16 15:15:50 -04:00
Alex Pilon e2eb70b737
remove satori/go.uuid in favor of google/uuid 2019-08-09 15:09:41 -04:00
Martin Atkins 135afaeb9c lang: "regex" and "regexall" functions
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
2019-08-06 11:52:14 -07:00
Pam Selle 234c1c482a Add a comment 2019-08-02 16:15:18 -04:00
Pam Selle cd83e6108d Fixes issue where attribute in nested is not of list type/is invalid and would panic 2019-08-02 16:08:16 -04:00
Kristin Laemmert 2b14a6b332
lang/funcs: lookup() can work with maps of lists, maps and objects (#22269)
* lang/funcs: lookup() can work with maps of lists, maps and objects

lookup() can already handle aribtrary objects of (whatever) and should
handle maps of (whatever) similarly.
2019-08-01 15:52:52 -04:00
Pam Selle 360068b3cb
Merge pull request #21922 from pselle/resource_for_each
Resource for_each
2019-07-26 11:41:56 -04:00
James Bardin 6e222375c7 invlaid use of dynamic with attrs should not panic
Mistakenly using dynamic on an attribute will lead to a panic when
attempting to resolve variable references with a partial body, because
the dynamic blocks have yet to be expanded and validated. Check that the
block element type is actually an object before generating a schema.
2019-07-22 17:16:18 -04:00
Pam Selle 7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Radek Simko 5c30bafee0
Merge pull request #22119 from hashicorp/b-chunklist-empty-list
lang/funcs: Pass through empty list in chunklist
2019-07-18 20:59:47 +01:00
Radek Simko 736fda1613
lang/funcs: Pass through empty list in chunklist 2019-07-18 15:01:00 +01:00
Saverio Miroddi 983af7a183 Allow null values in Compact function
The function would previously panic when one or more null values were among the arguments.

The new behavior treats nulls as empty strings, therefore, it removes them.
2019-07-11 20:16:24 +02:00
Andreas Sommer 042aead714 lang/funcs: add "abspath" function (#21409) 2019-07-02 08:30:30 -04:00
Paul Tyng ebea080aca Add missing bool case
Fixes #20572
2019-06-24 15:27:59 -04:00
Pam Selle 1255ad6722 Don't allow nulls in calls to merge 2019-06-14 15:26:24 -04:00
Lars Eric Scheidler aa07806bfc lang/funcs: New "uuidv5" function
This generates name-based uuids, rather than pseudorandom uuids as with the
"uuid" function.
2019-06-07 14:38:22 -07:00
Martin Atkins 382e1ca821 lang: yamldecode and yamlencode functions
These follow the same principle as jsondecode and jsonencode, but use
YAML instead of JSON.

YAML has a much more complex information model than JSON, so we can only
support a subset of it during decoding, but hopefully the subset supported
here is a useful one.

Because there are many different ways to _generate_ YAML, the yamlencode
function is forced to make some decisions, and those decisions are likely
to affect compatibility with other real-world YAML parsers. Although the
format here is intended to be generic and compatible, we may find that
there are problems with it that'll we'll want to adjust for in a future
release, so yamlencode is therefore marked as experimental for now until
the underlying library is ready to commit to ongoing byte-for-byte
compatibility in serialization.

The main use-case here is met by yamldecode, which will allow reading in
files written in YAML format by humans for use in Terraform modules, in
situations where a higher-level input format than direct Terraform
language declarations is helpful.
2019-06-04 16:24:09 -07:00
Martin Atkins f9a73d48db lang: "range" function
This is similar to the function of the same name in Python, generating a
sequence of numbers as a list that can then be used in other
sequence-oriented operations.

The primary use-case for it is to turn a count expressed as a number into
a list of that length, which can then be iterated over or passed to a
collection function to produce that number of something else, as shown
in the example at the end of its documentation page.
2019-06-04 16:20:17 -07:00
Kristin Laemmert c163ba71db simplify 2019-06-04 11:54:26 -04:00
Kristin Laemmert 30a924e162 lang/funcs: `matchkeys` - unify type for keys and searchset
Added higher-level test for matchkeys to exercise mixing
types in searchset. This had to be in the functions tests so the HCL
auto conversion from tuple to list would occur.
2019-06-04 08:57:02 -04:00
Kristin Laemmert f2a14d7c18 lang/funcs: fix error when `matchkeys` encountered a variable
`matchkeys` was returning a (false) error if the searchset was a
variable, since then the type of the keylist and searchset parameters
would not match.

This does slightly change the behavior: previously matchkeys would
produce an error if the parameters were not of the same type, for e.g.
if searchset was a list of strings and keylist was a list of integers.
  This no longer produces an error.
2019-06-03 18:00:53 -04:00
James Bardin 0133dda311 allow empty dynamic blocks with ConfigModeAttr
If a dynamic block is evaluated zero times, the body content will
contain 0 blocks. Allow the probe for ConfigModeAttr to accept that no
blocks with a matching attribute should still be converted to a block if
they are called with dynamicExpand.
2019-05-31 16:31:24 -05:00
Martin Atkins 55794eb658 lang/funcs: Don't panic when distinct called with empty list 2019-05-31 11:44:31 -05:00
Martin Atkins 742deca3e9 lang/funcs: Short-circuit if start or end index is unknown
Previously the type-selection codepath for an input tuple referred
unconditionally to the start and end index values. In the Type callback,
only the types of the arguments are guaranteed to be known, so any access
to the values must be guarded with an .IsKnown check, or else the function
will not short-circuit properly when an unknown value is passed.

Now we will check the start and end indices are in range when we have
enough information to do so, but we'll return an approximate result if
either is unknown.
2019-05-17 08:48:02 -07:00