lang: "try" and "can" functions

These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.

For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:

    try(local.example.foo, "fallback-foo")

Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.

lang/functions.go

@@ -3,6 +3,7 @@ package lang
 import (
 	"fmt"
 
+	"github.com/hashicorp/hcl/v2/ext/tryfunc"
 	ctyyaml "github.com/zclconf/go-cty-yaml"
 	"github.com/zclconf/go-cty/cty"
 	"github.com/zclconf/go-cty/cty/function"
@@ -39,6 +40,7 @@ func (s *Scope) Functions() map[string]function.Function {
 			"base64sha256":     funcs.Base64Sha256Func,
 			"base64sha512":     funcs.Base64Sha512Func,
 			"bcrypt":           funcs.BcryptFunc,
+			"can":              tryfunc.CanFunc,
 			"ceil":             funcs.CeilFunc,
 			"chomp":            funcs.ChompFunc,
 			"cidrhost":         funcs.CidrHostFunc,
@@ -122,6 +124,7 @@ func (s *Scope) Functions() map[string]function.Function {
 			"trimprefix":       funcs.TrimPrefixFunc,
 			"trimspace":        funcs.TrimSpaceFunc,
 			"trimsuffix":       funcs.TrimSuffixFunc,
+			"try":              tryfunc.TryFunc,
 			"upper":            stdlib.UpperFunc,
 			"urlencode":        funcs.URLEncodeFunc,
 			"uuid":             funcs.UUIDFunc,

lang/functions_test.go

@@ -109,6 +109,27 @@ func TestFunctions(t *testing.T) {
 			},
 		},
 
+		"can": {
+			{
+				`can(true)`,
+				cty.True,
+			},
+			{
+				// Note: "can" only works with expressions that pass static
+				// validation, because it only gets an opportunity to run in
+				// that case. The following "works" (captures the error) because
+				// Terraform understands it as a reference to an attribute
+				// that does not exist during dynamic evaluation.
+				//
+				// "can" doesn't work with references that could never possibly
+				// be valid and are thus caught during static validation, such
+				// as an expression like "foo" alone which would be understood
+				// as an invalid resource reference.
+				`can({}.baz)`,
+				cty.False,
+			},
+		},
+
 		"ceil": {
 			{
 				`ceil(1.2)`,
@@ -865,6 +886,30 @@ func TestFunctions(t *testing.T) {
 			},
 		},
 
+		"try": {
+			{
+				// Note: "try" only works with expressions that pass static
+				// validation, because it only gets an opportunity to run in
+				// that case. The following "works" (captures the error) because
+				// Terraform understands it as a reference to an attribute
+				// that does not exist during dynamic evaluation.
+				//
+				// "try" doesn't work with references that could never possibly
+				// be valid and are thus caught during static validation, such
+				// as an expression like "foo" alone which would be understood
+				// as an invalid resource reference. That's okay because this
+				// function exists primarily to ease access to dynamically-typed
+				// structures that Terraform can't statically validate by
+				// definition.
+				`try({}.baz, "fallback")`,
+				cty.StringVal("fallback"),
+			},
+			{
+				`try("fallback")`,
+				cty.StringVal("fallback"),
+			},
+		},
+
 		"upper": {
 			{
 				`upper("hello")`,

vendor/github.com/hashicorp/hcl/v2/ext/tryfunc/README.md

@@ -0,0 +1,44 @@
+# "Try" and "can" functions
+
+This Go package contains two `cty` functions intended for use in an
+`hcl.EvalContext` when evaluating HCL native syntax expressions.
+
+The first function `try` attempts to evaluate each of its argument expressions
+in order until one produces a result without any errors.
+
+```hcl
+try(non_existent_variable, 2) # returns 2
+```
+
+If none of the expressions succeed, the function call fails with all of the
+errors it encountered.
+
+The second function `can` is similar except that it ignores the result of
+the given expression altogether and simply returns `true` if the expression
+produced a successful result or `false` if it produced errors.
+
+Both of these are primarily intended for working with deep data structures
+which might not have a dependable shape. For example, we can use `try` to
+attempt to fetch a value from deep inside a data structure but produce a
+default value if any step of the traversal fails:
+
+```hcl
+result = try(foo.deep[0].lots.of["traversals"], null)
+```
+
+The final result to `try` should generally be some sort of constant value that
+will always evaluate successfully.
+
+## Using these functions
+
+Languages built on HCL can make `try` and `can` available to user code by
+exporting them in the `hcl.EvalContext` used for expression evaluation:
+
+```go
+ctx := &hcl.EvalContext{
+    Functions: map[string]function.Function{
+        "try": tryfunc.TryFunc,
+        "can": tryfunc.CanFunc,
+    },
+}
+```

vendor/github.com/hashicorp/hcl/v2/ext/tryfunc/tryfunc.go

@@ -0,0 +1,150 @@
+// Package tryfunc contains some optional functions that can be exposed in
+// HCL-based languages to allow authors to test whether a particular expression
+// can succeed and take dynamic action based on that result.
+//
+// These functions are implemented in terms of the customdecode extension from
+// the sibling directory "customdecode", and so they are only useful when
+// used within an HCL EvalContext. Other systems using cty functions are
+// unlikely to support the HCL-specific "customdecode" extension.
+package tryfunc
+
+import (
+	"errors"
+	"fmt"
+	"strings"
+
+	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/customdecode"
+	"github.com/zclconf/go-cty/cty"
+	"github.com/zclconf/go-cty/cty/function"
+)
+
+// TryFunc is a variadic function that tries to evaluate all of is arguments
+// in sequence until one succeeds, in which case it returns that result, or
+// returns an error if none of them succeed.
+var TryFunc function.Function
+
+// CanFunc tries to evaluate the expression given in its first argument.
+var CanFunc function.Function
+
+func init() {
+	TryFunc = function.New(&function.Spec{
+		VarParam: &function.Parameter{
+			Name: "expressions",
+			Type: customdecode.ExpressionClosureType,
+		},
+		Type: func(args []cty.Value) (cty.Type, error) {
+			v, err := try(args)
+			if err != nil {
+				return cty.NilType, err
+			}
+			return v.Type(), nil
+		},
+		Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
+			return try(args)
+		},
+	})
+	CanFunc = function.New(&function.Spec{
+		Params: []function.Parameter{
+			{
+				Name: "expression",
+				Type: customdecode.ExpressionClosureType,
+			},
+		},
+		Type: function.StaticReturnType(cty.Bool),
+		Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
+			return can(args[0])
+		},
+	})
+}
+
+func try(args []cty.Value) (cty.Value, error) {
+	if len(args) == 0 {
+		return cty.NilVal, errors.New("at least one argument is required")
+	}
+
+	// We'll collect up all of the diagnostics we encounter along the way
+	// and report them all if none of the expressions succeed, so that the
+	// user might get some hints on how to make at least one succeed.
+	var diags hcl.Diagnostics
+	for _, arg := range args {
+		closure := customdecode.ExpressionClosureFromVal(arg)
+		if dependsOnUnknowns(closure.Expression, closure.EvalContext) {
+			// We can't safely decide if this expression will succeed yet,
+			// and so our entire result must be unknown until we have
+			// more information.
+			return cty.DynamicVal, nil
+		}
+
+		v, moreDiags := closure.Value()
+		diags = append(diags, moreDiags...)
+		if moreDiags.HasErrors() {
+			continue // try the next one, if there is one to try
+		}
+		return v, nil // ignore any accumulated diagnostics if one succeeds
+	}
+
+	// If we fall out here then none of the expressions succeeded, and so
+	// we must have at least one diagnostic and we'll return all of them
+	// so that the user can see the errors related to whichever one they
+	// were expecting to have succeeded in this case.
+	//
+	// Because our function must return a single error value rather than
+	// diagnostics, we'll construct a suitable error message string
+	// that will make sense in the context of the function call failure
+	// diagnostic HCL will eventually wrap this in.
+	var buf strings.Builder
+	buf.WriteString("no expression succeeded:\n")
+	for _, diag := range diags {
+		if diag.Subject != nil {
+			buf.WriteString(fmt.Sprintf("- %s (at %s)\n  %s\n", diag.Summary, diag.Subject, diag.Detail))
+		} else {
+			buf.WriteString(fmt.Sprintf("- %s\n  %s\n", diag.Summary, diag.Detail))
+		}
+	}
+	buf.WriteString("\nAt least one expression must produce a successful result")
+	return cty.NilVal, errors.New(buf.String())
+}
+
+func can(arg cty.Value) (cty.Value, error) {
+	closure := customdecode.ExpressionClosureFromVal(arg)
+	if dependsOnUnknowns(closure.Expression, closure.EvalContext) {
+		// Can't decide yet, then.
+		return cty.UnknownVal(cty.Bool), nil
+	}
+
+	_, diags := closure.Value()
+	if diags.HasErrors() {
+		return cty.False, nil
+	}
+	return cty.True, nil
+}
+
+// dependsOnUnknowns returns true if any of the variables that the given
+// expression might access are unknown values or contain unknown values.
+//
+// This is a conservative result that prefers to return true if there's any
+// chance that the expression might derive from an unknown value during its
+// evaluation; it is likely to produce false-positives for more complex
+// expressions involving deep data structures.
+func dependsOnUnknowns(expr hcl.Expression, ctx *hcl.EvalContext) bool {
+	for _, traversal := range expr.Variables() {
+		val, diags := traversal.TraverseAbs(ctx)
+		if diags.HasErrors() {
+			// If the traversal returned a definitive error then it must
+			// not traverse through any unknowns.
+			continue
+		}
+		if !val.IsWhollyKnown() {
+			// The value will be unknown if either it refers directly to
+			// an unknown value or if the traversal moves through an unknown
+			// collection. We're using IsWhollyKnown, so this also catches
+			// situations where the traversal refers to a compound data
+			// structure that contains any unknown values. That's important,
+			// because during evaluation the expression might evaluate more
+			// deeply into this structure and encounter the unknowns.
+			return true
+		}
+	}
+	return false
+}

vendor/modules.txt

@@ -360,6 +360,7 @@ github.com/hashicorp/hcl/v2/hclparse
 github.com/hashicorp/hcl/v2/gohcl
 github.com/hashicorp/hcl/v2/ext/typeexpr
 github.com/hashicorp/hcl/v2/ext/dynblock
+github.com/hashicorp/hcl/v2/ext/tryfunc
 github.com/hashicorp/hcl/v2/ext/customdecode
 github.com/hashicorp/hcl/v2/hcltest
 # github.com/hashicorp/hil v0.0.0-20190212112733-ab17b08d6590

website/docs/configuration/functions/can.html.md

@@ -0,0 +1,67 @@
+---
+layout: "functions"
+page_title: "can - Functions - Configuration Language"
+sidebar_current: "docs-funcs-conversion-can"
+description: |-
+  The can function tries to evaluate an expression given as an argument and
+  indicates whether the evaluation succeeded.
+---
+
+# `can` Function
+
+-> **Note:** This page is about Terraform 0.12 and later. For Terraform 0.11 and
+earlier, see
+[0.11 Configuration Language: Interpolation Syntax](../../configuration-0-11/interpolation.html).
+
+`can` evaluates the given expression and returns a boolean value indicating
+whether the expression produced a result without any errors.
+
+This is a special function that is able to catch errors produced when evaluating
+its argument. This function should be used with care, considering many of the
+same caveats that apply to [`try`](./try.html), to avoid writing configurations
+that are hard to read and maintain.
+
+For most situations it's better to use [`try`](./try.html), because it allows
+for more concise definition of fallback values for failing expressions.
+
+The `can` function can only catch and handle _dynamic_ errors resulting from
+access to data that isn't known until runtime. It will not catch errors
+relating to expressions that can be proven to be invalid for any input, such
+as a malformed resource reference.
+
+~> **Warning:** The `can` function is intended only for concise testing of the
+presence of and types of object attributes. Although it can technically accept
+any sort of expression, we recommend using it only with simple attribute
+references and type conversion functions as shown in the [`try`](./try.html)
+examples. Overuse of `can` to suppress errors will lead to a configuration that
+is hard to understand and maintain.
+
+## Examples
+
+```
+> local.foo
+{
+  "bar" = "baz"
+}
+> can(local.foo.bar)
+true
+> can(local.foo.boop)
+false
+```
+
+The `can` function will _not_ catch errors relating to constructs that are
+provably invalid even before dynamic expression evaluation, such as a malformed
+reference or a reference to a top-level object that has not been declared:
+
+```
+> can(local.nonexist)
+
+Error: Reference to undeclared local value
+
+A local value with the name "nonexist" has not been declared.
+```
+
+## Related Functions
+
+* [`try`](./try.html), which tries evaluating a sequence of expressions and
+  returns the result of the first one that succeeds.

website/docs/configuration/functions/try.html.md

@@ -0,0 +1,117 @@
+---
+layout: "functions"
+page_title: "try - Functions - Configuration Language"
+sidebar_current: "docs-funcs-conversion-try"
+description: |-
+  The try function tries to evaluate a sequence of expressions given as
+  arguments and returns the result of the first one that does not produce
+  any errors.
+---
+
+# `try` Function
+
+-> **Note:** This page is about Terraform 0.12 and later. For Terraform 0.11 and
+earlier, see
+[0.11 Configuration Language: Interpolation Syntax](../../configuration-0-11/interpolation.html).
+
+`try` evaluates all of its argument expressions in turn and returns the result
+of the first one that does not produce any errors.
+
+This is a special function that is able to catch errors produced when evaluating
+its arguments, which is particularly useful when working with complex data
+structures whose shape is not well-known at implementation time.
+
+For example, if some data is retrieved from an external system in JSON or YAML
+format and then decoded, the result may have attributes that are not guaranteed
+to be set. We can use `try` to produce a normalized data structure which has
+a predictable type that can therefore be used more conveniently elsewhere in
+the configuration:
+
+```hcl
+locals {
+  raw_value = yamldecode("${path.module}/example.yaml")
+  normalized_value = {
+    name   = tostring(try(local.raw_value.name, null))
+    groups = try(local.raw_value.groups, [])
+  }
+}
+```
+
+With the above local value expressions, configuration elsewhere in the module
+can refer to `local.normalized_value` attributes without the need to repeatedly
+check for and handle absent attributes that would otherwise produce errors.
+
+We can also use `try` to deal with situations where a value might be provided
+in two different forms, allowing us to normalize to the most general form:
+
+```hcl
+variable "example" {
+  type = any
+}
+
+locals {
+  example = try(
+    [tostring(var.example)],
+    tolist(var.example),
+  )
+}
+```
+
+The above permits `var.example` to be either a list or a single string. If it's
+a single string then it'll be normalized to a single-element list containing
+that string, again allowing expressions elsewhere in the configuration to just
+assume that `local.example` is always a list.
+
+This second example contains two expressions that can both potentially fail.
+For example, if `var.example` were set to `{}` then it could be converted to
+neither a string nor a list. If `try` exhausts all of the given expressions
+without any succeeding, it will return an error describing all of the problems
+it encountered.
+
+We strongly suggest using `try` only in special local values whose expressions
+perform normalization, so that the error handling is confined to a single
+location in the module and the rest of the module can just use straightforward
+references to the normalized structure and thus be more readable for future
+maintainers.
+
+The `try` function can only catch and handle _dynamic_ errors resulting from
+access to data that isn't known until runtime. It will not catch errors
+relating to expressions that can be proven to be invalid for any input, such
+as a malformed resource reference.
+
+~> **Warning:** The `try` function is intended only for concise testing of the
+presence of and types of object attributes. Although it can technically accept
+any sort of expression, we recommend using it only with simple attribute
+references and type conversion functions as shown in the examples above.
+Overuse of `try` to suppress errors will lead to a configuration that is hard
+to understand and maintain.
+
+## Examples
+
+```
+> local.foo
+{
+  "bar" = "baz"
+}
+> try(local.foo.bar, "fallback")
+baz
+> try(local.foo.boop, "fallback")
+fallback
+```
+
+The `try` function will _not_ catch errors relating to constructs that are
+provably invalid even before dynamic expression evaluation, such as a malformed
+reference or a reference to a top-level object that has not been declared:
+
+```
+> try(local.nonexist, "fallback")
+
+Error: Reference to undeclared local value
+
+A local value with the name "nonexist" has not been declared.
+```
+
+## Related Functions
+
+* [`can`](./can.html), which tries evaluating an expression and returns a
+  boolean value indicating whether it succeeded.

website/layouts/functions.erb

@@ -448,6 +448,10 @@
             <a href="#docs-funcs-conversion">Type Conversion Functions</a>
             <ul class="nav" id="docs-funcs-conversion">
 
+              <li>
+                <a href="/docs/configuration/functions/can.html">can</a>
+              </li>
+
               <li>
                 <a href="/docs/configuration/functions/tobool.html">tobool</a>
               </li>
@@ -472,6 +476,10 @@
                 <a href="/docs/configuration/functions/tostring.html">tostring</a>
               </li>
 
+              <li>
+                <a href="/docs/configuration/functions/try.html">try</a>
+              </li>
+
             </ul>
           </li>