website: Full examples for for_each with flatten and setproduct

A very common question since we launched the two repetition constructs
is how to deal with situations where the input data structure doesn't
match one-to-one with the desired configuration.

This adds some full worked examples of two common situations that have
come up in questions. To avoid adding a lot of extra content to the
already-large "expressions" and "resources" pages, the main bulk of this
new content lives with the relevant functions themselves as a full example
of one thing they are good for, and then we'll link to them from the two
general documentation sections where folks are likely to be reading when
they encounter the problem.
This commit is contained in:
Martin Atkins 2019-10-11 12:02:24 -07:00
parent 0d27e8ca7d
commit 047733d20c
4 changed files with 201 additions and 3 deletions

View File

@ -722,9 +722,15 @@ to generate meta-argument blocks such as `lifecycle` and `provisioner`
blocks, since Terraform must process these before it is safe to evaluate
expressions.
If you need to iterate over combinations of values from multiple collections,
use [`setproduct`](./functions/setproduct.html) to create a single collection
containing all of the combinations.
The `for_each` value must be a map or set with one element per desired
nested block. If you need to declare resource instances based on a nested
data structure or combinations of elements from multiple data structures you
can use Terraform expressions and functions to derive a suitable value.
For some common examples of such situations, see the
[`flatten`](/docs/configuration/functions/flatten.html)
and
[`setproduct`](/docs/configuration/functions/setproduct.html)
functions.
### Best Practices for `dynamic` Blocks

View File

@ -31,3 +31,85 @@ flattened recursively:
```
Indirectly-nested lists, such as those in maps, are _not_ flattened.
## Flattening nested structures for `for_each`
The
[resource `for_each`](/docs/configuration/resources.html#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings)
and
[`dynamic` block](/docs/configuration/expressions.html#dynamic-blocks)
language features both require a collection value that has one element for
each repetition.
Sometimes your input data structure isn't naturally in a suitable shape for
use in a `for_each` argument, and `flatten` can be a useful helper function
when reducing a nested data structure into a flat one.
For example, consider a module that declares a variable like the following:
```hcl
variable "networks" {
type = map(object({
cidr_block = string
subnets = map(object({
cidr_block = string
})
})
}
```
The above is a reasonable way to model objects that naturally form a tree,
such as top-level networks and their subnets. The repetition for the top-level
networks can use this variable directly, because it's already in a form
where the resulting instances match one-to-one with map elements:
```hcl
resource "aws_vpc" "example" {
for_each = var.networks
cidr_block = each.value.cidr_block
}
```
However, in order to declare all of the _subnets_ with a single `resource`
block, we must first flatten the structure to produce a collection where each
top-level element represents a single subnet:
```hcl
locals {
# flatten ensures that this local value is a flat list of objects, rather
# than a list of lists of objects.
network_subnets = flatten([
for network_key, network in var.networks : [
for subnet_key, subnet in network.subnets : {
network_key = network_key
subnet_key = subnet_key
network_id = aws_vpc.example[network_key].id
cidr_block = subnet.cidr_block
}
]
])
}
resource "aws_subnet" "example" {
# local.network_subnets is a list, so we must now project it into a map
# where each key is unique. We'll combine the network and subnet keys to
# produce a single unique key per instance.
for_each = {
for subnet in local.network_subnets : "${subnet.network_key}.${subnet.subnet_key}" => subnet
}
vpc_id = each.value.network_id
availability_zone = each.value.subnet_key
cidr_block = each.value_cidr_block
}
```
The above results in one subnet instance per subnet object, while retaining
the associations between the subnets and their containing networks.
## Related Functions
* [`setproduct`](./setproduct.html) finds all of the combinations of multiple
lists or sets of values, which can also be useful when preparing collections
for use with `for_each` constructs.

View File

@ -118,10 +118,110 @@ elements all have a consistent type:
]
```
## Finding combinations for `for_each`
The
[resource `for_each`](/docs/configuration/resources.html#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings)
and
[`dynamic` block](/docs/configuration/expressions.html#dynamic-blocks)
language features both require a collection value that has one element for
each repetition.
Sometimes your input data comes in separate values that cannot be directly
used in a `for_each` argument, and `setproduct` can be a useful helper function
for the situation where you want to find all unique combinations of elements in
a number of different collections.
For example, consider a module that declares variables like the following:
```hcl
variable "networks" {
type = map(object({
base_cidr_block = string
}))
}
variable "subnets" {
type = map(object({
number = number
}))
}
```
If the goal is to create each of the defined subnets per each of the defined
networks, creating the top-level networks can directly use `var.networks`
because it's already in a form where the resulting instances match one-to-one
with map elements:
```hcl
resource "aws_vpc" "example" {
for_each = var.networks
cidr_block = each.value.base_cidr_block
}
```
However, in order to declare all of the _subnets_ with a single `resource`
block, we must first produce a collection whose elements represent all of
the combinations of networks and subnets, so that each element itself
represents a subnet:
```hcl
locals {
# setproduct works with sets and lists, but our variables are both maps
# so we'll need to convert them first.
networks = [
for key, network in var.networks : {
key = key
cidr_block = network.cidr_block
}
]
subnets = [
for key, subnet in var.subnets : {
key = key
number = subnet.number
}
]
network_subnets = [
# in pair, element zero is a network and element one is a subnet,
# in all unique combinations.
for pair in setproduct(local.networks, local.subnets) : {
network_key = pair[0].key
subnet_key = pair[1].key
network_id = aws_vpc.example[pair[0].key].id
# The cidr_block is derived from the corresponding network. See the
# cidrsubnet function for more information on how this calculation works.
cidr_block = cidrsubnet(pair[0].cidr_block, 4, pair[1].number)
}
]
}
resource "aws_subnet" "example" {
# local.network_subnets is a list, so we must now project it into a map
# where each key is unique. We'll combine the network and subnet keys to
# produce a single unique key per instance.
for_each = {
for subnet in local.network_subnets : "${subnet.network_key}.${subnet.subnet_key}" => subnet
}
vpc_id = each.value.network_id
availability_zone = each.value.subnet_key
cidr_block = each.value_cidr_block
}
```
The above results in one subnet instance per combination of network and subnet
elements in the input variables.
## Related Functions
* [`contains`](./contains.html) tests whether a given list or set contains
a given element value.
* [`flatten`](./flatten.html) is useful for flattening heirarchical data
into a single list, for situations where the relationships between two
object types are defined explicitly.
* [`setintersection`](./setintersection.html) computes the _intersection_ of
multiple sets.
* [`setunion`](./setunion.html) computes the _union_ of multiple

View File

@ -411,6 +411,16 @@ can't refer to any resource attributes that aren't known until after a
configuration is applied (such as a unique ID generated by the remote API when
an object is created).
The `for_each` value must be a map or set with one element per desired
resource instance. If you need to declare resource instances based on a nested
data structure or combinations of elements from multiple data structures you
can use Terraform expressions and functions to derive a suitable value.
For some common examples of such situations, see the
[`flatten`](/docs/configuration/functions/flatten.html)
and
[`setproduct`](/docs/configuration/functions/setproduct.html)
functions.
### `provider`: Selecting a Non-default Provider Configuration
[inpage-provider]: #provider-selecting-a-non-default-provider-configuration