A very common question since we launched the two repetition constructs
is how to deal with situations where the input data structure doesn't
match one-to-one with the desired configuration.
This adds some full worked examples of two common situations that have
come up in questions. To avoid adding a lot of extra content to the
already-large "expressions" and "resources" pages, the main bulk of this
new content lives with the relevant functions themselves as a full example
of one thing they are good for, and then we'll link to them from the two
general documentation sections where folks are likely to be reading when
they encounter the problem.
* upstream/master: (66 commits)
lang/eval: more evalContext fixups
Update CHANGELOG.md
Cleanup after v0.12.10 release
v0.12.10
website / help: reconcile 'validate' command docs
website: Document behavior of `self` object for provisioners
vendor: switch to HCL 2.0 in the HCL repository
Update communicator/ssh/communicator.go
copy client pointer for keep-alive loop
Update CHANGELOG.md
slow down tfce polling to 1s
typos. some code, some text.
Remove -check-variables flag from the docs
Merge cleanup, remove `license` parameter in favor of bool `accept_license`, adjust how license acceptance is done, update hab provisioner doc.
vendor latest go-tfe
clean up go mod for go-tfe
tfce test additions
update to go-tfe 0.3.23
cost estimation status polling
go-tfe dep update to 0.3.22
...
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
The cidrsubnets function signature is intentionally very low-level and
focused on the core requirement of generating addresses. This registry
module then wraps it with some additional functionality to make it more
convenient to generate and use subnet address ranges.
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.
Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
* command/import: properly use `-provider` supplied on the command line
The import command now attaches the provider configuration in the resource
instance, if set. That config is attached to the NodeAbstractResource
during the import graph building. This prevents errors when the implied
provider is not actually in the configuration at all, which may happen
when a configuration is using the `-beta` version of a provider (and
only that `-beta` version).
* command/import: fix variable reassignment and update docs
Fixes#22564
For a long time now we've been advising against the use of provisioners,
but our documentation for them is pretty prominent on the website in
comparision to the better alternatives, and so it's little surprise that
many users end up making significant use of them.
Although in the longer term a change to our information architecture would
probably address this even better, this is an attempt to be explicit about
the downsides of using provisioners and to prominently describe the
alternatives that are available for common use-cases, along with some
reasons why we consider them to be better.
I took the unusual step here of directly linking to specific provider
documentation pages about the alternatives, even though we normally try
to keep the core documentation provider-agnostic, because otherwise that
information tends to be rather buried in the provider documentation and
thus the reader would be reasonable to use provisioners just because we're
not giving specific enough alternative recommendations.
* website/formatdate: update example
The given example was showing HOUR:MONTH instead of HOUR:MINUTE
Fixes#22598
* website/import: remove reference to no-longer-working option
Users can no longer supply `-config=""` to tell Terraform not to load
configuration for import.
Fixes#22294
* website/provisioners: `host` is required in connection blocks
Fixes#21877
* website/variables: clarify variable definition precedence
It was not entirely obvious that a variable could not be assigned
multiples times in a single source.
Fixes#21682
* website/backend/local: add `workspace_dir` attribute
Fixes#21391
* website/output: `sensitive` outputs are redacted in output
Fixes#21502
* website/backends: sidebar order tweak
It makes sense for backend 'configuration' to appear before 'init'.
Fixes#13796
* Revert "website/formatdate: update example"
This reverts commit ccd93c86ddd15a21625c0767702ee1cc62e77254.
Reference: https://github.com/hashicorp/terraform/issues/16697
Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862
To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
The Terraform Enterprise brand has now been split into two parts:
- Terraform Cloud is the application that helps teams use Terraform together,
with remote state storage, a shared run environment, etc.
- Terraform Enterprise is the on-premise distribution that lets enterprises run
a private instance of the Terraform Cloud application.
The former TFE docs have been split accordingly.
- Make these descriptions more similar, since they do basically the same thing.
- Add some subheaders to break up the wall of text and make it more skimmable.
- Nudge people more firmly toward `for_each` if they need to actually
incorporate data from a variable into their instances.
- Add version note so you know whether you can use this yet.
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
We added the csvdecode function originally with the intent of it being
used with for_each, but because csvdecode was released first we had a
section in its documentation warning about the downsides of using it with
"count", since that seemed like something people would be likely to try.
With resource "for_each" now merged, we can replace that scary section
with a more positive example of using these two features together.
We still include a paragraph noting that "count" _could_ be used here, but
with a caution against doing so. This is in the hope of helping users
understand the difference between these two patterns and why for_each is
the superior choice for most situations.
Team tokens never worked with the `atlas` backend, but the `remote` backend
uses them as intended; they can perform plans and applies on workspaces where
the associated team has at least plan or write permissions, respectively.
The search "terraform leading zero" does not find the `format()`
function, which is perfectly capable of adding leading zeros.
Thus I have added this one word to help people find `format()`.
The correct environment variable corresponding to the `ca_file` variable is `CONSUL_CACERT` and not `CONSUL_CAFILE`.
See `backend/remote-state/consul/backend.go` line 77.
This also includes a previously-missing test that verifies the behavior
described here, implemented as a planning context test for consistency
with how the other ignore_changes tests are handled.
* Correct fmt -check
With `-check=false` the exit status is always zero.
With `-check=true` the exit status is zero when all files are properly formatted and non-zero otherwise.
* update fmt documentation to use short form for -diff and -check
We previously had some notes about handling configuration variants just
tacked on to the "dependency inversion" section as an afterthought, but
this idea is a major use-case for dependency inversion so it deserves its
own section and a specific example.
There have been a few questions about this so far which indicated that the
previous docs for this feature were very lacking. This is an attempt to
describe more completely what "any" means, and in particular that it isn't
actually a type at all but rather a placeholder for a type to be selected
dynamically.
Based on some common questions and feedback since the v0.12.0 release,
here we add some small additional content to the documentation for
"dynamic" blocks, covering how to access the keys of the collection being
iterated over and how to fold multiple collections into a single one to
achieve the effect of a nested iteration.
These follow the same principle as jsondecode and jsonencode, but use
YAML instead of JSON.
YAML has a much more complex information model than JSON, so we can only
support a subset of it during decoding, but hopefully the subset supported
here is a useful one.
Because there are many different ways to _generate_ YAML, the yamlencode
function is forced to make some decisions, and those decisions are likely
to affect compatibility with other real-world YAML parsers. Although the
format here is intended to be generic and compatible, we may find that
there are problems with it that'll we'll want to adjust for in a future
release, so yamlencode is therefore marked as experimental for now until
the underlying library is ready to commit to ongoing byte-for-byte
compatibility in serialization.
The main use-case here is met by yamldecode, which will allow reading in
files written in YAML format by humans for use in Terraform modules, in
situations where a higher-level input format than direct Terraform
language declarations is helpful.
This is similar to the function of the same name in Python, generating a
sequence of numbers as a list that can then be used in other
sequence-oriented operations.
The primary use-case for it is to turn a count expressed as a number into
a list of that length, which can then be iterated over or passed to a
collection function to produce that number of something else, as shown
in the example at the end of its documentation page.
Using az login and then terraform init from the command line I got `Error: Either an Access Key / SAS Token or the Resource Group for the Storage Account must be specified`
A longer-form guide will follow in the Sentinel section of the Terraform
Enterprise documentation, once it's ready. For now, this section isn't
saying anything useful since it was always just a stub for a guide we
planned to write later.
The upgrade tool is assuming that a type of "list" means list(string) and
a type of "map" means map(string), because that was what we documented
those as meaning.
In practice, Terraform 0.11 was lacking some validation which allowed
more complex nested structures in some cases even though they were pretty
inconvenient to use due to other language limitations.
The upgrade tool doesn't have enough context to make a reliable decision
on this, so instead we'll rely on the upgrade guide for this. We don't
need a TF-UPGRADE-TODO comment in this case because we reserve those for
things where a subsequent operation might cause the configuration to be
misinterpred, rather than just causing an error. Instead, we'll show an
example of the comment in the upgrade guide so the reader can easily
match it, and give some advice in the guide on how to address it.
We've seen in the past that some users try to use this form with the
ssh:// URL prefix, so we'll mention explicitly that this is invalid and
show a working example of how to use it without the URL scheme prefix.
Our original upgrade guide was drafted while some things were still in
flux and not all of our upgrade tooling was in place yet.
This redraft now attempts to be more specific and direct, showing exact
commands to run and (where relevant) exact error messages that Terraform
might return.
I also took this opportunity for some general copy-editing, though we'll
probably want to do one more pass of that alone (without changing any
content at the same time) before final release.
This new content presumes the existence of a Terraform v0.11.14 release,
which isn't published yet at the time of writing but should be published
before v0.12.0 final, once we've done final verification and review of
the upgrade path including it.
- Note that we intentionally omitted it from the sidebar, to reduce confusion.
- Write a summary up top so you can stop reading sooner if you don't actually need this.
* lang/funcs: testing of functions through the lang package API
The function-specific unit tests do not cover the HCL conversion that happens when the functions are called in a terraform configuration. For e.g., HCL converts sets to lists before passing it to the function. This means that we could not test passing a set in the function _unit_ tests.
This adds a higher-level acceptance test, plus a check that every (pure) function has a test.
* website/docs: update function documentation
There was some leftover v0.11-style interpolation syntax here.
We prefer to use a "naked" expression in situations like this where the result
isn't a string, because interpolations returning non-strings is a common source
of confusion for new users.
The division operator now always performs floating point division, whereas
before it would choose between float and int division based on the types
of its arguments.
We have a specific error message for when a fractional number is used as
an index in HCL, but this additional upgrade guidance provides a specific
solution to the problem: the floor function.
Sadly we don't have enough context in the current design of the upgrade
tool to make this fix automatic. With some refactoring it may be possible
to apply the fix automatically within list brackets, but since that is
a relatively complex change we'll first try this manual solution prompted
by an error message, because in practice so far we've seen this reported
only in the context of list indexing and our error check will catch that
and make the user aware of the need for a fix there.
* funcs/coalesce: return the first non-null, non-empty element from a
sequence.
The go-cty coalesce function, which was originally used here, returns the
first non-null element from a sequence. Terraform 0.11's coalesce,
however, returns the first non-empty string from a list of strings.
This new coalesce function aims to preserve terraform's documented
functionality while adding support for additional argument types. The
tests include those in go-cty and adapted tests from the 0.11 version of
coalesce.
* website/docs: update coalesce function document
The re-introduction of some of the ambiguity between argument and nested
block syntax (for compatibility with existing provider patterns)
unfortunately leads to some interesting consequences for attributes using
this mode.
While the behavior is generally as before in straightforward cases, this
page aims to spell out some of the different usage patterns explicitly
for the benefit of those writing more complex configurations, such as
generic re-usable modules where using argument vs. block syntax leads to
some real differences.
This page is intentionally not linked from anywhere in the part of the
website maintained in the Terraform repository. Instead, it can be linked
from the provider documentation for any argument where this pattern is
used, to help users understand the ways in which that argument might
deviate from the usual behaviors of arguments vs. nested blocks.
Some users are not accustomed to thinking of IP addresses in a bitwise
fashion, so the hope here is to give enough of an introduction to that way
of thinking for the reader to understand what the "newbits" and "netnum"
arguments represent.
The definition of split was referring the built-in function join. However, join is just one of the ways a string might have been created, and this could cause confusion.
In the terraform state documentation the verb "map" is widely used to
describe the relationship between an item in the state and in the real world
whereas the verb "attach" is not used anywhere.
For 0.11 I just specified the naming rules; for 0.12, I added some info about
referencing values and tightened up the layout of the optional arguments.
This commit also syncs up descriptions of `depends_on`.
'legacy' doesn't seem to be a thing anymore, and we were missing some
of the other values for -type. Also -no-color doesn't seem to be
relevant to this command.
1. The double "$" in the template confuses readers
2. As far as I can tell the variable "count" is not used either in this example
(it is in the next example though)
* docs: elaborate on supported remote backend versions
This PR adds a few lines to the docs to indicate which commands are
supported by what version of the remote backend and it makes a
recommendation about which version to use.
* Clarify remote state storage w/ TFE [skip ci]
Specifically, that this is the backend to use with remote state (all
tiers) and Free-Tier vs. Enterprise tiers differ in remote operations
* website: Arrange remote backend info differently
We have released the v0.12-oriented content to the website early in order
to support the beta process, but in some places we neglected to explicitly
mark features or content as being v0.12-only.
Here we add explicit markers to the main cases we've seen where readers
have reported confusion, along with some other tweaks in similar vein.
Terraform is way bigger than the core CLI tools and the language
now, and the docs have grown accordingly. So we're adding a
global index page to help users get around the many sections of the
docs site, and bumping the CLI/core docs down so they're no longer at the
top of the hierarchy.
The "right" (as in, conceptually pure) way to do this would be to actually
create a new level of directory hierarchy in between. But that would be real
expensive and annoying — the amount of 301s and links to edit would be
monumental, and it wouldn't gain us much beyond a certain picture-straightening
satisfaction, so I'm resisting the temptation.
As part of this, I'm copying the entire text of the 0.12
docs/configuration/modules.html page into docs/configuration-0-11/modules.html —
some of the 0.11 pages needed to be able to link to the moved content, I
didn't want to jump versions jarringly, and a close reading didn't reveal
anything in there that's inaccurate for 0.11.
The "terraform fmt" command produces a different canonical form than we
were showing in our examples here. Our examples should always reflect the
conventions applied by "terraform fmt" to avoid confusion.
(This particular decision is a pragmatic one because the formatter design
needs to use the same rules for the colon in the ? : conditional operator
as for the colon in "for" expressions.)
Since references to attributes of resources are by far the most common
reference type, and the mapping of resource type config to the attributes
is not always obvious, here we give some real examples of patterns for
accessing different configuration constructs within resource blocks along
with the resource type's exported attributes.
Since we don't have any real examples of labelled nested blocks yet (the
current SDK doesn't support them) I've included a hypothetical example for
now just to establish the patterns around them in preparation for
beginning to introduce them as we roll out this feature in the SDK.
As well as some general consolidation and reorganizing, this also includes
some updated advice for making the best use of new Terraform v0.12
features to create infrastructure building-blocks.
In particular, the "Module Usage" documentation is now consolidated into
the configuration section in order to bring all of our general language
documentation together, and the top-level "Modules" section is now
primarily focused on module _authors_ as an audience, covering topics such
as publishing modules and designing them for reuse.
* docs: update plan command documentation. Fixes#19235
* docs: added a missing reserved variable name. Fixes#19159.
* website: add note that resource names cannot start with a number
* website: add some notes to the 0.12 upgrade guide
The announcement post contains the information about the temporary
situation where not all of the providers are compatible yet. Linking there
rather than duplicating the information in the upgrade guide means we'll
be able to update in one place as the situation changes.
The upgrade guide had its last major upgrade while we were preparing for
the alpha releases. Now that the upgrade tool is more complete we can
describe the required changes in terms of that tool, and also add
additional information about provider upgrades.
We will revise this at least one more time before v0.12.0 final, but this
is an interim copy of the upgrade guide intended to help those who are
testing the beta releases.
Although /intro/getting-started includes docs content, those pages currently
redirect to the Learn platform, and so shouldn't be affected by the large unfurl
image.
Thomas Alton