Our module installer has a somewhat-informal idea of a "module package",
which is some external thing we can go fetch in order to add one or more
modules to the current configuration. Our documentation doesn't talk much
about it because most users seem to have found the distinction between
external and local modules pretty intuitive without us throwing a lot of
funny terminology at them, but there are some situations where the
distinction between a module and a module package are material to the
end-user.
One such situation is when using an absolute rather than relative
filesystem path: we treat that as an external package in order to make the
resulting working directory theoretically "portable" (although users can
do various other things to defeat that), and so Terraform will copy the
directory into .terraform/modules in the same way as it would download and
extract a remote archive package or clone a git repository.
A consequence of this, though, is that any relative paths called from
inside a module loaded from an absolute path will fail if they try to
traverse upward into the parent directory, because at runtime we're
actually running from a copy of the directory that's been taking out of
its original context.
A similar sort of situation can occur in a truly remote module package if
the author accidentally writes a "../" source path that traverses up out
of the package root, and so this commit introduces a special error message
for both situations that tries to be a bit clearer about there being a
package boundary and use that to explain why installation failed.
We would ideally have made escaping local references like that illegal in
the first place, but sadly we did not and so when we rebuilt the module
installer for Terraform v0.12 we ended up keeping the previous behavior of
just trying it and letting it succeed if there happened to somehow be a
matching directory at the given path, in order to remain compatible with
situations that had worked by coincidence rather than intention. For that
same reason, I've implemented this as a replacement error message we will
return only if local module installation was going to fail anyway, and
thus it only modifies the error message for some existing error situations
rather than introducing new error situations.
This also includes some light updates to the documentation to say a little
more about how Terraform treats absolute paths, though aiming not to get
too much into the weeds about module packages since it's something that
most users can get away with never knowing.
We got the replacement for this in earlier than anticipated, so these docs
were originally more pessimistic about when the alternative would be
available.
While we were working on and documenting these it wasn't clear exactly
what Terraform CLI version they would land in, and so we used
"Terraform v1.0" in the docs as a safe bound that was definitely going to
include all of them.
With everything now landed though, we can be more specific about which
v0.15.x minor release each of these appeared in.
Terraform 0.15.3 added support for a `-json` flag to the plan, apply,
and refresh commands, which renders the Terraform UI output in a
structured machine readable format. This commit adds documentation for
this interface.
The current documention was unclear about the full path of local mirrors
when using the XDG Base Directory Specification.
Also removed the trailing slashes for the other paths in this section.
- I'm using distinct subheaders and smaller paragraphs to try and make the info
about apply's two modes more skimmable.
- I'm also adding a separate "Plan Options" subheader (and keeping the section
tiny so it stays snugged up right next to the "Apply Options" one) to make it
extra-clear that Hey, There's More Options, They're Over There.
This is a light revamp of our plan output to make use of Terraform core's
new ability to report both the previous run state and the refreshed state,
allowing us to explicitly report changes made outside of Terraform.
Because whether a plan has "changes" or not is no longer such a
straightforward matter, this now merges views.Operation.Plan with
views.Operation.PlanNoChanges to produce a single function that knows how
to report all of the various permutations. This was also an opportunity
to fill some holes in our previous logic which caused it to produce some
confusing messages, including a new tailored message for when
"terraform destroy" detects that nothing needs to be destroyed.
This also allows users to request the refresh-only planning mode using a
new -refresh-only command line option. In that case, Terraform _only_
performs drift detection, and so applying a refresh-only plan only
involves writing a new state snapshot, without changing any real
infrastructure objects.
Previously the docs for this were rather confusing because they showed an
option to turn _on_ state locking, even though it's on by default.
Instead, we'll now show -lock=false in all cases and document it as
_disabling_ the default locking.
While working on this I also noticed that the equivalent docs on the
website were differently inconsistent. I've not made them fully consistent
here but at least moreso than they were before.
My original motivation here was to add the previously-missing -dry-run
option to the list of options
However, while in the area I noticed that this command hasn't had a
documentation refresh for a while and so I took the opportunity to update
it to match with our current writing style and terminology used in other
parts of the documentation, and so I've rewritten prose elsewhere on the
page to hopefully give the same information in a way that fits in better
with concepts discussed elsewhere in the documentation, and also to try
to add some additional context to connect this information with what
we've described in other places.
This rewrite also drops the example of moving from one "state file" to
another, because that's a legacy usage pattern that isn't supported when
using remote backends, and we recommend most folks to use remote backends
so it's strange to show an example that therefore won't work for most
people. Rather than adding additional qualifiers to that example I chose
to just remove it altogether, because we've generally been working to
de-emphasize these legacy local backend command line options elsewhere in
the documentation.
My original motivation here was to add the previously-missing -dry-run
option to the list of options
However, while in the area I noticed that this command hasn't had a
documentation refresh for a while and so I took the opportunity to update
it to match with our current writing style and terminology used in other
parts of the documentation, and so I've rewritten prose elsewhere on the
page to hopefully give the same information in a way that fits in better
with concepts discussed elsewhere in the documentation, and also to try
to add some additional context to connect this information with what
we've described in other places.
The Git book seems to be using a different anchor format now, and so this
link was previously effectively linking to the page as a whole rather
than to the specific section we're trying to refer to.
We previously had only very short descriptions of what
-ignore-remote-version does due to having the documentation for it inline
on many different command pages and -help output.
Instead, we'll now centralize the documentation about this argument on
the remote backend page, and link to it or refer to it from all other
locations. This then allows us to spend more words on discussing what
Terraform normally does _without_ this option and warning about the
consequences of using it.
This continues earlier precedent for some local-backend-specific options
which we also don't recommend for typical use. While this does make these
options a little more "buried" than before, that feels justified given
that they are all "exceptional use only" sort of options where users ought
to learn about various caveats before using them.
While there I also took this opportunity to fix some earlier omissions
with the local-backend-specific options and a few other minor consistency
tweaks.
* Add link to Modules in Package Sub-directories
Add link to "Modules in Package Sub-directories" section at top of page
* Fix broken links
* Update aws link, fixes missing anchor linkcheck
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
This pattern follows as a natural consequence of how for_each is defined,
but I've noticed from community forum Q&A that newcomers often don't
immediately notice the connection between what for_each expects as input
and what a for_each resource produces as a result, so my aim here is to
show a short example of that in the hope of helping folks see the link
here and get ideas on how to employ the technique in other situations.
The set of paths which caused a resource update to require replacement
has been stored in the plan since 0.15.0 (#28201). This commit adds a
simple JSON representation of these paths, allowing consumers of this
format to determine exactly which paths caused the resource to be
replaced.
This representation is intentionally more loosely encoded than the JSON
state serialization of paths used for sensitive attributes. Instead of a
path step being represented by an object with type and value, we use a
more-JavaScripty heterogenous array of numbers and strings. Any
practical consumer of this format will likely traverse an object tree
using the index operator, which should work more easily with this
format. It also allows easy prefix comparison for consumers which are
tracking paths.
While updating the documentation to include this new field, I noticed
that some others were missing, so added them too.
This allows a similar effect to pre-tainting an object but does the action
within the context of a normal plan and apply, avoiding the need for an
intermediate state where the old object still exists but is marked as
tainted.
The core functionality for this was already present, so this commit is
just the UI-level changes to make that option available for use and to
explain how it contributed to the resulting plan in Terraform's output.
It's been a long time since we gave this page an overhaul, and with our
ongoing efforts to make plan and apply incorporate all of the side-effects
that might need to be done against a configuration it seems like a good
time for some restructuring in that vein.
The starting idea here is to formally split the many "terraform plan"
options into a few different categories:
- Planning modes
- Planning options
- Other options
The planning modes and options are the subset that are also accepted by
"terraform apply" when it's running in its default mode of generating a
plan and then prompting for interactive approval of it. This then allows
us to avoid duplicating all of that information on the "terraform apply"
page, and thus allows us to spend more words discussing each of them.
This set of docs is intended as a fresh start into which we'll be able to
more surgically add in the information about -refresh-only and -replace=...
once we have those implemented. Consequently there are some parts of this
which may seem a little overwraught for what it's currently describing;
that's a result of my having prepared this by just deleting the
-refresh-only and -replace=... content from our initial docs draft and
submitted the result, in anticipation of re-adding the parts I've deleted
here in the very near future in other commits.
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.
For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.
This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.
So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
* lang/funcs: add (console-only) TypeFunction
The type() function, which is only available for terraform console,
prints out the type of a given value. This is mainly intended for
debugging - it's nice to be able to print out terraform's understanding
of a complex variable.
This introduces a new field for Scope: ConsoleMode. When ConsoleMode is true, any additional functions intended for use in the console (only) may be added.
* website: v0.15 upgrade guide for sensitive resource attributes
Our earlier draft of this guide didn't include a section about the
stabilization of the "provider_sensitive_attrs" language experiment. This
new section aims to address the situation where a module might previously
have been returning a sensitive value without having marked it as such,
and thus that module will begin returning an error after upgrading to
Terraform v0.15.
As part of that, I also reviewed the existing documentation about these
features and made some edits aiming to make these four different sections
work well together if users refer to them all at once, as they are likely
to do if they follow the new links from the upgrade guide. I aimed to
retain all of the content we had before, but some of it is now in a new
location.
In particular, I moved the discussion about the v0.14 language experiment
into the upgrade guide, because it seems like a topic only really relevant
to those upgrading from an earlier version and not something folks need to
know about if they are using Terraform for the first time in v0.15 or
later.
* minor fixups
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
In the Terraform language we typically use lists of zero or one values in
some sense interchangably with single values that might be null, because
various Terraform language constructs are designed to work with
collections rather than with nullable values.
In Terraform v0.12 we made the splat operator [*] have a "special power"
of concisely converting from a possibly-null single value into a
zero-or-one list as a way to make that common operation more concise.
In a sense this "one" function is the opposite operation to that special
power: it goes from a zero-or-one collection (list, set, or tuple) to a
possibly-null single value.
This is a concise alternative to the following clunky conditional
expression, with the additional benefit that the following expression is
also not viable for set values, and it also properly handles the case
where there's unexpectedly more than one value:
length(var.foo) != 0 ? var.foo[0] : null
Instead, we can write:
one(var.foo)
As with the splat operator, this is a tricky tradeoff because it could be
argued that it's not something that'd be immediately intuitive to someone
unfamiliar with Terraform. However, I think that's justified given how
often zero-or-one collections arise in typical Terraform configurations.
Unlike the splat operator, it should at least be easier to search for its
name and find its documentation the first time you see it in a
configuration.
My expectation that this will become a common pattern is also my
justification for giving it a short, concise name. Arguably it could be
better named something like "oneornull", but that's a pretty clunky name
and I'm not convinced it really adds any clarity for someone who isn't
already familiar with it.
We previously added a hint to both resource for_each and dynamic blocks
about using the "flatten" and "setproduct" situations to construct
suitable collections to repeat over.
However, we used the same text in both places which ended up stating that
dynamic blocks can only accept map or set values, which is a constraint
that applies to resource for_each (because we need to assign a unique
identifier to each instance) and not to dynamic blocks (which don't have
any uniqueness enforced by Terraform Core itself).
To remove that contradiction with the text above which talks about what
is valid here, I've just generalized this to say "collection", because
the primary point of this paragraph is the "one element per desired nested
block" part, not specifically what sort of collections are permitted in
this location. (Text further up describes the supported types.)
We have these funny extra options that date back to before Terraform even
had remote state, which we've preserved along the way by most recently
incorporating them as special-case overrides for the local backend.
The documentation we had for these has grown less accurate over time as
the details have shifted, and was in many cases missing the requisite
caveats that they are only for the local backend and that backend
configuration is the modern, preferred way to deal with the use-cases they
were intended for.
We always have a bit of a tension with this sort of legacy option because
we want to keep them documented just enough to be useful to someone who
finds an existing script/etc using them and wants to know what they do,
but not to take up so much space that they might distract users from
finding the modern alternative they should consider instead.
As a compromise in that vein here I've created a new section about these
options under the local backend documentation, which then gives us the
space to go into some detail about the various behaviors and interactions
and also to discuss their history and our recommended alternatives. I then
simplified all of the other mentions of these in command documentation
to just link to or refer to the local backend documentation. My hope then
is that folks who need to know what these do can still find the docs, but
that information can be kept out of the direct path of new users so they
can focus on learning about remote backends instead.
This is certainly not the most ideal thing ever, but it seemed like the
best compromise between the competing priorities I described above.
The traversal value is normally a valid HCL string, but can be
simplified if a traversal step has a complex index value (e.g. an
object). This means it is not always parseable HCL, so this commit
updates the documentation to clarify this and explicitly record that we
do not guarantee its contents are stable. The purpose of these values is
purely for building human-readable UI.
These aim to allow hinting to Terraform about situations where it's not
able to automatically infer value sensitivity.
"nonsensitive" is for situations where Terraform's behavior is too
conservative, such as when a new value is derived from a sensitive value
in such a way that all of the sensitive content is removed.
"sensitive", on the other hand, is for situations where Terraform can't
otherwise infer that a value is sensitive. These situations should be
pretty rare in a module that's making effective use of sensitive input
variables and output values, but the documentation shows one example of
an uncommon situation where a more direct hint via this function would
be needed.
Both of these functions are aimed at only occasional use in unusual
situations. They are here for reasons of pragmatism, not because we
expect them to be used routinely or recommend their use.
This is not currently a supported interface, but we plan to release
tool(s) that consume parts of it that are more dependable later,
separately from Terraform CLI itself.
In line with the other complex JSON output formats for plan and provider
schema, here we add an explicit `format_version` field to the JSON
output of terraform validate.
Fixes#27506
Add a new flag `-lockfile=readonly` to `terraform init`.
It would be useful to allow us to suppress dependency lockfile changes
explicitly.
The type of the `-lockfile` flag is string rather than bool, leaving
room for future extensions to other behavior variants.
The readonly mode suppresses lockfile changes, but should verify
checksums against the information already recorded. It should conflict
with the `-upgrade` flag.
Note: In the original use-case described in #27506, I would like to
suppress adding zh hashes, but a test code here suppresses adding h1
hashes because it's easy for testing.
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
Since this is still at an early phase and likely to change significantly
in future iterations, rather than attempting to guess on a suitable final
location for documenting the testing feature I've instead taken the
unusual approach of adding a new page that is explicitly about the
experiment. My expectation is that once we conclude the experiment we'll
replace this new page with a stub that just explains that there was once
an experiment and then links to whatever final feature unfolded from the
research.
The URL for this page is hard-coded into the warning message in the
"terraform test" command, so as we continue to evolve this feature in
future releases we'll need to update the callout note on the page about
which Terraform CLI version it's currently talking about, so users of
older versions can clearly see when they'd need to upgrade in order to
participate in a later incarnation of the experiment.
* Remove deprecation on undeclared variable
Remove deprecation and add docs specific to the behavior around
undeclared variable values
* Limit full warnings to 2 instances, then summary
This way, the third warning is a summary, rather than the fourth
warning being the summary
To make the command arguments easier to understand and extend, we are
moving away from positional arguments. This commit changes the graph
command to take a `-plan` flag instead of an optional trailing path.
During the language/CLI docs reorg, we noticed several pages that were no longer
viable; some were redundant, some useless, and some just very obsolete.
Since we were trying to avoid breaking links at the time, we opted to remove
them from the navs and leave them as "ghost pages" — still accessible, but not
findable.
This commit finally cleans these ghosts up and updates any remaining links to
relevant modern pages. Bustin' makes me feel good. 👻🚫
This variable mechanism was replaced long ago with a explicit `Allow
destroy plans` setting on a Terraform Cloud workspace, and no longer
does anything: https://www.terraform.io/docs/cloud/workspaces/settings.html#destruction-and-deletion
Rather than mention this new mechanism at all though, I've removed the
requisite from here entirely - the reason being that a consideration
like this is no different from other permission concerns (e.g. "You must
have Apply permission on a workspace to `apply`"), and without
enumerating _all_ of these here - which doesn't seem appropriate - we
just remove this concern entirely.
...and also shrink the explanation for alternate sharing approaches, a bit.
Actually, it looks like I already half-adopted it by accident. 😬 But this
commit adds it to the sidebar under "State", so users can browse to it. I'm
leaving the URL alone, because it's not urgent and we'll need to adjust a large
swath of URLs at some point anyway.
This change effectively stops presenting `terraform` as a provider in the normal
sense, and reduces /docs/providers/terraform/index.html to a ghost page in the
language section (to avoid breaking links for the time being). The message a
reader should get is that Terraform has one special built-in data source where
you don't need to think about the provider or its version.
As of December 18, 2020, we've redirected nearly all of the provider
documentation that used to live on terraform.io:
- For providers that got published on the Registry, we redirected each docs page
to the corresponding Registry docs page.
- For providers that never got adopted by a new publisher, we archived the
GitHub repository and redirected each docs page to the corresponding Markdown
source file on github.com. (For an example of these redirects, see
https://www.terraform.io/docs/providers/telefonicaopencloud/r/s3_bucket.html)
There are ten providers left that we haven't redirected. These ones got adopted
by new publishers and _will_ end up on the Registry, but they aren't quite ready
to ship and get their permanent redirects, and we don't want to sabotage their
SEO by 301ing to a temporary destination.
These links largely still go somewhere useful, but they have some kind of issue
revealed by our new link checker:
- Some of them point to a stale URL that redirects, and can be updated to the
new destination.
- Some of them point to anchors that don't exist (anymore?) in the destination.
- Some of them end up redirecting unnecessarily due to how the server handles
directory URLs without trailing slashes. Sorry, I know that's pointless, just,
humor me for the time being so we can get our CI green. 😭
In a couple cases, I've added invisible anchors to destination pages, either to
preserve an old habit or because the current anchors kind of suck due to being
particularly long or meandering.
* Add limitations section to for_each
Move limitations from a note to their own section,
to allow for expansion on disallowing sensitive values
in for_each
This PR updates the documentation of input variable of terraform. It's
mentioned that multiple `-var` is possible, but no example is given.
This PR adds an example of multiple `-var` option
* command/state list: list resources in nested and expaneded modules
A few distinct bugs fixed in here:
There was a bug in the logic checking if a given module was the child of
the targetAddr, now fixed. That resolved the basic issue where resources
in nested submodules were not listed.
The logic around allowMissing needed some tweaking to allow for empty
modules, as long as those modules had submodules with resources. state
list is the only command using allowMissing with false so this felt safe
to do.
Finally I extended the logic so list would included expanded modules,
which is to say giving module.foo would result in resources from
module.foo[1], module.foo[0], etc.
* update state list docs to show that module filtering includes any nested
modules
So far the output command has had a default output format intended for
human consumption and a JSON output format intended for machine
consumption.
However, until Terraform v0.14 the default output format for primitive
types happened to be _almost_ a raw string representation of the value,
and so users started using that as a more convenient way to access
primitive-typed output values from shell scripts, avoiding the need to
also use a tool like "jq" to decode the JSON.
Recognizing that primitive-typed output values are common and that
processing them with shell scripts is common, this commit introduces a new
-raw mode which is explicitly intended for that use-case, guaranteeing
that the result will always be the direct result of a string conversion
of the output value, or an error if no such conversion is possible.
Our policy elsewhere in Terraform is that we always use JSON for
machine-readable output. We adopted that policy because our other
machine-readable output has typically been complex data structures rather
than single primitive values. A special mode seems justified for output
values because it is common for root module output values to be just
strings, and so it's pragmatic to offer access to the raw value directly
rather than requiring a round-trip through JSON.
This is a repeated cause of confusion and questions in the community
forum, because both JSON and YAML valid syntax are hard to generate using
just string concatenation. Terraform has built-in functions for both of
these common serializations to avoid those problems, and so this will
hopefully make these better alternatives more discoverable.
When we did the earlier documentation rework for Terraform v0.12 we still
had one big "Expressions" page talking about the various operators and
constructs, and so we had to be a bit economical with the details about
some more complicated constructs in order to avoid the page becoming even
more overwhelming.
However, we've recently reorganized the language documentation again so
that the expressions section is split across several separate pages, and
that gives some freedom to go into some more detail about and show longer
examples for certain features.
My changes here are not intended to be an exhaustive rewrite but I did
try to focus on some areas I've commonly seen questions about when helping
in the community forum and elsewhere, and also to create a little more
connectivity between the different content so readers can hopefully find
what they are looking for more easily when they're not yet sure what
terminology to look for.
As of Terraform 0.13+, the get-plugins command has been
superceded by new provider installation mechanisms, and
general philosophy (providers are always installed, but
the sources may be customized). Updat the init command
to give users a warning if they are setting this flag,
to encourage them to remove it from their workflow, and
update relevant docs and docstrings as well
I originally drafted these docs in a context where I was relying on
GitHub's Markdown renderer, and carelessly imported them into the
Terraform website without verifying that the website's Markdown renderer
could process it. This particular quirk has bitten us before: the website
Markdown parser expects follow-on paragraphs in a list item to be indented
at least four spaces, and with less than that it ignores the leading
whitespace altogether and just understands a normal paragraph.
This change will cause the follow-on paragraphs to now correctly render
as part of the bullet points they are intended to be attached to.
This is under a heading "Sensitive Resource Attributes" on the assumption
that if we later stabilize this feature then this heading will live on
with some different content that describes the propagation of sensitive
values from resource attributes, rather than describing the experiment.
The resources, expressions, and modules pages were all split into smaller, more
navigable pages, but the old URLs had accumulated a large number of deep links
to their section headers. To help people recover when they click an old link, we
converted those old URLs to landing pages, which preserve all of the old in-page
anchors and point readers to the appropriate new destinations.
However, because the new link-to-new-page sections are so small, it was kind of
hard to tell which section you had clicked into! Especially if you were near the
bottom of the page and the browser wasn't able to position the desired section
at the very top of the window.
This commit aims to improve that by putting one full screen of whitespace in
between every linkable section on these landing pages. Yes, it's a hack, but
you're meant to only view these pages for three seconds or so before moving on
to the place you wanted to be, and this should help dispel any confusion about
which place that is.
This tutorial uses references to local values, conditional expressions,
and splat expressions, so I've added it to those pages as well as the
expressions overview.
We've historically made statements like this in response to requests for
more customization to the "terraform fmt" behavior, but the documentation
itself was somewhat vague about the intended goals of this command.
This is an attempt to be more explicit that consistency between codebases
is the primary goal of this command, and that the examples in the
Terraform documentation are our main guide for what is "idiomatic style"
when adding additional rules over time.
Nothing here is intended to be new policy, but instead as codifying
positions we've taken elsewhere in the past in the hope of allowing users
to decide how (and whether) they wish to make use of this tool.
iaoiui