Our module installer has a somewhat-informal idea of a "module package",
which is some external thing we can go fetch in order to add one or more
modules to the current configuration. Our documentation doesn't talk much
about it because most users seem to have found the distinction between
external and local modules pretty intuitive without us throwing a lot of
funny terminology at them, but there are some situations where the
distinction between a module and a module package are material to the
end-user.
One such situation is when using an absolute rather than relative
filesystem path: we treat that as an external package in order to make the
resulting working directory theoretically "portable" (although users can
do various other things to defeat that), and so Terraform will copy the
directory into .terraform/modules in the same way as it would download and
extract a remote archive package or clone a git repository.
A consequence of this, though, is that any relative paths called from
inside a module loaded from an absolute path will fail if they try to
traverse upward into the parent directory, because at runtime we're
actually running from a copy of the directory that's been taking out of
its original context.
A similar sort of situation can occur in a truly remote module package if
the author accidentally writes a "../" source path that traverses up out
of the package root, and so this commit introduces a special error message
for both situations that tries to be a bit clearer about there being a
package boundary and use that to explain why installation failed.
We would ideally have made escaping local references like that illegal in
the first place, but sadly we did not and so when we rebuilt the module
installer for Terraform v0.12 we ended up keeping the previous behavior of
just trying it and letting it succeed if there happened to somehow be a
matching directory at the given path, in order to remain compatible with
situations that had worked by coincidence rather than intention. For that
same reason, I've implemented this as a replacement error message we will
return only if local module installation was going to fail anyway, and
thus it only modifies the error message for some existing error situations
rather than introducing new error situations.
This also includes some light updates to the documentation to say a little
more about how Terraform treats absolute paths, though aiming not to get
too much into the weeds about module packages since it's something that
most users can get away with never knowing.
We got the replacement for this in earlier than anticipated, so these docs
were originally more pessimistic about when the alternative would be
available.
While we were working on and documenting these it wasn't clear exactly
what Terraform CLI version they would land in, and so we used
"Terraform v1.0" in the docs as a safe bound that was definitely going to
include all of them.
With everything now landed though, we can be more specific about which
v0.15.x minor release each of these appeared in.
Terraform 0.15.3 added support for a `-json` flag to the plan, apply,
and refresh commands, which renders the Terraform UI output in a
structured machine readable format. This commit adds documentation for
this interface.
The current documention was unclear about the full path of local mirrors
when using the XDG Base Directory Specification.
Also removed the trailing slashes for the other paths in this section.
- I'm using distinct subheaders and smaller paragraphs to try and make the info
about apply's two modes more skimmable.
- I'm also adding a separate "Plan Options" subheader (and keeping the section
tiny so it stays snugged up right next to the "Apply Options" one) to make it
extra-clear that Hey, There's More Options, They're Over There.
This is a light revamp of our plan output to make use of Terraform core's
new ability to report both the previous run state and the refreshed state,
allowing us to explicitly report changes made outside of Terraform.
Because whether a plan has "changes" or not is no longer such a
straightforward matter, this now merges views.Operation.Plan with
views.Operation.PlanNoChanges to produce a single function that knows how
to report all of the various permutations. This was also an opportunity
to fill some holes in our previous logic which caused it to produce some
confusing messages, including a new tailored message for when
"terraform destroy" detects that nothing needs to be destroyed.
This also allows users to request the refresh-only planning mode using a
new -refresh-only command line option. In that case, Terraform _only_
performs drift detection, and so applying a refresh-only plan only
involves writing a new state snapshot, without changing any real
infrastructure objects.
Previously the docs for this were rather confusing because they showed an
option to turn _on_ state locking, even though it's on by default.
Instead, we'll now show -lock=false in all cases and document it as
_disabling_ the default locking.
While working on this I also noticed that the equivalent docs on the
website were differently inconsistent. I've not made them fully consistent
here but at least moreso than they were before.
My original motivation here was to add the previously-missing -dry-run
option to the list of options
However, while in the area I noticed that this command hasn't had a
documentation refresh for a while and so I took the opportunity to update
it to match with our current writing style and terminology used in other
parts of the documentation, and so I've rewritten prose elsewhere on the
page to hopefully give the same information in a way that fits in better
with concepts discussed elsewhere in the documentation, and also to try
to add some additional context to connect this information with what
we've described in other places.
This rewrite also drops the example of moving from one "state file" to
another, because that's a legacy usage pattern that isn't supported when
using remote backends, and we recommend most folks to use remote backends
so it's strange to show an example that therefore won't work for most
people. Rather than adding additional qualifiers to that example I chose
to just remove it altogether, because we've generally been working to
de-emphasize these legacy local backend command line options elsewhere in
the documentation.
My original motivation here was to add the previously-missing -dry-run
option to the list of options
However, while in the area I noticed that this command hasn't had a
documentation refresh for a while and so I took the opportunity to update
it to match with our current writing style and terminology used in other
parts of the documentation, and so I've rewritten prose elsewhere on the
page to hopefully give the same information in a way that fits in better
with concepts discussed elsewhere in the documentation, and also to try
to add some additional context to connect this information with what
we've described in other places.
The Git book seems to be using a different anchor format now, and so this
link was previously effectively linking to the page as a whole rather
than to the specific section we're trying to refer to.
We previously had only very short descriptions of what
-ignore-remote-version does due to having the documentation for it inline
on many different command pages and -help output.
Instead, we'll now centralize the documentation about this argument on
the remote backend page, and link to it or refer to it from all other
locations. This then allows us to spend more words on discussing what
Terraform normally does _without_ this option and warning about the
consequences of using it.
This continues earlier precedent for some local-backend-specific options
which we also don't recommend for typical use. While this does make these
options a little more "buried" than before, that feels justified given
that they are all "exceptional use only" sort of options where users ought
to learn about various caveats before using them.
While there I also took this opportunity to fix some earlier omissions
with the local-backend-specific options and a few other minor consistency
tweaks.
* Add link to Modules in Package Sub-directories
Add link to "Modules in Package Sub-directories" section at top of page
* Fix broken links
* Update aws link, fixes missing anchor linkcheck
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
This pattern follows as a natural consequence of how for_each is defined,
but I've noticed from community forum Q&A that newcomers often don't
immediately notice the connection between what for_each expects as input
and what a for_each resource produces as a result, so my aim here is to
show a short example of that in the hope of helping folks see the link
here and get ideas on how to employ the technique in other situations.
The set of paths which caused a resource update to require replacement
has been stored in the plan since 0.15.0 (#28201). This commit adds a
simple JSON representation of these paths, allowing consumers of this
format to determine exactly which paths caused the resource to be
replaced.
This representation is intentionally more loosely encoded than the JSON
state serialization of paths used for sensitive attributes. Instead of a
path step being represented by an object with type and value, we use a
more-JavaScripty heterogenous array of numbers and strings. Any
practical consumer of this format will likely traverse an object tree
using the index operator, which should work more easily with this
format. It also allows easy prefix comparison for consumers which are
tracking paths.
While updating the documentation to include this new field, I noticed
that some others were missing, so added them too.
This allows a similar effect to pre-tainting an object but does the action
within the context of a normal plan and apply, avoiding the need for an
intermediate state where the old object still exists but is marked as
tainted.
The core functionality for this was already present, so this commit is
just the UI-level changes to make that option available for use and to
explain how it contributed to the resulting plan in Terraform's output.
It's been a long time since we gave this page an overhaul, and with our
ongoing efforts to make plan and apply incorporate all of the side-effects
that might need to be done against a configuration it seems like a good
time for some restructuring in that vein.
The starting idea here is to formally split the many "terraform plan"
options into a few different categories:
- Planning modes
- Planning options
- Other options
The planning modes and options are the subset that are also accepted by
"terraform apply" when it's running in its default mode of generating a
plan and then prompting for interactive approval of it. This then allows
us to avoid duplicating all of that information on the "terraform apply"
page, and thus allows us to spend more words discussing each of them.
This set of docs is intended as a fresh start into which we'll be able to
more surgically add in the information about -refresh-only and -replace=...
once we have those implemented. Consequently there are some parts of this
which may seem a little overwraught for what it's currently describing;
that's a result of my having prepared this by just deleting the
-refresh-only and -replace=... content from our initial docs draft and
submitted the result, in anticipation of re-adding the parts I've deleted
here in the very near future in other commits.
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.
For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.
This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.
So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
* lang/funcs: add (console-only) TypeFunction
The type() function, which is only available for terraform console,
prints out the type of a given value. This is mainly intended for
debugging - it's nice to be able to print out terraform's understanding
of a complex variable.
This introduces a new field for Scope: ConsoleMode. When ConsoleMode is true, any additional functions intended for use in the console (only) may be added.
Terraform v0.15 includes the conclusion of the deprecation cycle for some
renamed arguments in the "azure" backend.
We missed this on the first draft of the upgrade guide because this change
arrived along with various other more innocuous updates and so we didn't
spot it during our change review.
Unfortunately it seems that this link got lost in a merge conflict when
we did the big nav refactor earlier in the v0.15 cycle, so here we'll
retoractively add it to the new location for upgrade guide nav, in the
language layout rather than the downloads layout.
* website: v0.15 upgrade guide for sensitive resource attributes
Our earlier draft of this guide didn't include a section about the
stabilization of the "provider_sensitive_attrs" language experiment. This
new section aims to address the situation where a module might previously
have been returning a sensitive value without having marked it as such,
and thus that module will begin returning an error after upgrading to
Terraform v0.15.
As part of that, I also reviewed the existing documentation about these
features and made some edits aiming to make these four different sections
work well together if users refer to them all at once, as they are likely
to do if they follow the new links from the upgrade guide. I aimed to
retain all of the content we had before, but some of it is now in a new
location.
In particular, I moved the discussion about the v0.14 language experiment
into the upgrade guide, because it seems like a topic only really relevant
to those upgrading from an earlier version and not something folks need to
know about if they are using Terraform for the first time in v0.15 or
later.
* minor fixups
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
In the Terraform language we typically use lists of zero or one values in
some sense interchangably with single values that might be null, because
various Terraform language constructs are designed to work with
collections rather than with nullable values.
In Terraform v0.12 we made the splat operator [*] have a "special power"
of concisely converting from a possibly-null single value into a
zero-or-one list as a way to make that common operation more concise.
In a sense this "one" function is the opposite operation to that special
power: it goes from a zero-or-one collection (list, set, or tuple) to a
possibly-null single value.
This is a concise alternative to the following clunky conditional
expression, with the additional benefit that the following expression is
also not viable for set values, and it also properly handles the case
where there's unexpectedly more than one value:
length(var.foo) != 0 ? var.foo[0] : null
Instead, we can write:
one(var.foo)
As with the splat operator, this is a tricky tradeoff because it could be
argued that it's not something that'd be immediately intuitive to someone
unfamiliar with Terraform. However, I think that's justified given how
often zero-or-one collections arise in typical Terraform configurations.
Unlike the splat operator, it should at least be easier to search for its
name and find its documentation the first time you see it in a
configuration.
My expectation that this will become a common pattern is also my
justification for giving it a short, concise name. Arguably it could be
better named something like "oneornull", but that's a pretty clunky name
and I'm not convinced it really adds any clarity for someone who isn't
already familiar with it.
We previously added a hint to both resource for_each and dynamic blocks
about using the "flatten" and "setproduct" situations to construct
suitable collections to repeat over.
However, we used the same text in both places which ended up stating that
dynamic blocks can only accept map or set values, which is a constraint
that applies to resource for_each (because we need to assign a unique
identifier to each instance) and not to dynamic blocks (which don't have
any uniqueness enforced by Terraform Core itself).
To remove that contradiction with the text above which talks about what
is valid here, I've just generalized this to say "collection", because
the primary point of this paragraph is the "one element per desired nested
block" part, not specifically what sort of collections are permitted in
this location. (Text further up describes the supported types.)
We have these funny extra options that date back to before Terraform even
had remote state, which we've preserved along the way by most recently
incorporating them as special-case overrides for the local backend.
The documentation we had for these has grown less accurate over time as
the details have shifted, and was in many cases missing the requisite
caveats that they are only for the local backend and that backend
configuration is the modern, preferred way to deal with the use-cases they
were intended for.
We always have a bit of a tension with this sort of legacy option because
we want to keep them documented just enough to be useful to someone who
finds an existing script/etc using them and wants to know what they do,
but not to take up so much space that they might distract users from
finding the modern alternative they should consider instead.
As a compromise in that vein here I've created a new section about these
options under the local backend documentation, which then gives us the
space to go into some detail about the various behaviors and interactions
and also to discuss their history and our recommended alternatives. I then
simplified all of the other mentions of these in command documentation
to just link to or refer to the local backend documentation. My hope then
is that folks who need to know what these do can still find the docs, but
that information can be kept out of the direct path of new users so they
can focus on learning about remote backends instead.
This is certainly not the most ideal thing ever, but it seemed like the
best compromise between the competing priorities I described above.
Judith Malnick