When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.
This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.
To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.
Terraform version compatibility is defined as:
- For all releases before 0.14.0, local must exactly equal remote, as
two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
we will not change the state version number in a patch release.
If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.
Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.
In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
The Registry is a web service whose behavior isn't directly tied to Terraform
core's release cycle; therefore, its docs should be decoupled from that release
cycle as well.
https://github.com/hashicorp/terraform-website/pull/1517 adopts the registry
docs into hashicorp/terraform-website, which already hosts several other
corpuses of documentation that aren't tied to Terraform core's version (like
Terraform Cloud, Terraform Enterprise, and Extending Terraform). Once that PR is
merged, we should remove the registry docs from this repository to avoid
confusing anyone.
The local-exec provisioner documentation includes an example which refers
to an attribute of the current resource using its full traversal path,
rather than using "self" as we typically expect.
Due to some coincidences in how Terraform builds the dependency graph,
referring to the resource in this way happens to work when the resource
has only a single instance (the graph builder just skips that
self-referential dependency edge), but it fails if the user later tries
to add "count" or "for_each" to the resource, because at that point all
of the instances become dependent on one another, which creates a
dependency cycle.
Using "self" to access the current instance attributes is the usual
approach, so I've updated the documentation to show that.
As written previously this seemed to suggest using "app.terraform.io" (the
"hostname you use to access the Terraform Cloud application) to access a
private registry in Terraform Enterprise, but that isn't true and I assume
isn't what was intended.
Instead, the hostname for a Terraform Enterprise instance is the hostname
where the Terraform Enterprise application is running, which is both the
hostname where users would find its web UI and the hostname they'd use
to configure the "remote" backend for remote operations and state storage.
In order to be able to predict a result type even if arguments are not yet
known, coalesce requires all of its arguments to be of the same type. Our
usual automatic conversion rules mean that in some cases the result is
a silent type conversion rather than an explicit error, so we'll at least
document that so that folks who encounter it can understand what is
causing the likely-surprising behavior.
If we were building this function over again today I expect we'd make it
always return an error under type mismatch, but to do so now would be a
breaking change and the potential cost of that seems too high for
something that doesn't seem to arise incredibly often in practice.
This one is a lot like the previous two commits, but slightly more complex:
- Only adding one new meta-argument page, for `providers`; otherwise, it just
re-uses the dual-purpose pages I made in the resources commit.
- About that `providers` argument: The stuff that was relevant to consumers of a
module went in that meta-argument page, but there was also a huge deep dive on
how the _author_ of a re-usable module should handle provider configurations
in cases where inheriting the default providers isn't sufficient. THAT, I
moved into a new page in the module development section. (For the consumer of
a module, this should all be an implementation detail; the module README
should tell you which aliased providers you need to configure and pass, and
then you just do it, without worrying about proxy configuration blocks etc.)
- The "standard module structure" recommendations in the main module development
page gets a page of its own, to make it more prominent and discoverable.
- Same deal with using the old URL as a landing page, at least for the main
module calls page. It didn't seem necessary for the module development page.
- Resource behavior gets its own page.
- Meta-arguments all get their own pages.
- Stuff about resource syntax itself gets a page.
In the process of breaking the meta-arguments out into their own pages, I
revised them (with the exception of `provider`) so that they apply to both
resources and modules.
Like with Expressions, this commit repurposes the old resources.html URL as a
landing page for old links.
This commit converts the previous URL for this content to a landing page, which
captures all of the previous in-page anchors and directs readers to the new home
for each section.
For some time now we've been recommending explicitly passing data between
configurations using separate resource types and data sources, rather than
always using terraform_remote_state, for reasons including reducing
coupling between subsystems and allowing a configuration's state snapshots
to be under restrictive access controls.
However, those recommendations have so far not appeared directly in the
documentation for terraform_remote_state, and have instead just been
alluded to elsewhere in the documentation when discussing ways to pass
data between configurations.
This change, then, is an attempt to be clear and explicit about the
recommendation and to give a variety of specific examples of how to
implement it. The terraform_remote_state data source page is admittedly
not the most obvious place in the information architecture to put a set
of alternatives to it, but it does appear that this documentation page is
where people most commonly end up when researching options in this area
and so I've put this here in an attempt to "meet people where they are".
Possibly in a future documentation reorganization we might have an
separate page specifically about sharing data between configurations, but
we don't currently have time to do that bigger reorganization. If we do so
later, the content on this page could potentially be replaced with a
summary of the recommendation and a link to another place for the details,
but the goal here is to make this information visible in the existing
location people look for it, rather than blocking until there's a better
place for it to live.
This also includes a small amount of editing of some existing content on
the page to use terminology and style more similar to how our main
configuration language documentation is written,.
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
Some hasty, incorrect merge conflict fixing caused this page to have a
strange mix of terminology between "system" and "provider". Along with
that, there were also several editorial errors caused by text on this
page having originally been derived from the provider registry
documentation.
This documentation will now consistently talk about being a module
registry protocol rather than a provider registry protocol, and it will
consistently use the term "system" as a generic term for the final part
of the module source address, aside from noting that there is an optional
convention to name it after the "type" part of an official provider when
possible.
This is a new part of the existing module_variable_optional_attrs
experiment, because it's intended to complement the ability to declare
an input variable whose type constraint is an object type with optional
attributes. Module authors can use this to replace null values (that were
either explicitly set or implied by attribute omission) with other
non-null values of the same type.
This function is a bit more type-fussy than our functions typically are
because it's intended for use primarily with input variables that have
fully-specified type constraints, and thus it uses that type information
to help inform how the defaults data structure should be interpreted.
Other uses of this function will probably be harder today because it takes
a lot of extra annotation to build a value of a specific type if it isn't
passing through a variable type constraint. Perhaps later language
features for more general type conversion will make this more applicable,
but for now the more general form of this problem is better solved other
ways.
The new nav structure demanded a few new pages that give context about a feature
or workflow. In a few cases, they take text from an existing page.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Judith Malnick <judith.patudith@gmail.com>
We're splitting the current Terraform CLI docs into two top-level categories,
and these are the new nav sidebars for those sections.
As of this commit, they refer to some new "glue" pages that don't exist yet.
The HashiCorp engineering services team has set up APT and Yum
repositories as alternative installation methods for various HashiCorp
products, now including Terraform.
We don't really have a great place to talk about these in our current
website structure. There is a longer-term plan to revamp the downloads
page to include other options, but we are already getting lots of
questions about how to use these repositories and so my goal here is to
publish at least a first pass of documentation, linked from the Downloads
page sidebar as a placeholder for now, so we'll have somewhere to refer to
when answering such questions.
My intent is that even once we have a revamped Downloads page that
mentions these options more clearly, we'll still need to link out to
another page to talk about various details, and so the two new URLs this
creates would be the home of that content, even if we rewrite the specific
prose here to work better in the context of the new Downloads page.
The example configuration now uses Terraform 0.12+ syntax, and the
output examples are up to date with the current text UI. We also add an
explicit recommendation to use the `-json` option for a consistent and
stable output format, for use in automation.
Prior to Terraform 0.12 these two functions were the only way to construct
literal lists and maps (respectively) in HIL expressions. Terraform 0.12,
by switching to HCL 2, introduced first-class syntax for constructing
tuple and object values, which can then be converted into list and map
values using the tolist and tomap type conversion functions.
We marked both of these functions as deprecated in the Terraform v0.12
release and have since then mentioned in the docs that they will be
removed in a future Terraform version. The "terraform 0.12upgrade" tool
from Terraform v0.12 also included a rule to automatically rewrite uses
of these functions into equivalent new syntax.
The main motivation for removing these now is just to get this change made
prior to Terraform 1.0. as we'll be doing with various other deprecations.
However, a specific reason for these two functions in particular is that
their existence is what caused us to invent the idea of a "type expression"
as a distinct kind of expression in Terraform v0.12, and so removing them
now would allow potentially unifying type expressions with value
expressions in a future release.
We do not have any current specific plans to make that change, but one
potential motivation for doing so would be to take another attempt at a
generalized "convert" function which takes a type as one of its arguments.
Our previous attempt to implement such a function was foiled by the fact
that Terraform's expression validator doesn't have any way to know to
treat one argument of a particular function as special, and so it was
generating incorrect error messages. We won't necessarily do that, but
having these "list" and "map" functions out of the way leaves the option
open.
The documentation states that an explicit type conversion to set is needed, but it does not say why implicit type conversion does not work.
Co-authored-by: Nick Fagerlund <nick@hashicorp.com>
* The index must be non-negative integer
and added instructions on how to get the last value in the list.
* Typo fix
Co-authored-by: Nick Fagerlund <nick@hashicorp.com>
* Update module-registry-protocol.html.md
1: There is a mismatch in the segment labels for the version query URL (system vs provider)
2: There is a discrepancy between the documentation and the actual generated request for retrieving module source code (URL segments 4 vs 3)
- There is no segment for "provider"
* Update module-registry-protocol.html.md
Changed ```:system``` to ```:provider``` for versions and source API URLs
These pages are thoroughly obsolete. Later, we'll delete and redirect them; for
now, we'll make sure the relevant pages are front-and-center in the sidebar if
someone somehow ends up on here.
Guides was already mostly gone. The two major remaining bits (the "core
workflow" guide and the "recommended practices" guide, which deserves a rename)
make much more sense as highly technical marketing material, and thus have a
natural home in the intro section.
Several `terraform` subcommands include sub-sub-commands; with our old sidebar
system, viewing those took you to an isolated "island" nav sidebar, away from
the main docs. The new navigation will adopt all these pages, so we don't need
to exile the reader to odd places.
As of this commit, that layout doesn't exist yet, but I'm isolating the one-line
changes to their own commit to try and keep your eyes from glazing over.
We typically try to avoid making subjective, boasty claims in our
documentation in recent times, but there remained both some older
documentation that we've not recently revised and also some newer examples
that are, in retrospect, also perhaps more "boasty" than they need to be.
We prefer not to use this sort of boasty language because not everyone
using Terraform has the same background and experience, and so what is
"easy" or "intuitive" to one person may not be so to another person, and
that should not suggest that the second person is in any way wrong or
inadequate.
In reviewing some of our use of the word "easy" here I tried as much as
possible to surgically revise the existing content without getting drawn
into a big rewrite, but in some cases the content was either pretty
unsalvageable (due to talking about obsolete features that were removed
long ago) or required some broader changes to make the result hopefully
still get the same facts across. In those cases I've both removed some
content entirely or adjusted larger paragraphs.
This was not an exhaustive review and so I'm sure there's still plenty of
room for similar improvements elsewhere. I also resisted the urge to
update some pages that contain outdated information about currently-active
features.
My initial motivation here was to update the example output from
Terraform's top-level help list to match recent updates in the layout
and language used.
However, while here I took the opportunity to update some dated language
that was not consistent with our modern documentation writing style,
in particular including a totally unnecessary and potentially-alienating
claim that Terraform is "very easy to use". Our modern writing style
discourages this sort of "boastful" language and encourages us to focus on
the facts at hand.
This is an analog to the "alltrue" function, using OR as the reduce
operator rather than AND.
This also includes some simplification of the "alltrue" implementation
to implement it similarly as a sort of reduce operation with AND
as the reduce operator, but with the same effective behavior.
* Fixes#26684
* Update provider-requirements.html.md
Removing additional/extra newlines
* Update provider-requirements.html.md
And now some trailing spaces. le sigh
* Update config.html.md
When reading this page, I couldn't find the list of the "supported backends to the left". They're actually on a different page, so thought I'd update it so that others wouldn't find it confusing like me.
If this is ok with you, would it be possible to label this PR with 'hacktoberfest-accepted'? I'm still new to this. If not, I'd be alright. Thank you!
* Update config.html.md
Swapped the full URL in the link for a relative path
Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
* ADD CLI option position for force-unlock command
* Update force-unlock.html.markdown
Made a change to also include the missing [DIR]
Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.
"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.
The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
* Add note to upgrade guide about provider sensitivity
Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
For normal provider installation we want to associate each provider with
a selected version number and find a suitable package for that version
that conforms to the official hashes for that release.
Those requirements are very onerous for a provider developer currently
testing a not-yet-released build, though. To allow for that case this new
CLI configuration feature allows overriding specific providers to refer
to give local filesystem directories.
Any provider overridden in this way is not subject to the usual
restrictions about selected versions or checksum conformance, and
activating an override won't cause any changes to the selections recorded
in the lock file because it's intended to be a temporary setting for one
developer only.
This is, in a sense, a spiritual successor of an old capability we had to
override specific plugins in the CLI configuration file. There were
some vestiges of that left in the main package and CLI config package
but nothing has actually been honoring them for several versions now and
so this commit removes them to avoid confusion with the new mechanism.
Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.
The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.
Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.
After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:
- The local cache directory will now be .terraform/providers rather than
.terraform/plugins, because .terraform/plugins is effectively "poisoned"
by the incorrect usage that we can't reliably distinguish from prior
version correct usage.
- The .terraform/plugins directory is now the "legacy cache directory". It
is intentionally _not_ now a filesystem mirror directory, because that
would risk incorrectly interpreting providers automatically installed
by Terraform v0.13 as if they were a local mirror, and thus upgrades
and checksum fetches from the origin registry would be blocked.
- Because of the previous two points, someone who _was_ trying to use the
legacy cache directory as a filesystem mirror would see installation
fail for any providers they manually added to the legacy directory.
To avoid leaving that user stumped as to what went wrong, there's a
heuristic for the case where a non-official provider fails installation
and yet we can see it in the legacy cache directory. If that heuristic
matches then we'll produce a warning message hinting to move the
provider under the terraform.d/plugins directory, which is a _correct_
location for "bundled" provider plugins that belong only to a single
configuration (as opposed to being installed globally on a system).
This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.
The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
The upgrade requirements for this release are considerably more modest
than for Terraform v0.13, so this time we just have some notes about a
few changes in behavior that may be impactful to some users.
This first pass is intended to be included as part of a forthcoming beta
testers' guide as we begin the v0.14 beta testing period. We will make
further changes to this upgrade guide based on feedback from those who
participate in the beta process.
Note that this upgrade guide is not intended as release marketing material
and so its presentation is focused on addressing concerns users might
encounter while upgrading. We'll share highlights from the release in
other contexts, such as the changelog and in the product blog.
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.
In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.
Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
This includes both the main documentation about the lock file itself and
changes to related documentation about Terraform commands that interact
with the lock file.
We will likely continue to update this first pass of documentation as we
get feedback and questions during the prerelease period.
* website: Update all Learn crosslinks
The URL structure on Learn recently changed, so it's time to update some URLs.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Move the information about state from the "caveats" to the main
info section, using similar information to sensitive outputs.
Updates the header of the section from similar inspiration.
We can remove the caveat about changing map elements.
Add a little more text about the intended use case for ignore_changes,
as the common case of fixing erroneous provider behavior should not be
the primary motivation for the maintenance of this feature.
* Put link to tutorial in its own section, call it a tutorial instead of guide, and use new canonical URL.
* Mention limitations of using import with a remote backed
* Typo fix
Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>
This commit adds an `alltrue` function to Terraform configuration. A
reason we might want this function is because it will enable more
powerful custom variable validations. For example:
```hcl
variable "amis" {
type = list(object({
id = string
}))
validation {
condition = (alltrue([
for a in var.amis : length(a.id) > 4 && substr(a.id, 0, 4) == "ami-"
]))
error_message = "The ID of at least one AMI was invalid."
}
}
```
For a credentials helper plugin to be useful with Terraform 0.13+, we
need to cope with the case of having no credentials for a host without
this being an error. This is to allow the public Terraform Registry to
be accessed without supplying a token.
The way to implement this is to respond to queries for credentials for a
host which has no credentials stored with an empty object and a success
exit code. This contradicts the previous documentation, which calls for
an error response in this case.
* Adding not about data-sources and depends-on for 0.12 users
* Bold
* A little more markdown
* A little more markdown for data_sources in 0.12
* Some iteration based on good feedback
In addition to the directories previously listed, Terraform looks in the
CLI config directory ($HOME/.terraform.d/plugins on macOS/Linux/UNIX,
and %APPDATA%/terraform.d/plugins on Windows). List this in the
documentation for clarity.
We also add a note about the working directory relative "vendor"
location, ./terraform.d/plugins.
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
This new option is intended to address the previous inconsistencies where
some older subcommands supported partially changing the target directory
(where Terraform would use the new directory inconsistently) where newer
commands did not support that override at all.
Instead, now Terraform will accept a -chdir command at the start of the
command line (before the subcommand) and will interpret it as a request
to direct all actions that would normally be taken in the current working
directory into the target directory instead. This is similar to options
offered by some other similar tools, such as the -C option in "make".
The new option is only accepted at the start of the command line (before
the subcommand) as a way to reflect that it is a global command (not
specific to a particular subcommand) and that it takes effect _before_
executing the subcommand. This also means it'll be forced to appear before
any other command-specific arguments that take file paths, which hopefully
communicates that those other arguments are interpreted relative to the
overridden path.
As a measure of pragmatism for existing uses, the path.cwd object in
the Terraform language will continue to return the _original_ working
directory (ignoring -chdir), in case that is important in some exceptional
workflows. The path.root object gives the root module directory, which
will always match the overriden working directory unless the user
simultaneously uses one of the legacy directory override arguments, which
is not a pattern we intend to support in the long run.
As a first step down the deprecation path, this commit adjusts the
documentation to de-emphasize the inconsistent old command line arguments,
including specific guidance on what to use instead for the main three
workflow commands, but all of those options remain supported in the same
way as they were before. In a later commit we'll make those arguments
produce a visible deprecation warning in Terraform's output, and then
in an even later commit we'll remove them entirely so that -chdir is the
single supported way to run Terraform from a directory other than the
one containing the root module configuration.
The subtle difference in keywords when creating vs. accessing locals trips
people up, even more than the "variable" vs. "var" distinction. It deserves its
own subheader on the page, plus a nice noisy callout.
I've just wasted an hour to two hours trying to find the problem to finally realize that although I declare a "locals" block, it's referred to as "local". This is pretty weird! So let's be be clear about this.
The error diagnostic shown when legacy state contains resources from
in-house providers has changed, so update references to it in the 0.13
upgrade guide.
We previously had this just stubbed out because it was a stretch goal for
the v0.13.0 release and it ultimately didn't make it in.
Here we fill out the existing stub -- with a minor change to its interface
so it can access credentials -- with a client implementation that is
compatible with the directory structure produced by the
"terraform providers mirror" subcommand, were the result to be published
on a static file server.
- Edits to registry overview
- Add index link as 'overview' (header links are semi-invisible)
- move providers/overview.html to providers/index.html
- Edits to providers overview
- fix filename of os-arch
- edits to provider publishing
Terraform's design assumes that each remote object in Terraform's care is
bound to one resource instance and one alone. If the same object is bound
to multiple instances then confusing behavior will often result, such as
two resource configurations competing to update a single object, or
objects being "left behind" when all existing Terraform deployments are
destroyed.
This assumption was previously only implied, though. This change is an
attempt to be more explicit about it, although these are additions to some
older documentation sections that have not been revised for some time and
so this is just a best effort to make this information discoverable
without getting drawn into a full-on reorganization of these sections.
While revising this there were some particular oddities that I decided to
revise while I was there, but I'll leave a fuller revision of this older
content for a later commit when we have more time to review it in greater
detail.
* Make sidebar nav in language docs more intuitive
* Minor display fixes for registry docs
* Explain providers in the registry in the providers index
* Revise a bunch of language docs around provider reqs
This is mostly an effort to smooth out some of the explanations, make sure
things are presented in a helpful order, make sure terminology lines up, draw
connections between related concepts, make default behavior more apparent, and
the like. It shouldn't include very much new information, but there might be one
or two things that came out of a conversation somewhere.
Co-authored-by: Judith Malnick <judith@hashicorp.com>
As part of documenting the new module for_each capabilities we added a
section noting that shared modules using the legacy pattern of declaring
their own provider configurations would not be compatible with them.
However, that also applies to the new module depends_on and several folks
participating in the beta pointed out that the documentation wasn't
discussing that at all.
In order to generalize the advice, I've moved the old content we had
(since v0.11) recommending against provider configurations in shared
modules out into its own section, now being more explicit that it is
a legacy pattern and not recommended, and then folded the content about
for_each and count, now also including depends_on, into that expanded
section.
As is often the case, that had some knock-on effects on the content on
the rest of this page, so there's some general editing and reorganization
here. In particular, I moved the "Multiple Instances of a Module" section
much further up the page because it's content relevant to users of
shared modules, while the later content on this page is more aimed at
authors of shared modules, including the new section about the legacy
pattern.
Part of the upgrade process happens in the first "terraform apply" after
adding explicit source addresses in the configuration. Previously we just
left that implied under the assumption that everyone would run
"terraform apply" shortly after anyway, but there is a specific tricky
situation where the first change after upgrading is to remove a resource
from the configuration, leaving Terraform unable to complete the upgrade.
Because of that, we'll now explicitly direct users to run
"terraform apply" after upgrading. Along with that, there's a reminder to
make sure that "terraform plan" indicates no changes before upgrading, so
that completing the upgrade doesn't involve also applying changes to
remote objects.
* website: Terraform Registry Provider Publishing
* website: (Registry) remove OS/arch recommendation
Until we have a canonical list to point to
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
* command/init: return an error with invalid -backend-config files
The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.
Fixes#24845
* emphasize backend configuration file in docs
* Azure backend: support snapshots/versioning
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
* Azure backend: Versioning -> Snapshot
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
Although this command is removed in Terraform 0.13, our documentation is
for all versions of Terraform that remain in common use and keeping this
documented will be helpful for folks who are still using Terraform 0.11
and planning their upgrades to Terraform 0.12.
Both of the upgrade commands now include notes that they are only
available in their specific major version, along with a link to the
relevant upgrade guide for other background information about the upgrade,
in case the user finds the command documentation first. (The command docs
are, I think, a little more discoverable than the upgrade guides.)
We previously covered everything about using providers on a single page,
but that was getting unwieldy already and we now have a lot more to
discuss with v0.13 introducing a new source address syntax and some other
concepts.
Here we split the provider-related content into two parts: "Provider
Requirements" covers how to find and declare dependencies on providers,
and then "Provider Configuration" (formerly just "Providers") then focuses
primarily on how to write zero or more provider configurations for a
particular provider.
Because "Provider Requirements" is now presented before "Provider
Configuration" in the navigation, I've also moved some of the introductory
content about providers in general onto the "Requirements" page. The
first paragraph of that content is duplicated onto the "Configuration"
page for discoverability, but we now link to the requirements page to get
the full story.
The "Configuration Language" section was becoming rather unweildy, both
by having a lot of pages and by some of the pages being quite large in
themselves.
This is a first step towards breaking things up a little more, starting
with two changes:
- The "Configuration Language" navigation is now split into two
sub-headings "Configuration Blocks" and "Syntax".
- Some of the information about sub-blocks of the "terraform" block are
now given their own pages, because their content is quite complex
in itself.
- "Version Constraints" is now a page in its own right, rather than this
content being duplicated in slightly different forms across multiple
contexts that make use of user-specified version constraints.
We previously had the module registry protocol documented only as an
undefined subset of the full API of the official registry implementation.
However, the vast majority of endpoints documented in the official API
docs are not needed for a headless third-party module registry that only
intends to make modules available to Terraform CLI.
To make this clearer to potential third-party implementors, and also for
consistency with how the provider registry protocol is now documented,
here we create a new page to describe the subset required for all
registries, and then explain in the docs for the offical API that
potential third-party implementors should refer to the new page instead.
The longer page describing the full API of the official implementations
remains for those who wish to write clients for that API, because it is
part of the API surface area for Terraform Cloud and Terraform Enterprise.
I also took this opportunity to address the fact that module addresses
don't really contain "provider names" at all, but rather than the fourth
field in the address is _conventionally_ an official provider name but
can really be any string that serves to differentiate multiple
implementations of the same abstraction. The new docs therefore refer to
this field as "system" rather than "provider".
Currently the example config for the Consul backend uses a live Consul demo cluster at `demo.consul.io`. This results in TF state with sensitive information and all being stored on a public site when users just copy and paste the config. This PR changes it so that the config address isn't the public demo cluster.
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.
This initial version is not customizable aside from being able to select
multiple platforms to install packages for.
Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.
In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
When helping folks in the community forum, I commonly see questions around
more complex patterns in transforming deep data structures into different
shapes to work with for_each. We have examples of these patterns in the
docs for the functions that they rely on, but they were not previously
very discoverable in the main configuration language documentation
sections.
Here I've moved the "Using Expressions in for_each" subsection on the
Resources page above some of the other sub-sections to hopefully make it
easier to see, and written out in more detail the two specific patterns
that answer a significant number of for_each-related user questions in
the hope that readers will be more likely to realize that the links are
relevant to what their goals.
I also added some more elaboration about the behavior of converting from
list to set in the "Using Sets" subsection, because this feature is often
a user's first encounter with the set data type and I've inferred from
some of the questions I've answered that a number of Terraform users don't
have prior experience with set data types in other languages to draw
assumptions from.
Finally, I added some similar links to the for_each patterns within the
for expression documentation itself, to try to make those examples more
visible to those who might be discovering the documentation in a different
sequence, e.g. by following a deep link shared in an answer to a question
in the community forum.
The "apply" documentation contained a simple typo, while the "plan"
documentation contained outdated information about using
"terraform plan PLANFILE" to view a plan. The latter is now a separate
command entirely, since Terraform 0.12: "terraform show PLANFILE".
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.
This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".
This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.
The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.
We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.
This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
as with this version of this doc users receives a warning like this if them use quotes with parameters of 'on_failure' setting:
"
on_failure = "continue"
In this context, keywords are expected literally rather than in quotes.
Terraform 0.11 and earlier required quotes, but quoted keywords are now
deprecated and will be removed in a future version of Terraform. Remove the
quotes surrounding this keyword to silence this warning.
"
same with:
when = "destroy"
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.
Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.
Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
* website: Edit text of new TF_IGNORE env var docs
Fixing one broken link, and tidying the sentences a bit.
* typo
Co-authored-by: Pam Selle <pam@hashicorp.com>
This example doesn't really show how these values should be used. The
default of retry_on_exit_code is now already when most people want, so
this line is not needed in most cases.
I think the docs describe the new options just fine, so lets leave this
out...
This is an initial draft of documentation for this new feature of the
CLI configuration. This is mainly intended as a placeholder for now,
because there are other documentation updates pending for the new provider
namespacing and installation scheme and we'll likely want to revise these
docs to better complement the broader documentation once it's written.
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.
Additional tests to exercise providers in modules and providers from
state have been included.
A proposed pull request to the AWS provider would change the import behavior of
`aws_security_group`. This preemptive change will help keep the docs accurate if
that gets merged.
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.
Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.
Co-authored-by: Paul Tyng <paul@paultyng.net>
Previously the templatefile function would permit any arbitrary string as
a variable name, but due to the HCL template syntax it would be impossible
to refer to one that isn't a valid HCL identifier without causing an
HCL syntax error.
The HCL syntax errors are correct, but don't really point to the root
cause of the problem. Instead, we'll pre-verify that the variable names
are valid before we even try to render the template, and given a
specialized error message that refers to the vars argument expression as
the problematic part, which will hopefully make the resolution path
clearer for a user encountering this situation.
The syntax error still remains for situations where all of the variable
names are correct but e.g. the user made a typo referring to one, which
makes sense because in that case the problem _is_ inside the template.
* add TencentCloud COS backend for remote state
* add vendor of dependence
* fixed error not handle and remove default value for prefix argument
* get appid from TF_COS_APPID environment variables
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib
* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
This guide now lives at:
- https://learn.hashicorp.com/terraform#getting-started
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
This document now lives at:
- https://learn.hashicorp.com/terraform/development/running-terraform-in-automation
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
The existing "type" argument allows specifying a type constraint that
allows for some basic validation, but often there are more constraints on
a variable value than just its type.
This new feature (requiring an experiment opt-in for now, while we refine
it) allows specifying arbitrary validation rules for any variable which
can then cause custom error messages to be returned when a caller provides
an inappropriate value.
variable "example" {
validation {
condition = var.example != "nope"
error_message = "Example value must not be \"nope\"."
}
}
The core parts of this are designed to do as little new work as possible
when no validations are specified, and thus the main new checking codepath
here can therefore only run when the experiment is enabled in order to
permit having validations.
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.
For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:
try(local.example.foo, "fallback-foo")
Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
In earlier versions of Terraform the result of terraform state show was
in the pre-0.12 "flatmap" structure that was unable to reflect nested
data structures. That was fixed in Terraform 0.12, but as a consequence
this statement about the output being machine-parseable (which was
debateable even in older versions) is incorrect.
Fortunately, we now have "terraform show -json" to get output that is
intentionally machine-parseable, so we'll recommend to use that instead
here. The JSON output of that command is a superset of what's produced by
"terraform state show", so should be usable to meet any use-case that
might previously have been met by parsing the "terraform state show"
output.
Right now, the only environment variable available is the same
environment variable that will be picked up by the GCP provider. Users
would like to be able to store state in separate projects or accounts or
otherwise authenticate to the provider with a service account that
doesn't have access to the state. This seems like a reasonable enough
practice to me, and the solution seems straightforward--offer an
environment variable that doesn't mean anything to the provider to
configure the backend credentials. I've added GOOGLE_BACKEND_CREDENTIALS
to manage just the backend credentials, and documented it appropriately.
It's a common source of errors to try to produce JSON or YAML syntax
using string concatenation via our template language but to miss some
details like correct string escaping, quoting, required commas, etc.
The jsonencode and yamlencode functions are a better way to generate JSON
and YAML, but it's not immediately obvious that both of these functions
are available for use in external templates (via templatefile) too.
Given that questions related to this come up a lot in our community forum
and elsewhere, it seems worth having a documentation section to show the
pattern of having a template that consists only of a single function call.
When warnings appear in isolation (not accompanied by an error) it's
reasonable to want to defer resolving them for a while because they are
not actually blocking immediate work.
However, our warning messages tend to be long by default in order to
include all of the necessary context to understand the implications of
the warning, and that can make them overwhelming when combined with other
output.
As a compromise, this adds a new CLI option -compact-warnings which is
supported for all the main operation commands and which uses a more
compact format to print out warnings as long as they aren't also
accompanied by errors.
The default remains unchanged except that the threshold for consolidating
warning messages is reduced to one so that we'll now only show one of
each distinct warning summary.
Full warning messages are always shown if there's at least one error
included in the diagnostic set too, because in that case the warning
message could contain additional context to help understand the error.
There are a few situations that we've seen arise quite commonly for folks
upgrading from Terraform 0.11 to 0.12. These particular problems are not
things that Terraform 0.12 can fix automatically during upgrading, but
we can at least give some better feedback to users that they ought to be
addressed _before_ upgrading.
The provider address problem is already detected and flagged by the
"terraform 0.11checklist" command that folks should run as part of their
upgrade process, but the module address problem is not something we
noticed was lacking validation in 0.11 and so the checklist tool doesn't
cover it. Due to the lack of coverage in the checklist tool, this commit
also includes an additional section in the upgrade guide that mentions
the problem and gives instructions on how to address it.
I've seen folks ask about how to express this in resource address syntax
a number of times now, so adding this example here to illustrate how it
looks when there are multiple levels of module to traverse through.
This is redundant with other information further up the page, but having
it as an entirely separate example gives an opportunity to include more
introductory text to explain what the example is showing.
There are some differences between the Terraform CLI and Terraform Cloud ideas of workspaces.
This documentation aims to explain those differences and show different patterns for configuring the remote backend and the implications of different approaches.
As mentioned in #17871 the current example can hide the fact that the module
path plays an important role. The example's explanation is expanded.
Moreover, the verb "attach" is replaced with "map" to make the vocabulary
consistent with the wording in the documentation of the terraform state.
A very common question since we launched the two repetition constructs
is how to deal with situations where the input data structure doesn't
match one-to-one with the desired configuration.
This adds some full worked examples of two common situations that have
come up in questions. To avoid adding a lot of extra content to the
already-large "expressions" and "resources" pages, the main bulk of this
new content lives with the relevant functions themselves as a full example
of one thing they are good for, and then we'll link to them from the two
general documentation sections where folks are likely to be reading when
they encounter the problem.
* upstream/master: (66 commits)
lang/eval: more evalContext fixups
Update CHANGELOG.md
Cleanup after v0.12.10 release
v0.12.10
website / help: reconcile 'validate' command docs
website: Document behavior of `self` object for provisioners
vendor: switch to HCL 2.0 in the HCL repository
Update communicator/ssh/communicator.go
copy client pointer for keep-alive loop
Update CHANGELOG.md
slow down tfce polling to 1s
typos. some code, some text.
Remove -check-variables flag from the docs
Merge cleanup, remove `license` parameter in favor of bool `accept_license`, adjust how license acceptance is done, update hab provisioner doc.
vendor latest go-tfe
clean up go mod for go-tfe
tfce test additions
update to go-tfe 0.3.23
cost estimation status polling
go-tfe dep update to 0.3.22
...
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
The cidrsubnets function signature is intentionally very low-level and
focused on the core requirement of generating addresses. This registry
module then wraps it with some additional functionality to make it more
convenient to generate and use subnet address ranges.
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.
Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
* command/import: properly use `-provider` supplied on the command line
The import command now attaches the provider configuration in the resource
instance, if set. That config is attached to the NodeAbstractResource
during the import graph building. This prevents errors when the implied
provider is not actually in the configuration at all, which may happen
when a configuration is using the `-beta` version of a provider (and
only that `-beta` version).
* command/import: fix variable reassignment and update docs
Fixes#22564
For a long time now we've been advising against the use of provisioners,
but our documentation for them is pretty prominent on the website in
comparision to the better alternatives, and so it's little surprise that
many users end up making significant use of them.
Although in the longer term a change to our information architecture would
probably address this even better, this is an attempt to be explicit about
the downsides of using provisioners and to prominently describe the
alternatives that are available for common use-cases, along with some
reasons why we consider them to be better.
I took the unusual step here of directly linking to specific provider
documentation pages about the alternatives, even though we normally try
to keep the core documentation provider-agnostic, because otherwise that
information tends to be rather buried in the provider documentation and
thus the reader would be reasonable to use provisioners just because we're
not giving specific enough alternative recommendations.
* website/formatdate: update example
The given example was showing HOUR:MONTH instead of HOUR:MINUTE
Fixes#22598
* website/import: remove reference to no-longer-working option
Users can no longer supply `-config=""` to tell Terraform not to load
configuration for import.
Fixes#22294
* website/provisioners: `host` is required in connection blocks
Fixes#21877
* website/variables: clarify variable definition precedence
It was not entirely obvious that a variable could not be assigned
multiples times in a single source.
Fixes#21682
* website/backend/local: add `workspace_dir` attribute
Fixes#21391
* website/output: `sensitive` outputs are redacted in output
Fixes#21502
* website/backends: sidebar order tweak
It makes sense for backend 'configuration' to appear before 'init'.
Fixes#13796
* Revert "website/formatdate: update example"
This reverts commit ccd93c86ddd15a21625c0767702ee1cc62e77254.
Reference: https://github.com/hashicorp/terraform/issues/16697
Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862
To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
The Terraform Enterprise brand has now been split into two parts:
- Terraform Cloud is the application that helps teams use Terraform together,
with remote state storage, a shared run environment, etc.
- Terraform Enterprise is the on-premise distribution that lets enterprises run
a private instance of the Terraform Cloud application.
The former TFE docs have been split accordingly.
- Make these descriptions more similar, since they do basically the same thing.
- Add some subheaders to break up the wall of text and make it more skimmable.
- Nudge people more firmly toward `for_each` if they need to actually
incorporate data from a variable into their instances.
- Add version note so you know whether you can use this yet.
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
We added the csvdecode function originally with the intent of it being
used with for_each, but because csvdecode was released first we had a
section in its documentation warning about the downsides of using it with
"count", since that seemed like something people would be likely to try.
With resource "for_each" now merged, we can replace that scary section
with a more positive example of using these two features together.
We still include a paragraph noting that "count" _could_ be used here, but
with a caution against doing so. This is in the hope of helping users
understand the difference between these two patterns and why for_each is
the superior choice for most situations.
Alisdair McDiarmid