* ADD CLI option position for force-unlock command
* Update force-unlock.html.markdown
Made a change to also include the missing [DIR]
Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.
"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.
The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
* Add note to upgrade guide about provider sensitivity
Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
For normal provider installation we want to associate each provider with
a selected version number and find a suitable package for that version
that conforms to the official hashes for that release.
Those requirements are very onerous for a provider developer currently
testing a not-yet-released build, though. To allow for that case this new
CLI configuration feature allows overriding specific providers to refer
to give local filesystem directories.
Any provider overridden in this way is not subject to the usual
restrictions about selected versions or checksum conformance, and
activating an override won't cause any changes to the selections recorded
in the lock file because it's intended to be a temporary setting for one
developer only.
This is, in a sense, a spiritual successor of an old capability we had to
override specific plugins in the CLI configuration file. There were
some vestiges of that left in the main package and CLI config package
but nothing has actually been honoring them for several versions now and
so this commit removes them to avoid confusion with the new mechanism.
Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.
The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.
Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.
After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:
- The local cache directory will now be .terraform/providers rather than
.terraform/plugins, because .terraform/plugins is effectively "poisoned"
by the incorrect usage that we can't reliably distinguish from prior
version correct usage.
- The .terraform/plugins directory is now the "legacy cache directory". It
is intentionally _not_ now a filesystem mirror directory, because that
would risk incorrectly interpreting providers automatically installed
by Terraform v0.13 as if they were a local mirror, and thus upgrades
and checksum fetches from the origin registry would be blocked.
- Because of the previous two points, someone who _was_ trying to use the
legacy cache directory as a filesystem mirror would see installation
fail for any providers they manually added to the legacy directory.
To avoid leaving that user stumped as to what went wrong, there's a
heuristic for the case where a non-official provider fails installation
and yet we can see it in the legacy cache directory. If that heuristic
matches then we'll produce a warning message hinting to move the
provider under the terraform.d/plugins directory, which is a _correct_
location for "bundled" provider plugins that belong only to a single
configuration (as opposed to being installed globally on a system).
This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.
The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
The upgrade requirements for this release are considerably more modest
than for Terraform v0.13, so this time we just have some notes about a
few changes in behavior that may be impactful to some users.
This first pass is intended to be included as part of a forthcoming beta
testers' guide as we begin the v0.14 beta testing period. We will make
further changes to this upgrade guide based on feedback from those who
participate in the beta process.
Note that this upgrade guide is not intended as release marketing material
and so its presentation is focused on addressing concerns users might
encounter while upgrading. We'll share highlights from the release in
other contexts, such as the changelog and in the product blog.
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.
In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.
Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
This includes both the main documentation about the lock file itself and
changes to related documentation about Terraform commands that interact
with the lock file.
We will likely continue to update this first pass of documentation as we
get feedback and questions during the prerelease period.
* website: Update all Learn crosslinks
The URL structure on Learn recently changed, so it's time to update some URLs.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Move the information about state from the "caveats" to the main
info section, using similar information to sensitive outputs.
Updates the header of the section from similar inspiration.
We can remove the caveat about changing map elements.
Add a little more text about the intended use case for ignore_changes,
as the common case of fixing erroneous provider behavior should not be
the primary motivation for the maintenance of this feature.
* Put link to tutorial in its own section, call it a tutorial instead of guide, and use new canonical URL.
* Mention limitations of using import with a remote backed
* Typo fix
Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>
This commit adds an `alltrue` function to Terraform configuration. A
reason we might want this function is because it will enable more
powerful custom variable validations. For example:
```hcl
variable "amis" {
type = list(object({
id = string
}))
validation {
condition = (alltrue([
for a in var.amis : length(a.id) > 4 && substr(a.id, 0, 4) == "ami-"
]))
error_message = "The ID of at least one AMI was invalid."
}
}
```
For a credentials helper plugin to be useful with Terraform 0.13+, we
need to cope with the case of having no credentials for a host without
this being an error. This is to allow the public Terraform Registry to
be accessed without supplying a token.
The way to implement this is to respond to queries for credentials for a
host which has no credentials stored with an empty object and a success
exit code. This contradicts the previous documentation, which calls for
an error response in this case.
* Adding not about data-sources and depends-on for 0.12 users
* Bold
* A little more markdown
* A little more markdown for data_sources in 0.12
* Some iteration based on good feedback
In addition to the directories previously listed, Terraform looks in the
CLI config directory ($HOME/.terraform.d/plugins on macOS/Linux/UNIX,
and %APPDATA%/terraform.d/plugins on Windows). List this in the
documentation for clarity.
We also add a note about the working directory relative "vendor"
location, ./terraform.d/plugins.
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
This new option is intended to address the previous inconsistencies where
some older subcommands supported partially changing the target directory
(where Terraform would use the new directory inconsistently) where newer
commands did not support that override at all.
Instead, now Terraform will accept a -chdir command at the start of the
command line (before the subcommand) and will interpret it as a request
to direct all actions that would normally be taken in the current working
directory into the target directory instead. This is similar to options
offered by some other similar tools, such as the -C option in "make".
The new option is only accepted at the start of the command line (before
the subcommand) as a way to reflect that it is a global command (not
specific to a particular subcommand) and that it takes effect _before_
executing the subcommand. This also means it'll be forced to appear before
any other command-specific arguments that take file paths, which hopefully
communicates that those other arguments are interpreted relative to the
overridden path.
As a measure of pragmatism for existing uses, the path.cwd object in
the Terraform language will continue to return the _original_ working
directory (ignoring -chdir), in case that is important in some exceptional
workflows. The path.root object gives the root module directory, which
will always match the overriden working directory unless the user
simultaneously uses one of the legacy directory override arguments, which
is not a pattern we intend to support in the long run.
As a first step down the deprecation path, this commit adjusts the
documentation to de-emphasize the inconsistent old command line arguments,
including specific guidance on what to use instead for the main three
workflow commands, but all of those options remain supported in the same
way as they were before. In a later commit we'll make those arguments
produce a visible deprecation warning in Terraform's output, and then
in an even later commit we'll remove them entirely so that -chdir is the
single supported way to run Terraform from a directory other than the
one containing the root module configuration.
The subtle difference in keywords when creating vs. accessing locals trips
people up, even more than the "variable" vs. "var" distinction. It deserves its
own subheader on the page, plus a nice noisy callout.
I've just wasted an hour to two hours trying to find the problem to finally realize that although I declare a "locals" block, it's referred to as "local". This is pretty weird! So let's be be clear about this.
The error diagnostic shown when legacy state contains resources from
in-house providers has changed, so update references to it in the 0.13
upgrade guide.
We previously had this just stubbed out because it was a stretch goal for
the v0.13.0 release and it ultimately didn't make it in.
Here we fill out the existing stub -- with a minor change to its interface
so it can access credentials -- with a client implementation that is
compatible with the directory structure produced by the
"terraform providers mirror" subcommand, were the result to be published
on a static file server.
- Edits to registry overview
- Add index link as 'overview' (header links are semi-invisible)
- move providers/overview.html to providers/index.html
- Edits to providers overview
- fix filename of os-arch
- edits to provider publishing
Terraform's design assumes that each remote object in Terraform's care is
bound to one resource instance and one alone. If the same object is bound
to multiple instances then confusing behavior will often result, such as
two resource configurations competing to update a single object, or
objects being "left behind" when all existing Terraform deployments are
destroyed.
This assumption was previously only implied, though. This change is an
attempt to be more explicit about it, although these are additions to some
older documentation sections that have not been revised for some time and
so this is just a best effort to make this information discoverable
without getting drawn into a full-on reorganization of these sections.
While revising this there were some particular oddities that I decided to
revise while I was there, but I'll leave a fuller revision of this older
content for a later commit when we have more time to review it in greater
detail.
* Make sidebar nav in language docs more intuitive
* Minor display fixes for registry docs
* Explain providers in the registry in the providers index
* Revise a bunch of language docs around provider reqs
This is mostly an effort to smooth out some of the explanations, make sure
things are presented in a helpful order, make sure terminology lines up, draw
connections between related concepts, make default behavior more apparent, and
the like. It shouldn't include very much new information, but there might be one
or two things that came out of a conversation somewhere.
Co-authored-by: Judith Malnick <judith@hashicorp.com>
As part of documenting the new module for_each capabilities we added a
section noting that shared modules using the legacy pattern of declaring
their own provider configurations would not be compatible with them.
However, that also applies to the new module depends_on and several folks
participating in the beta pointed out that the documentation wasn't
discussing that at all.
In order to generalize the advice, I've moved the old content we had
(since v0.11) recommending against provider configurations in shared
modules out into its own section, now being more explicit that it is
a legacy pattern and not recommended, and then folded the content about
for_each and count, now also including depends_on, into that expanded
section.
As is often the case, that had some knock-on effects on the content on
the rest of this page, so there's some general editing and reorganization
here. In particular, I moved the "Multiple Instances of a Module" section
much further up the page because it's content relevant to users of
shared modules, while the later content on this page is more aimed at
authors of shared modules, including the new section about the legacy
pattern.
Part of the upgrade process happens in the first "terraform apply" after
adding explicit source addresses in the configuration. Previously we just
left that implied under the assumption that everyone would run
"terraform apply" shortly after anyway, but there is a specific tricky
situation where the first change after upgrading is to remove a resource
from the configuration, leaving Terraform unable to complete the upgrade.
Because of that, we'll now explicitly direct users to run
"terraform apply" after upgrading. Along with that, there's a reminder to
make sure that "terraform plan" indicates no changes before upgrading, so
that completing the upgrade doesn't involve also applying changes to
remote objects.
* website: Terraform Registry Provider Publishing
* website: (Registry) remove OS/arch recommendation
Until we have a canonical list to point to
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
* command/init: return an error with invalid -backend-config files
The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.
Fixes#24845
* emphasize backend configuration file in docs
* Azure backend: support snapshots/versioning
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
* Azure backend: Versioning -> Snapshot
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
Although this command is removed in Terraform 0.13, our documentation is
for all versions of Terraform that remain in common use and keeping this
documented will be helpful for folks who are still using Terraform 0.11
and planning their upgrades to Terraform 0.12.
Both of the upgrade commands now include notes that they are only
available in their specific major version, along with a link to the
relevant upgrade guide for other background information about the upgrade,
in case the user finds the command documentation first. (The command docs
are, I think, a little more discoverable than the upgrade guides.)
We previously covered everything about using providers on a single page,
but that was getting unwieldy already and we now have a lot more to
discuss with v0.13 introducing a new source address syntax and some other
concepts.
Here we split the provider-related content into two parts: "Provider
Requirements" covers how to find and declare dependencies on providers,
and then "Provider Configuration" (formerly just "Providers") then focuses
primarily on how to write zero or more provider configurations for a
particular provider.
Because "Provider Requirements" is now presented before "Provider
Configuration" in the navigation, I've also moved some of the introductory
content about providers in general onto the "Requirements" page. The
first paragraph of that content is duplicated onto the "Configuration"
page for discoverability, but we now link to the requirements page to get
the full story.
The "Configuration Language" section was becoming rather unweildy, both
by having a lot of pages and by some of the pages being quite large in
themselves.
This is a first step towards breaking things up a little more, starting
with two changes:
- The "Configuration Language" navigation is now split into two
sub-headings "Configuration Blocks" and "Syntax".
- Some of the information about sub-blocks of the "terraform" block are
now given their own pages, because their content is quite complex
in itself.
- "Version Constraints" is now a page in its own right, rather than this
content being duplicated in slightly different forms across multiple
contexts that make use of user-specified version constraints.
We previously had the module registry protocol documented only as an
undefined subset of the full API of the official registry implementation.
However, the vast majority of endpoints documented in the official API
docs are not needed for a headless third-party module registry that only
intends to make modules available to Terraform CLI.
To make this clearer to potential third-party implementors, and also for
consistency with how the provider registry protocol is now documented,
here we create a new page to describe the subset required for all
registries, and then explain in the docs for the offical API that
potential third-party implementors should refer to the new page instead.
The longer page describing the full API of the official implementations
remains for those who wish to write clients for that API, because it is
part of the API surface area for Terraform Cloud and Terraform Enterprise.
I also took this opportunity to address the fact that module addresses
don't really contain "provider names" at all, but rather than the fourth
field in the address is _conventionally_ an official provider name but
can really be any string that serves to differentiate multiple
implementations of the same abstraction. The new docs therefore refer to
this field as "system" rather than "provider".
Currently the example config for the Consul backend uses a live Consul demo cluster at `demo.consul.io`. This results in TF state with sensitive information and all being stored on a public site when users just copy and paste the config. This PR changes it so that the config address isn't the public demo cluster.
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.
This initial version is not customizable aside from being able to select
multiple platforms to install packages for.
Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.
In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
When helping folks in the community forum, I commonly see questions around
more complex patterns in transforming deep data structures into different
shapes to work with for_each. We have examples of these patterns in the
docs for the functions that they rely on, but they were not previously
very discoverable in the main configuration language documentation
sections.
Here I've moved the "Using Expressions in for_each" subsection on the
Resources page above some of the other sub-sections to hopefully make it
easier to see, and written out in more detail the two specific patterns
that answer a significant number of for_each-related user questions in
the hope that readers will be more likely to realize that the links are
relevant to what their goals.
I also added some more elaboration about the behavior of converting from
list to set in the "Using Sets" subsection, because this feature is often
a user's first encounter with the set data type and I've inferred from
some of the questions I've answered that a number of Terraform users don't
have prior experience with set data types in other languages to draw
assumptions from.
Finally, I added some similar links to the for_each patterns within the
for expression documentation itself, to try to make those examples more
visible to those who might be discovering the documentation in a different
sequence, e.g. by following a deep link shared in an answer to a question
in the community forum.
The "apply" documentation contained a simple typo, while the "plan"
documentation contained outdated information about using
"terraform plan PLANFILE" to view a plan. The latter is now a separate
command entirely, since Terraform 0.12: "terraform show PLANFILE".
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.
This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".
This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.
The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.
We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.
This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
as with this version of this doc users receives a warning like this if them use quotes with parameters of 'on_failure' setting:
"
on_failure = "continue"
In this context, keywords are expected literally rather than in quotes.
Terraform 0.11 and earlier required quotes, but quoted keywords are now
deprecated and will be removed in a future version of Terraform. Remove the
quotes surrounding this keyword to silence this warning.
"
same with:
when = "destroy"
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.
Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.
Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
* website: Edit text of new TF_IGNORE env var docs
Fixing one broken link, and tidying the sentences a bit.
* typo
Co-authored-by: Pam Selle <pam@hashicorp.com>
This example doesn't really show how these values should be used. The
default of retry_on_exit_code is now already when most people want, so
this line is not needed in most cases.
I think the docs describe the new options just fine, so lets leave this
out...
This is an initial draft of documentation for this new feature of the
CLI configuration. This is mainly intended as a placeholder for now,
because there are other documentation updates pending for the new provider
namespacing and installation scheme and we'll likely want to revise these
docs to better complement the broader documentation once it's written.
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.
Additional tests to exercise providers in modules and providers from
state have been included.
A proposed pull request to the AWS provider would change the import behavior of
`aws_security_group`. This preemptive change will help keep the docs accurate if
that gets merged.
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.
Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.
Co-authored-by: Paul Tyng <paul@paultyng.net>
Previously the templatefile function would permit any arbitrary string as
a variable name, but due to the HCL template syntax it would be impossible
to refer to one that isn't a valid HCL identifier without causing an
HCL syntax error.
The HCL syntax errors are correct, but don't really point to the root
cause of the problem. Instead, we'll pre-verify that the variable names
are valid before we even try to render the template, and given a
specialized error message that refers to the vars argument expression as
the problematic part, which will hopefully make the resolution path
clearer for a user encountering this situation.
The syntax error still remains for situations where all of the variable
names are correct but e.g. the user made a typo referring to one, which
makes sense because in that case the problem _is_ inside the template.
* add TencentCloud COS backend for remote state
* add vendor of dependence
* fixed error not handle and remove default value for prefix argument
* get appid from TF_COS_APPID environment variables
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib
* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
This guide now lives at:
- https://learn.hashicorp.com/terraform#getting-started
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
This document now lives at:
- https://learn.hashicorp.com/terraform/development/running-terraform-in-automation
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
The existing "type" argument allows specifying a type constraint that
allows for some basic validation, but often there are more constraints on
a variable value than just its type.
This new feature (requiring an experiment opt-in for now, while we refine
it) allows specifying arbitrary validation rules for any variable which
can then cause custom error messages to be returned when a caller provides
an inappropriate value.
variable "example" {
validation {
condition = var.example != "nope"
error_message = "Example value must not be \"nope\"."
}
}
The core parts of this are designed to do as little new work as possible
when no validations are specified, and thus the main new checking codepath
here can therefore only run when the experiment is enabled in order to
permit having validations.
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.
For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:
try(local.example.foo, "fallback-foo")
Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
In earlier versions of Terraform the result of terraform state show was
in the pre-0.12 "flatmap" structure that was unable to reflect nested
data structures. That was fixed in Terraform 0.12, but as a consequence
this statement about the output being machine-parseable (which was
debateable even in older versions) is incorrect.
Fortunately, we now have "terraform show -json" to get output that is
intentionally machine-parseable, so we'll recommend to use that instead
here. The JSON output of that command is a superset of what's produced by
"terraform state show", so should be usable to meet any use-case that
might previously have been met by parsing the "terraform state show"
output.
Right now, the only environment variable available is the same
environment variable that will be picked up by the GCP provider. Users
would like to be able to store state in separate projects or accounts or
otherwise authenticate to the provider with a service account that
doesn't have access to the state. This seems like a reasonable enough
practice to me, and the solution seems straightforward--offer an
environment variable that doesn't mean anything to the provider to
configure the backend credentials. I've added GOOGLE_BACKEND_CREDENTIALS
to manage just the backend credentials, and documented it appropriately.
It's a common source of errors to try to produce JSON or YAML syntax
using string concatenation via our template language but to miss some
details like correct string escaping, quoting, required commas, etc.
The jsonencode and yamlencode functions are a better way to generate JSON
and YAML, but it's not immediately obvious that both of these functions
are available for use in external templates (via templatefile) too.
Given that questions related to this come up a lot in our community forum
and elsewhere, it seems worth having a documentation section to show the
pattern of having a template that consists only of a single function call.
When warnings appear in isolation (not accompanied by an error) it's
reasonable to want to defer resolving them for a while because they are
not actually blocking immediate work.
However, our warning messages tend to be long by default in order to
include all of the necessary context to understand the implications of
the warning, and that can make them overwhelming when combined with other
output.
As a compromise, this adds a new CLI option -compact-warnings which is
supported for all the main operation commands and which uses a more
compact format to print out warnings as long as they aren't also
accompanied by errors.
The default remains unchanged except that the threshold for consolidating
warning messages is reduced to one so that we'll now only show one of
each distinct warning summary.
Full warning messages are always shown if there's at least one error
included in the diagnostic set too, because in that case the warning
message could contain additional context to help understand the error.
There are a few situations that we've seen arise quite commonly for folks
upgrading from Terraform 0.11 to 0.12. These particular problems are not
things that Terraform 0.12 can fix automatically during upgrading, but
we can at least give some better feedback to users that they ought to be
addressed _before_ upgrading.
The provider address problem is already detected and flagged by the
"terraform 0.11checklist" command that folks should run as part of their
upgrade process, but the module address problem is not something we
noticed was lacking validation in 0.11 and so the checklist tool doesn't
cover it. Due to the lack of coverage in the checklist tool, this commit
also includes an additional section in the upgrade guide that mentions
the problem and gives instructions on how to address it.
I've seen folks ask about how to express this in resource address syntax
a number of times now, so adding this example here to illustrate how it
looks when there are multiple levels of module to traverse through.
This is redundant with other information further up the page, but having
it as an entirely separate example gives an opportunity to include more
introductory text to explain what the example is showing.
There are some differences between the Terraform CLI and Terraform Cloud ideas of workspaces.
This documentation aims to explain those differences and show different patterns for configuring the remote backend and the implications of different approaches.
As mentioned in #17871 the current example can hide the fact that the module
path plays an important role. The example's explanation is expanded.
Moreover, the verb "attach" is replaced with "map" to make the vocabulary
consistent with the wording in the documentation of the terraform state.
A very common question since we launched the two repetition constructs
is how to deal with situations where the input data structure doesn't
match one-to-one with the desired configuration.
This adds some full worked examples of two common situations that have
come up in questions. To avoid adding a lot of extra content to the
already-large "expressions" and "resources" pages, the main bulk of this
new content lives with the relevant functions themselves as a full example
of one thing they are good for, and then we'll link to them from the two
general documentation sections where folks are likely to be reading when
they encounter the problem.
* upstream/master: (66 commits)
lang/eval: more evalContext fixups
Update CHANGELOG.md
Cleanup after v0.12.10 release
v0.12.10
website / help: reconcile 'validate' command docs
website: Document behavior of `self` object for provisioners
vendor: switch to HCL 2.0 in the HCL repository
Update communicator/ssh/communicator.go
copy client pointer for keep-alive loop
Update CHANGELOG.md
slow down tfce polling to 1s
typos. some code, some text.
Remove -check-variables flag from the docs
Merge cleanup, remove `license` parameter in favor of bool `accept_license`, adjust how license acceptance is done, update hab provisioner doc.
vendor latest go-tfe
clean up go mod for go-tfe
tfce test additions
update to go-tfe 0.3.23
cost estimation status polling
go-tfe dep update to 0.3.22
...
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
The cidrsubnets function signature is intentionally very low-level and
focused on the core requirement of generating addresses. This registry
module then wraps it with some additional functionality to make it more
convenient to generate and use subnet address ranges.
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.
Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
* command/import: properly use `-provider` supplied on the command line
The import command now attaches the provider configuration in the resource
instance, if set. That config is attached to the NodeAbstractResource
during the import graph building. This prevents errors when the implied
provider is not actually in the configuration at all, which may happen
when a configuration is using the `-beta` version of a provider (and
only that `-beta` version).
* command/import: fix variable reassignment and update docs
Fixes#22564
For a long time now we've been advising against the use of provisioners,
but our documentation for them is pretty prominent on the website in
comparision to the better alternatives, and so it's little surprise that
many users end up making significant use of them.
Although in the longer term a change to our information architecture would
probably address this even better, this is an attempt to be explicit about
the downsides of using provisioners and to prominently describe the
alternatives that are available for common use-cases, along with some
reasons why we consider them to be better.
I took the unusual step here of directly linking to specific provider
documentation pages about the alternatives, even though we normally try
to keep the core documentation provider-agnostic, because otherwise that
information tends to be rather buried in the provider documentation and
thus the reader would be reasonable to use provisioners just because we're
not giving specific enough alternative recommendations.
* website/formatdate: update example
The given example was showing HOUR:MONTH instead of HOUR:MINUTE
Fixes#22598
* website/import: remove reference to no-longer-working option
Users can no longer supply `-config=""` to tell Terraform not to load
configuration for import.
Fixes#22294
* website/provisioners: `host` is required in connection blocks
Fixes#21877
* website/variables: clarify variable definition precedence
It was not entirely obvious that a variable could not be assigned
multiples times in a single source.
Fixes#21682
* website/backend/local: add `workspace_dir` attribute
Fixes#21391
* website/output: `sensitive` outputs are redacted in output
Fixes#21502
* website/backends: sidebar order tweak
It makes sense for backend 'configuration' to appear before 'init'.
Fixes#13796
* Revert "website/formatdate: update example"
This reverts commit ccd93c86ddd15a21625c0767702ee1cc62e77254.
Reference: https://github.com/hashicorp/terraform/issues/16697
Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862
To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
The Terraform Enterprise brand has now been split into two parts:
- Terraform Cloud is the application that helps teams use Terraform together,
with remote state storage, a shared run environment, etc.
- Terraform Enterprise is the on-premise distribution that lets enterprises run
a private instance of the Terraform Cloud application.
The former TFE docs have been split accordingly.
- Make these descriptions more similar, since they do basically the same thing.
- Add some subheaders to break up the wall of text and make it more skimmable.
- Nudge people more firmly toward `for_each` if they need to actually
incorporate data from a variable into their instances.
- Add version note so you know whether you can use this yet.
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
We added the csvdecode function originally with the intent of it being
used with for_each, but because csvdecode was released first we had a
section in its documentation warning about the downsides of using it with
"count", since that seemed like something people would be likely to try.
With resource "for_each" now merged, we can replace that scary section
with a more positive example of using these two features together.
We still include a paragraph noting that "count" _could_ be used here, but
with a caution against doing so. This is in the hope of helping users
understand the difference between these two patterns and why for_each is
the superior choice for most situations.
Ran into an issue requiring `-var='foo=bar'` over `-var 'foo=bar'`.
Without the `=` the generic response came back:
```
Usage: terraform plan [options] [DIR]
Generates an execution plan for Terraform.
This execution plan can be reviewed...
```
Terraform `0.12.2`
Team tokens never worked with the `atlas` backend, but the `remote` backend
uses them as intended; they can perform plans and applies on workspaces where
the associated team has at least plan or write permissions, respectively.
The search "terraform leading zero" does not find the `format()`
function, which is perfectly capable of adding leading zeros.
Thus I have added this one word to help people find `format()`.
The correct environment variable corresponding to the `ca_file` variable is `CONSUL_CACERT` and not `CONSUL_CAFILE`.
See `backend/remote-state/consul/backend.go` line 77.
This also includes a previously-missing test that verifies the behavior
described here, implemented as a planning context test for consistency
with how the other ignore_changes tests are handled.
* Correct fmt -check
With `-check=false` the exit status is always zero.
With `-check=true` the exit status is zero when all files are properly formatted and non-zero otherwise.
* update fmt documentation to use short form for -diff and -check
We previously had some notes about handling configuration variants just
tacked on to the "dependency inversion" section as an afterthought, but
this idea is a major use-case for dependency inversion so it deserves its
own section and a specific example.
There have been a few questions about this so far which indicated that the
previous docs for this feature were very lacking. This is an attempt to
describe more completely what "any" means, and in particular that it isn't
actually a type at all but rather a placeholder for a type to be selected
dynamically.
Based on some common questions and feedback since the v0.12.0 release,
here we add some small additional content to the documentation for
"dynamic" blocks, covering how to access the keys of the collection being
iterated over and how to fold multiple collections into a single one to
achieve the effect of a nested iteration.
These follow the same principle as jsondecode and jsonencode, but use
YAML instead of JSON.
YAML has a much more complex information model than JSON, so we can only
support a subset of it during decoding, but hopefully the subset supported
here is a useful one.
Because there are many different ways to _generate_ YAML, the yamlencode
function is forced to make some decisions, and those decisions are likely
to affect compatibility with other real-world YAML parsers. Although the
format here is intended to be generic and compatible, we may find that
there are problems with it that'll we'll want to adjust for in a future
release, so yamlencode is therefore marked as experimental for now until
the underlying library is ready to commit to ongoing byte-for-byte
compatibility in serialization.
The main use-case here is met by yamldecode, which will allow reading in
files written in YAML format by humans for use in Terraform modules, in
situations where a higher-level input format than direct Terraform
language declarations is helpful.
This is similar to the function of the same name in Python, generating a
sequence of numbers as a list that can then be used in other
sequence-oriented operations.
The primary use-case for it is to turn a count expressed as a number into
a list of that length, which can then be iterated over or passed to a
collection function to produce that number of something else, as shown
in the example at the end of its documentation page.
Using az login and then terraform init from the command line I got `Error: Either an Access Key / SAS Token or the Resource Group for the Storage Account must be specified`
A longer-form guide will follow in the Sentinel section of the Terraform
Enterprise documentation, once it's ready. For now, this section isn't
saying anything useful since it was always just a stub for a guide we
planned to write later.
The upgrade tool is assuming that a type of "list" means list(string) and
a type of "map" means map(string), because that was what we documented
those as meaning.
In practice, Terraform 0.11 was lacking some validation which allowed
more complex nested structures in some cases even though they were pretty
inconvenient to use due to other language limitations.
The upgrade tool doesn't have enough context to make a reliable decision
on this, so instead we'll rely on the upgrade guide for this. We don't
need a TF-UPGRADE-TODO comment in this case because we reserve those for
things where a subsequent operation might cause the configuration to be
misinterpred, rather than just causing an error. Instead, we'll show an
example of the comment in the upgrade guide so the reader can easily
match it, and give some advice in the guide on how to address it.
We've seen in the past that some users try to use this form with the
ssh:// URL prefix, so we'll mention explicitly that this is invalid and
show a working example of how to use it without the URL scheme prefix.
Our original upgrade guide was drafted while some things were still in
flux and not all of our upgrade tooling was in place yet.
This redraft now attempts to be more specific and direct, showing exact
commands to run and (where relevant) exact error messages that Terraform
might return.
I also took this opportunity for some general copy-editing, though we'll
probably want to do one more pass of that alone (without changing any
content at the same time) before final release.
This new content presumes the existence of a Terraform v0.11.14 release,
which isn't published yet at the time of writing but should be published
before v0.12.0 final, once we've done final verification and review of
the upgrade path including it.
- Note that we intentionally omitted it from the sidebar, to reduce confusion.
- Write a summary up top so you can stop reading sooner if you don't actually need this.
* lang/funcs: testing of functions through the lang package API
The function-specific unit tests do not cover the HCL conversion that happens when the functions are called in a terraform configuration. For e.g., HCL converts sets to lists before passing it to the function. This means that we could not test passing a set in the function _unit_ tests.
This adds a higher-level acceptance test, plus a check that every (pure) function has a test.
* website/docs: update function documentation
There was some leftover v0.11-style interpolation syntax here.
We prefer to use a "naked" expression in situations like this where the result
isn't a string, because interpolations returning non-strings is a common source
of confusion for new users.
The division operator now always performs floating point division, whereas
before it would choose between float and int division based on the types
of its arguments.
We have a specific error message for when a fractional number is used as
an index in HCL, but this additional upgrade guidance provides a specific
solution to the problem: the floor function.
Sadly we don't have enough context in the current design of the upgrade
tool to make this fix automatic. With some refactoring it may be possible
to apply the fix automatically within list brackets, but since that is
a relatively complex change we'll first try this manual solution prompted
by an error message, because in practice so far we've seen this reported
only in the context of list indexing and our error check will catch that
and make the user aware of the need for a fix there.
* funcs/coalesce: return the first non-null, non-empty element from a
sequence.
The go-cty coalesce function, which was originally used here, returns the
first non-null element from a sequence. Terraform 0.11's coalesce,
however, returns the first non-empty string from a list of strings.
This new coalesce function aims to preserve terraform's documented
functionality while adding support for additional argument types. The
tests include those in go-cty and adapted tests from the 0.11 version of
coalesce.
* website/docs: update coalesce function document
The re-introduction of some of the ambiguity between argument and nested
block syntax (for compatibility with existing provider patterns)
unfortunately leads to some interesting consequences for attributes using
this mode.
While the behavior is generally as before in straightforward cases, this
page aims to spell out some of the different usage patterns explicitly
for the benefit of those writing more complex configurations, such as
generic re-usable modules where using argument vs. block syntax leads to
some real differences.
This page is intentionally not linked from anywhere in the part of the
website maintained in the Terraform repository. Instead, it can be linked
from the provider documentation for any argument where this pattern is
used, to help users understand the ways in which that argument might
deviate from the usual behaviors of arguments vs. nested blocks.
Some users are not accustomed to thinking of IP addresses in a bitwise
fashion, so the hope here is to give enough of an introduction to that way
of thinking for the reader to understand what the "newbits" and "netnum"
arguments represent.
The definition of split was referring the built-in function join. However, join is just one of the ways a string might have been created, and this could cause confusion.
In the terraform state documentation the verb "map" is widely used to
describe the relationship between an item in the state and in the real world
whereas the verb "attach" is not used anywhere.
For 0.11 I just specified the naming rules; for 0.12, I added some info about
referencing values and tightened up the layout of the optional arguments.
This commit also syncs up descriptions of `depends_on`.
Diod FR