Currently the example config for the Consul backend uses a live Consul demo cluster at `demo.consul.io`. This results in TF state with sensitive information and all being stored on a public site when users just copy and paste the config. This PR changes it so that the config address isn't the public demo cluster.
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.
This initial version is not customizable aside from being able to select
multiple platforms to install packages for.
Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.
In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
When helping folks in the community forum, I commonly see questions around
more complex patterns in transforming deep data structures into different
shapes to work with for_each. We have examples of these patterns in the
docs for the functions that they rely on, but they were not previously
very discoverable in the main configuration language documentation
sections.
Here I've moved the "Using Expressions in for_each" subsection on the
Resources page above some of the other sub-sections to hopefully make it
easier to see, and written out in more detail the two specific patterns
that answer a significant number of for_each-related user questions in
the hope that readers will be more likely to realize that the links are
relevant to what their goals.
I also added some more elaboration about the behavior of converting from
list to set in the "Using Sets" subsection, because this feature is often
a user's first encounter with the set data type and I've inferred from
some of the questions I've answered that a number of Terraform users don't
have prior experience with set data types in other languages to draw
assumptions from.
Finally, I added some similar links to the for_each patterns within the
for expression documentation itself, to try to make those examples more
visible to those who might be discovering the documentation in a different
sequence, e.g. by following a deep link shared in an answer to a question
in the community forum.
The "apply" documentation contained a simple typo, while the "plan"
documentation contained outdated information about using
"terraform plan PLANFILE" to view a plan. The latter is now a separate
command entirely, since Terraform 0.12: "terraform show PLANFILE".
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.
This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".
This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.
The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.
We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.
This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
as with this version of this doc users receives a warning like this if them use quotes with parameters of 'on_failure' setting:
"
on_failure = "continue"
In this context, keywords are expected literally rather than in quotes.
Terraform 0.11 and earlier required quotes, but quoted keywords are now
deprecated and will be removed in a future version of Terraform. Remove the
quotes surrounding this keyword to silence this warning.
"
same with:
when = "destroy"
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.
Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.
Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
* website: Edit text of new TF_IGNORE env var docs
Fixing one broken link, and tidying the sentences a bit.
* typo
Co-authored-by: Pam Selle <pam@hashicorp.com>
This example doesn't really show how these values should be used. The
default of retry_on_exit_code is now already when most people want, so
this line is not needed in most cases.
I think the docs describe the new options just fine, so lets leave this
out...
This is an initial draft of documentation for this new feature of the
CLI configuration. This is mainly intended as a placeholder for now,
because there are other documentation updates pending for the new provider
namespacing and installation scheme and we'll likely want to revise these
docs to better complement the broader documentation once it's written.
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.
Additional tests to exercise providers in modules and providers from
state have been included.
A proposed pull request to the AWS provider would change the import behavior of
`aws_security_group`. This preemptive change will help keep the docs accurate if
that gets merged.
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.
Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.
Co-authored-by: Paul Tyng <paul@paultyng.net>
Previously the templatefile function would permit any arbitrary string as
a variable name, but due to the HCL template syntax it would be impossible
to refer to one that isn't a valid HCL identifier without causing an
HCL syntax error.
The HCL syntax errors are correct, but don't really point to the root
cause of the problem. Instead, we'll pre-verify that the variable names
are valid before we even try to render the template, and given a
specialized error message that refers to the vars argument expression as
the problematic part, which will hopefully make the resolution path
clearer for a user encountering this situation.
The syntax error still remains for situations where all of the variable
names are correct but e.g. the user made a typo referring to one, which
makes sense because in that case the problem _is_ inside the template.
* add TencentCloud COS backend for remote state
* add vendor of dependence
* fixed error not handle and remove default value for prefix argument
* get appid from TF_COS_APPID environment variables
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib
* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
This guide now lives at:
- https://learn.hashicorp.com/terraform#getting-started
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
This document now lives at:
- https://learn.hashicorp.com/terraform/development/running-terraform-in-automation
...and terraform.io has been redirecting to there for quite a while. This commit
removes the extra copy so that the text of the two versions doesn't drift, and
updates existing links to point to the new location.
The existing "type" argument allows specifying a type constraint that
allows for some basic validation, but often there are more constraints on
a variable value than just its type.
This new feature (requiring an experiment opt-in for now, while we refine
it) allows specifying arbitrary validation rules for any variable which
can then cause custom error messages to be returned when a caller provides
an inappropriate value.
variable "example" {
validation {
condition = var.example != "nope"
error_message = "Example value must not be \"nope\"."
}
}
The core parts of this are designed to do as little new work as possible
when no validations are specified, and thus the main new checking codepath
here can therefore only run when the experiment is enabled in order to
permit having validations.
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.
For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:
try(local.example.foo, "fallback-foo")
Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
In earlier versions of Terraform the result of terraform state show was
in the pre-0.12 "flatmap" structure that was unable to reflect nested
data structures. That was fixed in Terraform 0.12, but as a consequence
this statement about the output being machine-parseable (which was
debateable even in older versions) is incorrect.
Fortunately, we now have "terraform show -json" to get output that is
intentionally machine-parseable, so we'll recommend to use that instead
here. The JSON output of that command is a superset of what's produced by
"terraform state show", so should be usable to meet any use-case that
might previously have been met by parsing the "terraform state show"
output.
Right now, the only environment variable available is the same
environment variable that will be picked up by the GCP provider. Users
would like to be able to store state in separate projects or accounts or
otherwise authenticate to the provider with a service account that
doesn't have access to the state. This seems like a reasonable enough
practice to me, and the solution seems straightforward--offer an
environment variable that doesn't mean anything to the provider to
configure the backend credentials. I've added GOOGLE_BACKEND_CREDENTIALS
to manage just the backend credentials, and documented it appropriately.
It's a common source of errors to try to produce JSON or YAML syntax
using string concatenation via our template language but to miss some
details like correct string escaping, quoting, required commas, etc.
The jsonencode and yamlencode functions are a better way to generate JSON
and YAML, but it's not immediately obvious that both of these functions
are available for use in external templates (via templatefile) too.
Given that questions related to this come up a lot in our community forum
and elsewhere, it seems worth having a documentation section to show the
pattern of having a template that consists only of a single function call.
When warnings appear in isolation (not accompanied by an error) it's
reasonable to want to defer resolving them for a while because they are
not actually blocking immediate work.
However, our warning messages tend to be long by default in order to
include all of the necessary context to understand the implications of
the warning, and that can make them overwhelming when combined with other
output.
As a compromise, this adds a new CLI option -compact-warnings which is
supported for all the main operation commands and which uses a more
compact format to print out warnings as long as they aren't also
accompanied by errors.
The default remains unchanged except that the threshold for consolidating
warning messages is reduced to one so that we'll now only show one of
each distinct warning summary.
Full warning messages are always shown if there's at least one error
included in the diagnostic set too, because in that case the warning
message could contain additional context to help understand the error.
There are a few situations that we've seen arise quite commonly for folks
upgrading from Terraform 0.11 to 0.12. These particular problems are not
things that Terraform 0.12 can fix automatically during upgrading, but
we can at least give some better feedback to users that they ought to be
addressed _before_ upgrading.
The provider address problem is already detected and flagged by the
"terraform 0.11checklist" command that folks should run as part of their
upgrade process, but the module address problem is not something we
noticed was lacking validation in 0.11 and so the checklist tool doesn't
cover it. Due to the lack of coverage in the checklist tool, this commit
also includes an additional section in the upgrade guide that mentions
the problem and gives instructions on how to address it.
I've seen folks ask about how to express this in resource address syntax
a number of times now, so adding this example here to illustrate how it
looks when there are multiple levels of module to traverse through.
This is redundant with other information further up the page, but having
it as an entirely separate example gives an opportunity to include more
introductory text to explain what the example is showing.
There are some differences between the Terraform CLI and Terraform Cloud ideas of workspaces.
This documentation aims to explain those differences and show different patterns for configuring the remote backend and the implications of different approaches.
As mentioned in #17871 the current example can hide the fact that the module
path plays an important role. The example's explanation is expanded.
Moreover, the verb "attach" is replaced with "map" to make the vocabulary
consistent with the wording in the documentation of the terraform state.
A very common question since we launched the two repetition constructs
is how to deal with situations where the input data structure doesn't
match one-to-one with the desired configuration.
This adds some full worked examples of two common situations that have
come up in questions. To avoid adding a lot of extra content to the
already-large "expressions" and "resources" pages, the main bulk of this
new content lives with the relevant functions themselves as a full example
of one thing they are good for, and then we'll link to them from the two
general documentation sections where folks are likely to be reading when
they encounter the problem.
* upstream/master: (66 commits)
lang/eval: more evalContext fixups
Update CHANGELOG.md
Cleanup after v0.12.10 release
v0.12.10
website / help: reconcile 'validate' command docs
website: Document behavior of `self` object for provisioners
vendor: switch to HCL 2.0 in the HCL repository
Update communicator/ssh/communicator.go
copy client pointer for keep-alive loop
Update CHANGELOG.md
slow down tfce polling to 1s
typos. some code, some text.
Remove -check-variables flag from the docs
Merge cleanup, remove `license` parameter in favor of bool `accept_license`, adjust how license acceptance is done, update hab provisioner doc.
vendor latest go-tfe
clean up go mod for go-tfe
tfce test additions
update to go-tfe 0.3.23
cost estimation status polling
go-tfe dep update to 0.3.22
...
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
The cidrsubnets function signature is intentionally very low-level and
focused on the core requirement of generating addresses. This registry
module then wraps it with some additional functionality to make it more
convenient to generate and use subnet address ranges.
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.
Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
* command/import: properly use `-provider` supplied on the command line
The import command now attaches the provider configuration in the resource
instance, if set. That config is attached to the NodeAbstractResource
during the import graph building. This prevents errors when the implied
provider is not actually in the configuration at all, which may happen
when a configuration is using the `-beta` version of a provider (and
only that `-beta` version).
* command/import: fix variable reassignment and update docs
Fixes#22564
For a long time now we've been advising against the use of provisioners,
but our documentation for them is pretty prominent on the website in
comparision to the better alternatives, and so it's little surprise that
many users end up making significant use of them.
Although in the longer term a change to our information architecture would
probably address this even better, this is an attempt to be explicit about
the downsides of using provisioners and to prominently describe the
alternatives that are available for common use-cases, along with some
reasons why we consider them to be better.
I took the unusual step here of directly linking to specific provider
documentation pages about the alternatives, even though we normally try
to keep the core documentation provider-agnostic, because otherwise that
information tends to be rather buried in the provider documentation and
thus the reader would be reasonable to use provisioners just because we're
not giving specific enough alternative recommendations.
* website/formatdate: update example
The given example was showing HOUR:MONTH instead of HOUR:MINUTE
Fixes#22598
* website/import: remove reference to no-longer-working option
Users can no longer supply `-config=""` to tell Terraform not to load
configuration for import.
Fixes#22294
* website/provisioners: `host` is required in connection blocks
Fixes#21877
* website/variables: clarify variable definition precedence
It was not entirely obvious that a variable could not be assigned
multiples times in a single source.
Fixes#21682
* website/backend/local: add `workspace_dir` attribute
Fixes#21391
* website/output: `sensitive` outputs are redacted in output
Fixes#21502
* website/backends: sidebar order tweak
It makes sense for backend 'configuration' to appear before 'init'.
Fixes#13796
* Revert "website/formatdate: update example"
This reverts commit ccd93c86ddd15a21625c0767702ee1cc62e77254.
Reference: https://github.com/hashicorp/terraform/issues/16697
Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862
To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
The Terraform Enterprise brand has now been split into two parts:
- Terraform Cloud is the application that helps teams use Terraform together,
with remote state storage, a shared run environment, etc.
- Terraform Enterprise is the on-premise distribution that lets enterprises run
a private instance of the Terraform Cloud application.
The former TFE docs have been split accordingly.
- Make these descriptions more similar, since they do basically the same thing.
- Add some subheaders to break up the wall of text and make it more skimmable.
- Nudge people more firmly toward `for_each` if they need to actually
incorporate data from a variable into their instances.
- Add version note so you know whether you can use this yet.
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
We added the csvdecode function originally with the intent of it being
used with for_each, but because csvdecode was released first we had a
section in its documentation warning about the downsides of using it with
"count", since that seemed like something people would be likely to try.
With resource "for_each" now merged, we can replace that scary section
with a more positive example of using these two features together.
We still include a paragraph noting that "count" _could_ be used here, but
with a caution against doing so. This is in the hope of helping users
understand the difference between these two patterns and why for_each is
the superior choice for most situations.
Ran into an issue requiring `-var='foo=bar'` over `-var 'foo=bar'`.
Without the `=` the generic response came back:
```
Usage: terraform plan [options] [DIR]
Generates an execution plan for Terraform.
This execution plan can be reviewed...
```
Terraform `0.12.2`
Team tokens never worked with the `atlas` backend, but the `remote` backend
uses them as intended; they can perform plans and applies on workspaces where
the associated team has at least plan or write permissions, respectively.
The search "terraform leading zero" does not find the `format()`
function, which is perfectly capable of adding leading zeros.
Thus I have added this one word to help people find `format()`.
The correct environment variable corresponding to the `ca_file` variable is `CONSUL_CACERT` and not `CONSUL_CAFILE`.
See `backend/remote-state/consul/backend.go` line 77.
This also includes a previously-missing test that verifies the behavior
described here, implemented as a planning context test for consistency
with how the other ignore_changes tests are handled.
* Correct fmt -check
With `-check=false` the exit status is always zero.
With `-check=true` the exit status is zero when all files are properly formatted and non-zero otherwise.
* update fmt documentation to use short form for -diff and -check
We previously had some notes about handling configuration variants just
tacked on to the "dependency inversion" section as an afterthought, but
this idea is a major use-case for dependency inversion so it deserves its
own section and a specific example.
There have been a few questions about this so far which indicated that the
previous docs for this feature were very lacking. This is an attempt to
describe more completely what "any" means, and in particular that it isn't
actually a type at all but rather a placeholder for a type to be selected
dynamically.
Based on some common questions and feedback since the v0.12.0 release,
here we add some small additional content to the documentation for
"dynamic" blocks, covering how to access the keys of the collection being
iterated over and how to fold multiple collections into a single one to
achieve the effect of a nested iteration.
These follow the same principle as jsondecode and jsonencode, but use
YAML instead of JSON.
YAML has a much more complex information model than JSON, so we can only
support a subset of it during decoding, but hopefully the subset supported
here is a useful one.
Because there are many different ways to _generate_ YAML, the yamlencode
function is forced to make some decisions, and those decisions are likely
to affect compatibility with other real-world YAML parsers. Although the
format here is intended to be generic and compatible, we may find that
there are problems with it that'll we'll want to adjust for in a future
release, so yamlencode is therefore marked as experimental for now until
the underlying library is ready to commit to ongoing byte-for-byte
compatibility in serialization.
The main use-case here is met by yamldecode, which will allow reading in
files written in YAML format by humans for use in Terraform modules, in
situations where a higher-level input format than direct Terraform
language declarations is helpful.
This is similar to the function of the same name in Python, generating a
sequence of numbers as a list that can then be used in other
sequence-oriented operations.
The primary use-case for it is to turn a count expressed as a number into
a list of that length, which can then be iterated over or passed to a
collection function to produce that number of something else, as shown
in the example at the end of its documentation page.
Using az login and then terraform init from the command line I got `Error: Either an Access Key / SAS Token or the Resource Group for the Storage Account must be specified`
A longer-form guide will follow in the Sentinel section of the Terraform
Enterprise documentation, once it's ready. For now, this section isn't
saying anything useful since it was always just a stub for a guide we
planned to write later.
The upgrade tool is assuming that a type of "list" means list(string) and
a type of "map" means map(string), because that was what we documented
those as meaning.
In practice, Terraform 0.11 was lacking some validation which allowed
more complex nested structures in some cases even though they were pretty
inconvenient to use due to other language limitations.
The upgrade tool doesn't have enough context to make a reliable decision
on this, so instead we'll rely on the upgrade guide for this. We don't
need a TF-UPGRADE-TODO comment in this case because we reserve those for
things where a subsequent operation might cause the configuration to be
misinterpred, rather than just causing an error. Instead, we'll show an
example of the comment in the upgrade guide so the reader can easily
match it, and give some advice in the guide on how to address it.
We've seen in the past that some users try to use this form with the
ssh:// URL prefix, so we'll mention explicitly that this is invalid and
show a working example of how to use it without the URL scheme prefix.
Our original upgrade guide was drafted while some things were still in
flux and not all of our upgrade tooling was in place yet.
This redraft now attempts to be more specific and direct, showing exact
commands to run and (where relevant) exact error messages that Terraform
might return.
I also took this opportunity for some general copy-editing, though we'll
probably want to do one more pass of that alone (without changing any
content at the same time) before final release.
This new content presumes the existence of a Terraform v0.11.14 release,
which isn't published yet at the time of writing but should be published
before v0.12.0 final, once we've done final verification and review of
the upgrade path including it.
- Note that we intentionally omitted it from the sidebar, to reduce confusion.
- Write a summary up top so you can stop reading sooner if you don't actually need this.
* lang/funcs: testing of functions through the lang package API
The function-specific unit tests do not cover the HCL conversion that happens when the functions are called in a terraform configuration. For e.g., HCL converts sets to lists before passing it to the function. This means that we could not test passing a set in the function _unit_ tests.
This adds a higher-level acceptance test, plus a check that every (pure) function has a test.
* website/docs: update function documentation
There was some leftover v0.11-style interpolation syntax here.
We prefer to use a "naked" expression in situations like this where the result
isn't a string, because interpolations returning non-strings is a common source
of confusion for new users.
The division operator now always performs floating point division, whereas
before it would choose between float and int division based on the types
of its arguments.
We have a specific error message for when a fractional number is used as
an index in HCL, but this additional upgrade guidance provides a specific
solution to the problem: the floor function.
Sadly we don't have enough context in the current design of the upgrade
tool to make this fix automatic. With some refactoring it may be possible
to apply the fix automatically within list brackets, but since that is
a relatively complex change we'll first try this manual solution prompted
by an error message, because in practice so far we've seen this reported
only in the context of list indexing and our error check will catch that
and make the user aware of the need for a fix there.
* funcs/coalesce: return the first non-null, non-empty element from a
sequence.
The go-cty coalesce function, which was originally used here, returns the
first non-null element from a sequence. Terraform 0.11's coalesce,
however, returns the first non-empty string from a list of strings.
This new coalesce function aims to preserve terraform's documented
functionality while adding support for additional argument types. The
tests include those in go-cty and adapted tests from the 0.11 version of
coalesce.
* website/docs: update coalesce function document
Phil Sautter