Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.
The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.
Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.
After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:
- The local cache directory will now be .terraform/providers rather than
.terraform/plugins, because .terraform/plugins is effectively "poisoned"
by the incorrect usage that we can't reliably distinguish from prior
version correct usage.
- The .terraform/plugins directory is now the "legacy cache directory". It
is intentionally _not_ now a filesystem mirror directory, because that
would risk incorrectly interpreting providers automatically installed
by Terraform v0.13 as if they were a local mirror, and thus upgrades
and checksum fetches from the origin registry would be blocked.
- Because of the previous two points, someone who _was_ trying to use the
legacy cache directory as a filesystem mirror would see installation
fail for any providers they manually added to the legacy directory.
To avoid leaving that user stumped as to what went wrong, there's a
heuristic for the case where a non-official provider fails installation
and yet we can see it in the legacy cache directory. If that heuristic
matches then we'll produce a warning message hinting to move the
provider under the terraform.d/plugins directory, which is a _correct_
location for "bundled" provider plugins that belong only to a single
configuration (as opposed to being installed globally on a system).
This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.
The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
The upgrade requirements for this release are considerably more modest
than for Terraform v0.13, so this time we just have some notes about a
few changes in behavior that may be impactful to some users.
This first pass is intended to be included as part of a forthcoming beta
testers' guide as we begin the v0.14 beta testing period. We will make
further changes to this upgrade guide based on feedback from those who
participate in the beta process.
Note that this upgrade guide is not intended as release marketing material
and so its presentation is focused on addressing concerns users might
encounter while upgrading. We'll share highlights from the release in
other contexts, such as the changelog and in the product blog.
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.
In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.
Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
This includes both the main documentation about the lock file itself and
changes to related documentation about Terraform commands that interact
with the lock file.
We will likely continue to update this first pass of documentation as we
get feedback and questions during the prerelease period.
* website: Update all Learn crosslinks
The URL structure on Learn recently changed, so it's time to update some URLs.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Move the information about state from the "caveats" to the main
info section, using similar information to sensitive outputs.
Updates the header of the section from similar inspiration.
We can remove the caveat about changing map elements.
Add a little more text about the intended use case for ignore_changes,
as the common case of fixing erroneous provider behavior should not be
the primary motivation for the maintenance of this feature.
* Put link to tutorial in its own section, call it a tutorial instead of guide, and use new canonical URL.
* Mention limitations of using import with a remote backed
* Typo fix
Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>
This commit adds an `alltrue` function to Terraform configuration. A
reason we might want this function is because it will enable more
powerful custom variable validations. For example:
```hcl
variable "amis" {
type = list(object({
id = string
}))
validation {
condition = (alltrue([
for a in var.amis : length(a.id) > 4 && substr(a.id, 0, 4) == "ami-"
]))
error_message = "The ID of at least one AMI was invalid."
}
}
```
For a credentials helper plugin to be useful with Terraform 0.13+, we
need to cope with the case of having no credentials for a host without
this being an error. This is to allow the public Terraform Registry to
be accessed without supplying a token.
The way to implement this is to respond to queries for credentials for a
host which has no credentials stored with an empty object and a success
exit code. This contradicts the previous documentation, which calls for
an error response in this case.
* Adding not about data-sources and depends-on for 0.12 users
* Bold
* A little more markdown
* A little more markdown for data_sources in 0.12
* Some iteration based on good feedback
In addition to the directories previously listed, Terraform looks in the
CLI config directory ($HOME/.terraform.d/plugins on macOS/Linux/UNIX,
and %APPDATA%/terraform.d/plugins on Windows). List this in the
documentation for clarity.
We also add a note about the working directory relative "vendor"
location, ./terraform.d/plugins.
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
This new option is intended to address the previous inconsistencies where
some older subcommands supported partially changing the target directory
(where Terraform would use the new directory inconsistently) where newer
commands did not support that override at all.
Instead, now Terraform will accept a -chdir command at the start of the
command line (before the subcommand) and will interpret it as a request
to direct all actions that would normally be taken in the current working
directory into the target directory instead. This is similar to options
offered by some other similar tools, such as the -C option in "make".
The new option is only accepted at the start of the command line (before
the subcommand) as a way to reflect that it is a global command (not
specific to a particular subcommand) and that it takes effect _before_
executing the subcommand. This also means it'll be forced to appear before
any other command-specific arguments that take file paths, which hopefully
communicates that those other arguments are interpreted relative to the
overridden path.
As a measure of pragmatism for existing uses, the path.cwd object in
the Terraform language will continue to return the _original_ working
directory (ignoring -chdir), in case that is important in some exceptional
workflows. The path.root object gives the root module directory, which
will always match the overriden working directory unless the user
simultaneously uses one of the legacy directory override arguments, which
is not a pattern we intend to support in the long run.
As a first step down the deprecation path, this commit adjusts the
documentation to de-emphasize the inconsistent old command line arguments,
including specific guidance on what to use instead for the main three
workflow commands, but all of those options remain supported in the same
way as they were before. In a later commit we'll make those arguments
produce a visible deprecation warning in Terraform's output, and then
in an even later commit we'll remove them entirely so that -chdir is the
single supported way to run Terraform from a directory other than the
one containing the root module configuration.
The subtle difference in keywords when creating vs. accessing locals trips
people up, even more than the "variable" vs. "var" distinction. It deserves its
own subheader on the page, plus a nice noisy callout.
I've just wasted an hour to two hours trying to find the problem to finally realize that although I declare a "locals" block, it's referred to as "local". This is pretty weird! So let's be be clear about this.
The error diagnostic shown when legacy state contains resources from
in-house providers has changed, so update references to it in the 0.13
upgrade guide.
We previously had this just stubbed out because it was a stretch goal for
the v0.13.0 release and it ultimately didn't make it in.
Here we fill out the existing stub -- with a minor change to its interface
so it can access credentials -- with a client implementation that is
compatible with the directory structure produced by the
"terraform providers mirror" subcommand, were the result to be published
on a static file server.
- Edits to registry overview
- Add index link as 'overview' (header links are semi-invisible)
- move providers/overview.html to providers/index.html
- Edits to providers overview
- fix filename of os-arch
- edits to provider publishing
Terraform's design assumes that each remote object in Terraform's care is
bound to one resource instance and one alone. If the same object is bound
to multiple instances then confusing behavior will often result, such as
two resource configurations competing to update a single object, or
objects being "left behind" when all existing Terraform deployments are
destroyed.
This assumption was previously only implied, though. This change is an
attempt to be more explicit about it, although these are additions to some
older documentation sections that have not been revised for some time and
so this is just a best effort to make this information discoverable
without getting drawn into a full-on reorganization of these sections.
While revising this there were some particular oddities that I decided to
revise while I was there, but I'll leave a fuller revision of this older
content for a later commit when we have more time to review it in greater
detail.
* Make sidebar nav in language docs more intuitive
* Minor display fixes for registry docs
* Explain providers in the registry in the providers index
* Revise a bunch of language docs around provider reqs
This is mostly an effort to smooth out some of the explanations, make sure
things are presented in a helpful order, make sure terminology lines up, draw
connections between related concepts, make default behavior more apparent, and
the like. It shouldn't include very much new information, but there might be one
or two things that came out of a conversation somewhere.
Co-authored-by: Judith Malnick <judith@hashicorp.com>
As part of documenting the new module for_each capabilities we added a
section noting that shared modules using the legacy pattern of declaring
their own provider configurations would not be compatible with them.
However, that also applies to the new module depends_on and several folks
participating in the beta pointed out that the documentation wasn't
discussing that at all.
In order to generalize the advice, I've moved the old content we had
(since v0.11) recommending against provider configurations in shared
modules out into its own section, now being more explicit that it is
a legacy pattern and not recommended, and then folded the content about
for_each and count, now also including depends_on, into that expanded
section.
As is often the case, that had some knock-on effects on the content on
the rest of this page, so there's some general editing and reorganization
here. In particular, I moved the "Multiple Instances of a Module" section
much further up the page because it's content relevant to users of
shared modules, while the later content on this page is more aimed at
authors of shared modules, including the new section about the legacy
pattern.
Part of the upgrade process happens in the first "terraform apply" after
adding explicit source addresses in the configuration. Previously we just
left that implied under the assumption that everyone would run
"terraform apply" shortly after anyway, but there is a specific tricky
situation where the first change after upgrading is to remove a resource
from the configuration, leaving Terraform unable to complete the upgrade.
Because of that, we'll now explicitly direct users to run
"terraform apply" after upgrading. Along with that, there's a reminder to
make sure that "terraform plan" indicates no changes before upgrading, so
that completing the upgrade doesn't involve also applying changes to
remote objects.
* website: Terraform Registry Provider Publishing
* website: (Registry) remove OS/arch recommendation
Until we have a canonical list to point to
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
Co-authored-by: Paul Tyng <ptyng@hashicorp.com>
* command/init: return an error with invalid -backend-config files
The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.
Fixes#24845
* emphasize backend configuration file in docs
* Azure backend: support snapshots/versioning
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
* Azure backend: Versioning -> Snapshot
Co-authored-by: Reda Ahdjoudj <reda.ahdjoudj@gmail.com>
Co-authored-by: Patrick F. Marques <patrickfmarques@gmail.com>
Martin Atkins